180

I work in a team that writes software to facilitate one of the key business units of the company. I joined the team a few months ago and found out that there is high turn-over in my team due to one person. This person (let's call him Mr. A) has been with the company for 7 years, he is very difficult to work with and he repeatedly makes bad decisions on purpose to keep the software product unstable, difficult to maintain and troubleshoot. This way, when there is an issue, only he can fix it.

He had left the company a few years ago because the company didn't allow him to work from home, but as soon as he left, the company had to hire him back (and allow him to work 100% from home) because the software had issues and no one knew how to fix it.

My manager knows this, but he says there is nothing he can do about Mr. A.

What can I do to fix this situation? I want to make the software modern, maintainable and stable.

FYI, the software monitors events, does some processing on them and then takes appropriate actions. Mr. A has:

  • Purposefully kept away from modern software development frameworks;
  • Written core business logic in languages that can't be tested;
  • Re-architected software components into 30 modules to add complexity and version certification issues, and;
  • Designed it in a non-scalable fashion, ensuring there is no HA (high availability) capabilities.

Update:

Regarding Untestable code, business logic is being moved from Java to groovy scripts embedded in XML. The Java code's unit tests have been discarded.

Regarding modularity/complexity, each module has been given its own git repo and has its own versioning. Now only Mr. A knows which versions are compatible together. You can't release 2.0 version of the product and deploy all 2.0 modules. You have to release Module A 2.0, which will work with Module B 1.0-2.0 and Module C 1.0-1.5. For me, that's bad design, it should all be versioned under one repo like Spring framework (Spring 5.0 means Spring-Core-5.0, Spring-Context-5.0, Spring-Web-5.0, Spring-Security-5.0, etc).

Manager says he can't do anything about this because Mr. A was let go of at first but then when issues popped up he had to be called back to fix it. So the product cannot be maintained without him.

I see this as my problem since I don't want to abandon the manager as he has been very nice to me. And my first instinct is to solve a problem, not abandon it, although I can see how abandoning this would be really easy, and part of me is tempted to do that.

Others have left the team because of him, because at lunch he is what everyone complains about. Any time there is a meeting with Mr. A, people come out of it shaking their heads (for hours).

2nd Update:

HA is abbreviation for High-Availability. In Software Architecture this means developing your software in a way that it can be hosted/deployed in production environment in a redundant fashion so if one instance of it goes down, the other instance(s) can take the load resulting in zero downtime. End user wouldn't even know something had gone wrong.

Regarding: This is seems like normal large code base. I don't think this is because of large code base as the product is not that rich in functionality. It's a backend system that crunches data. Other companies have similar products to meet their business needs, they're able to do this using modern HA/Scalable options, so I don't understand why this team needs to do it in Java 6 without HA/Scalability.

3rd Update:

Regarding 'Do latest versions of all modules work together?': Not necessarily. He's been rolling back certain modules in prod if there is a bug identified, but rolling back has introduced more bugs since certain module versions aren't compatible. All of this could be avoided if all modules were versioned and released together because then the entire product would be tested and as a whole would guarantee certain functionality. In other companies/projects where I've worked, that's how they've been able to develop and deploy far more complex projects with ease.

@pipe: I'm not fresh out of school. I've worked at various companies and on large projects for past 10+ years and everything I see Mr. A propose goes against convention and common sense. The 30 modules (in separate repos) was how he had originally set up the source tree. A smart developer who was in the team for 1 year, saw the compatibility issues and pushed for combining everything to one repo, making a multi-module maven project. That developer got tired of Mr. A's nature so he found a job at one of the top 5 IT companies. I won't name the company to keep this anonymous, but by top 5 IT companies I mean Microsoft, Google, Apple, Facebook, and Amazon. So this developer wasn't dumb nor was he incompetent. He had 8 years or experience. Mr. A reverted that change back to the way it was, 30 modules in separate repos. So these 30 modules weren't added to handle complexity of a large code base. They were put in place to ensure there are compatibility issues in prod. Unnecessary complexity.

More regarding "Why is this your problem?": When I talk to developers who work at (or have friends who work at) Microsoft, Google, Amazon, Facebook, Apple; I'm told that often you find people that are difficult to work with. I see this situation as a challenge that I will face repeatedly wherever I work no matter how awesome the company is. So I need to know how to handle this properly in order to continue to grow in my field.

Goal (for keeping this question on-topic):

I am asking this question to know what is the best way to handle situations like the one described above to meet the goals outlined below. I believe that difficult coworkers are impossible to avoid so based on others' experience maybe we can all learn something.

  • Improve stability of product by minimizing spaghetti code and unnecessary complexity, as per management's request.

  • Make it HA as per management's request.

  • Use modern frameworks and language spec (Java 6 vs Java 8) so new developers are easy to find in the market and they can be productive quicker.

  • Eliminate dependency on single person.

  • 6
    Comments are not for extended discussion; this conversation has been moved to chat. – Masked Man Jun 4 '18 at 11:44
  • 4
    Is it known when the person is going to retire? How old are they, approximately? Such information matters. – Nemo Jun 4 '18 at 11:54
  • 5
    Moderator note: Please use comments only to ask for clarification or suggest improvements to the post. Avoid using comments for off-topic discussions and for posting answers. Any further irrelevant comments will be deleted without notice. – Masked Man Jun 7 '18 at 3:33
  • 10
    Your question makes a lot of assumptions about the internal motivations of Mr. A. Strip out your assumptions about his motivations and focus on the facts only. Instead of saying "He has deliberately done harmful thing X" just state that he has done thing X and it is harmful. Unless you have psychic powers you cannot possibly know that he deliberately introduces bugs or deliberately adds complexity in order to cause problems, and frankly, the odds of that being true are remote. – barbecue Jun 8 '18 at 13:45
  • 1
    I realize advocating paying the ransom, but concomitant with this should be heavy investment in moving to software that can't be used against you. Just bite the bullet and pay for the support contract while you move to other software not written by Red Hat. – L0j1k May 9 at 8:46

19 Answers 19

350

What can I do to fix this situation?

Nothing, you've only been there a few months, it's not your role and you don't have the power to do anything except whine about it.

Your superiors have a lot of recourses, but haven't used them in 7 years, so at this point you're just second guessing their reasons and analysing a colleague instead of concentrating on your own responsibilities and tasks.

189

Hire two or three of the smartest developers that you can find. Hand them all the code. Let them verify that they do indeed have all the code, everything that is needed to run the software. Have them learn what the code does, document it, refactor it, until they reach the point where they can fix problems faster than Mr. A. All this obviously without any knowledge of A.

At that point you make sure that you shut off Mr. A completely from any company resources, transfer the job to your new developers, and inform Mr. A that his employment has finished.

I would think that with Mr A's development methods, the amount of work that his code does is actually a lot less than seven years development would normally produce, and that the code is obfuscated, but not actually difficult, which makes the new guys' job a lot easier.

PS. Due to some comments, I want to emphasize this again: The problem is not just the money, the problem is that the software is much less well developed than it could be, because A focuses on making it hard to develop, not on improving the software.

  • Comments are not for extended discussion; this conversation has been moved to chat. – Masked Man Jun 7 '18 at 1:13
  • 6
    (+1) "The problem is not just the money, ..." Much worse: if the software is a key component of the company workflow, if mr.A left or died, the company would risk a catastrophe. The OP should ask himself a question (well, actually, the management should): "Could the company survive if suddenly the SW wasn't available any longer? And for how long could it survive?" – Lorenzo Donati Jun 8 '18 at 14:43
  • Since A is working 100% from home, setting up a development team unbeknownst to him could indeed be quite easy - easier than if he was on site, in any case. – Law29 Jun 9 '18 at 21:14
  • 3
    @jennifer or not - if the entire team shares in their disdain for this character, getting to watch his replacement team grow would probably lead to a perverted morale boost – Our_Benefactors Jun 10 '18 at 12:12
  • 3
    You can't find three smart developers to figure out his code. They are too smart to take that job. Instead hire them and let them write a replacement. This is one of the few cases where a big bang rewrite is justified. – Ben Mz Jun 11 '18 at 16:34
139

Short Answer: Your organisation currently is in grave danger of The Bus Factor.

This is why you never let one person hold all the knowledge. It is a huge risk. What would happen if this person decided to quit, or literally got hit by a bus? If the situation is as you outline it, then the whole company is gone. You need to flag this to your managers as a organisational risk, not an HR issue.

Start getting other people across the code, with or without Mr A. Preferably without, because you've already identified him as the issue.

Remember, if Mr A won't assist in this risk mitigation to your organisation, then he himself is a danger to the organisation and needs to be managed out. Take away his power so that if he really does get hit by a bus that you don't all end up with no way forward.

  • 2
    @Dan Exactly, and management needs to be made aware of the risk that places on their organisation. As in my answer, this is not an HR issue, it is a risk mitigation issue. – Jane S Jun 4 '18 at 22:06
  • 3
    @jane He had been fired before and management hired him back. – gnasher729 Jun 5 '18 at 13:45
  • 26
    @user2023861 "Bus Factor" is a well established industry term. – Jane S Jun 5 '18 at 21:50
  • 7
    By firing him initially, and by rehiring him management fully understand the risk that Mr A creates. But what is foolish is that management is at fault because they have no plan to minimize his impact. Sure, there is a crisis situation that needs "whatever it takes", but you plan to prevent it from happening again. This is the perfect example of the phrase "Fool me once, shame on you; fool me twice, shame on me"... meanwhile Mr A keeps laughing. There are dark clouds on the horizon and this is a %$&* storm waiting to happen. – Phil M Jun 6 '18 at 23:26
  • 3
    @user2023861 If the person wins the lottery, you might be able to somehow persuade them to help you a bit. It might be very expensive, but it’s conceivably possible. If the person is struck and killed, it is literally impossible for you to get any further information out of them. Anything that wasn’t shared or written down is gone forever and you’ll have to make do without. – cpast Jun 10 '18 at 7:01
94

The other answers are pretty great, but there's one additional thing I'd consider:

My personal advice would be to consider starting to look for other places to work... if management hasn't taken any action in 7 years, I'm not sure this is a place you'd like to work for in the long term. To me, this reads as a warning sign for other sorts of problems in the company.

  • 4
    It's impossible to know from the outside, but is it possible this employee is holding the company hostage in some other way? It sounds a bit unreal that they would bury their head in the sand for so long. – l0b0 Jun 4 '18 at 8:23
  • 2
    @l0b0 - It's possible they've been deceived and kept in the dark. But ultimately, management either wants to fix this issue, or they want things to continue. If they want it to continue, then it's the OP's decision if they want to work at such a place or not. – Wesley Long Jun 4 '18 at 18:07
  • 17
    Every question on this site can be answered „start looking for another job”. – Mateusz Stefek Jun 4 '18 at 19:30
  • 1
    When I entered my career (long time ago), I was told #1 rule is "No one is indispensable" in any business. I agree with this answer because the management responsible for the mess described in the question did not follow this rule for 7 years. Therefore, I think this answer is an option for the OP. – scaaahu Jun 6 '18 at 7:04
63

and he repeated makes bad decisions on purpose to keep the software product unstable, difficult to maintain and troubleshoot

So why is your team letting these "bad decisions" get past design review or code review? If you don't have a code review process or even a design review process, advocate for that in the long run.

But until then, everything you need to know is in the Joel on Software blog article: Getting Things Done When You’re Only a Grunt

And he has a specific callout for dealing with bozos: FILE BUGS. Per Spolsky:

As a grunt, your goal is damage-minimization, a.k.a. containment. At some point, one of these geniuses will spend two weeks writing a bit of code that is so unbelievably bad that it can never work. You’re tempted to spend the fifteen minutes that it takes to rewrite the thing correctly from scratch. Resist the temptation. You’ve got a perfect opportunity to neutralize this moron for several months. Just keep reporting bugs against their code. They will have no choice but to keep slogging away at it for months until you can’t find any more bugs. Those are months in which they can’t do any damage anywhere else.

Otherwise, as a new hire on the team, your goal should be to develop your own reputation of excellence.

  • 1
    Depending on how "hands-on" the manager is with code review, the OP could also mention to the manager about how many times Mr A's code gets rejected, as long as it isn't said in a critical manner. Making it a question can help the manager "think of it on their own", which can lead to reasons the manager gets rid of Mr. A without the "fault" falling on the OP. Ex.: "I've been looking at Mr. A's pull requests to further my knowledge in the app. Is it normal for code reviews to have so many rejects before getting deployed/accepted?" – computercarguy Jun 4 '18 at 19:16
  • 1
    The team hasn't questioned him so far because till now he had people with 0-3 years of experience. Now there is me and another guy who just joined, we have 10 years or so of experience. When we question him, he just says that's how he wants it, or he becomes condescending, .e.g "Why do you want to make the git history a mess?" He said that when someone refused to alter git commits instead of committing multiple times like normal people do. He wants devs to alter git commits. – MightyPizza Jun 5 '18 at 23:40
  • 1
    @Rob The problem I have with his approach is that he is asking developers to do a commit for a change, then push it to remote, then continue working on the same change, amend local commit, and then force push to remote (because now regular push will fail due to previous push). To me, that is wrong. Doing a force push as part of regular coding seems bad. The squash part, I agree with. I don't have any problems doing that. It's the amending of commits and force pushes and I'm not happy with. Thoughts? – MightyPizza Jun 6 '18 at 1:42
  • 1
    @Rob Also, each commit is going to be atomic, for me a commit should be done when a meaningful change has been completed, and source tree can still be compiled. It doesn't mean entire change is done for that ticket/branch. Additional commits would be normal, e.g. 1: modified module A to add feature X with unit tests, 2: modified module B to add feature X with unit tests, 3: Fixed issue with unit tests in module A for feature X – MightyPizza Jun 6 '18 at 1:45
  • 2
    on force push: git fetch; git rebase. work on your branch, squash the commit and push. that single commit is accepted or rejected as a whole, because you aren't presenting a string of commits that depend on each other. if you fetched and rebased, your pushes will fastforward 100% of the time. you will never ever see a conflict on merge. if somebody pushed right after you rebased, fetch/rebase and push again. you will almost never experience merge headaches if you work this way. only force push would be your branch that is up for review when rejected. – Rob Jun 6 '18 at 2:10
44

Just to say it more clearly than the current answers...

Your problem:

no one knew how to fix it.

Your solution:

  • Learn how to fix it yourself.

There, now the company doesn't need the other guy anymore.

  • 10
    This has an interesting dynamic. By taking on the task of understanding the obfuscated crap, That Guy is no longer the only person with this knowledge. Explaining to the manager that you're doing this (probably on your own time initially) specifically to spread the institutional knowledge is displaying initiative, good sense and ultimately provides an opportunity for you to replace this individual who's been providing a toxic development environment for years. And if they don't like you doing this, quit immediately. It's clearly a bad environment – Ruadhan2300 Jun 4 '18 at 10:57
  • 7
    I like this answer because this is the only way to drive such software into a brighter future. The company has honestly learned just two things over the years: 1) Mr. A has built software which keeps things moving. 2) Countless newbies have left because they are not good enough. – MonkeyZeus Jun 4 '18 at 12:04
  • 2
    This is the answer. If they hired the guy back, then that app is a cash cow. If nobody else can work on it, then they can't also claim to know a better way to do it. They need to actually successfully make a better system to know that. – Rob Jun 6 '18 at 0:49
  • Unless learning it (*cough* deobfuscating it) requires a bigger investment than keeping him hired for now... When the system usefulness eventually dies, hopefully management will have learned the lesson and already has a proper team documenting and producing readable code in the new system... – CPHPython Jun 8 '18 at 14:19
36

Post this on the wall: (You will have to change some of the names, and places).

Some years ago I spent a week giving an in-house program design course at a manufacturing company in the mid-west of the United States. On the Friday afternoon it was all over. The DP Manager, who had arranged the course and was paying for it out of his budget, asked me into his office. “What do you think?” he asked. He was asking me to tell him my impressions of his operation and his staff. “Pretty good,” I said. “You've got some good people there.” Program design courses are hard work; I was very tired; and staff evaluation consultancy is charged extra. Anyway, I knew he really wanted to tell me his own thoughts.

“What did you think of Fred?” he asked. We all think Fred is brilliant.” “He's very clever,” I said. “He's not very enthusiastic about methods, but he knows a lot about programming.” “Yes,” said the DP Manager. He swivelled round in his chair to face a huge flowchart stuck to the wall: about five large sheets of line printer paper, maybe two hundred symbols, hundreds of connecting lines. “Fred did that. It's the build-up of gross pay for our weekly payroll. No one else except Fred understands it.” His voice dropped to a reverent hush. “Fred tells me that he's not sure he understands it himself.”

“Terrific,” I mumbled respectfully. I got the picture clearly. Fred as Frankenstein, Fred the brilliant creator of the uncontrollable monster flowchart. “But what about Jane?” I said. “I thought Jane was very good. She picked up the program design ideas very fast.”

“Yes,” said the DP Manager. “Jane came to us with a great reputation. We thought she was going to be as brilliant as Fred. But she hasn't really proved herself yet. We've given her a few problems that we thought were going to be really tough, but when she finished it turned out they weren't really difficult at all. Most of them turned out pretty simple. She hasn't really proved herself yet --- if you see what I mean?”

“I saw what he meant.”

— Extract from the book Software Requirements & Specifications — Michel Jackson (Well worth a read).

  • 11
    I understand what the author was saying. Jane is almost certainly the truly brilliant one but in typical manager fashion; the manager thinks the person who creates the indecipherable monstrosity is terrific, after all they understand such a complex problem. When the reality is that truly great developers have a knack for turning the complex into relatively simple solutions. My initial comment was just offering a suggestion to improve your answer because I like your answer but I think many people will miss the point without your explicitly stating the point. – Dunk Jun 4 '18 at 22:23
  • 11
    Wow! This really applies to my team. My manager told me today he values Mr. A because when things break in prod, he fixes them and saves the day. What my manager didn't care about was why things were breaking in prod in the first place. – MightyPizza Jun 6 '18 at 23:07
  • 3
    @MightyPizza From all the discussions and what you said I almost think Mr.A could be someone (I may be wrong) that could dare putting some bugs in the code purposefully in order to fix them later, just to keep the company hostage. If this is true, this might be a crime in some countries. If you ever suspect this and have evidence, I would consider talking with your manager about that. Of course this is risky business, because it seems Mr.A is a smart fellow, and if he really did something like that it would be hard to prove. I can understand why many people left the company. – Lorenzo Donati Jun 8 '18 at 15:21
  • 2
    @MightyPizza never put down to maliciousness, what can more easily be attributed to stupidity. In my experience it was not a conscious decision. Often it is just following the reward (this is what he has been rewarded for in the past; This is what his employer values.). – ctrl-alt-delor Jun 8 '18 at 15:33
  • 1
    Thanks for this answer. I remember the story, but didn't remember which book it came from. – gnasher729 Jun 9 '18 at 22:29
27

The answer is simple: fire him. You may have to pay out a non-trivial amount of money in the short term to fix the mess he's made, but Mr. A isn't smarter than anyone else in the world - somebody else will be able to maintain the software in the short term, and make it better in the long term.

  • 1
    Indeed. And it's a short-term investment that would ultimately have massive long-term benefits, potentially even including financial (high turnover is expensive!) – cale_b Jun 4 '18 at 17:18
  • 9
    Downvoted for the simple reason that no, even the smartest guy in the world is not able to penetrate several years of work in a few months, especially if the person in charge seems to have a vested interest to obfuscate code, building traps or use hidden compiler tools. The guy was already fired! First, really smart guys are not growing on trees, you need to identify them and they are very expensive. Second, as gnasher729 pointed out, have a look first by experts how bad the situation really is without letting A know and decide only then (!) what can be done. – Thorsten S. Jun 4 '18 at 19:10
  • 1
    If you fire him, the company shows up on 6 o'clock news as badly affecting 100+ nation-wide businesses. – MightyPizza Jun 5 '18 at 23:29
  • 1
    @MightyPizza - same situation if he decides to leave or gets hit by a bus. It's an untenable situation they need to deal with – NKCampbell Jun 5 '18 at 23:57
  • 1
    @ThorstenS. Yep! I agree! I once had to improve a piece of SW written by a computational linguist which was to leave the company for a position in an university abroad. The guy was a sort of "genius" in his field, but quite a poor programmer (well, I'd say a poor SW engineer). The SW was a ~10kLOC+ monster PERL quasi-monolithic script! It worked, but it was a mess (and he wasn't trying to obfuscate anything). After 1 month of analysis and feedback from him, after I wrote down the specification of what the SW did, I decided to rewrite it completely in Java! ... – Lorenzo Donati Jun 8 '18 at 15:05
24

There is a perspective to consider.

The company likes it this way. If they didn't 7 years is a long time to let it persist.

We tend to forget as developers, that it's not our job to write good or smart code. It's our job to poof a product into existence. Writing good code just makes that process better, but you can a totally awesome product with totally crap code.

The company has stood behind his decisions, and even hired him back. They "like" him and his way of doing things. You're not likely to change this, even if you somehow managed to get him fired. What is likely to happen is they pick a new person to be "the guy" and the process starts all over again.

It's not your job to run the company. The company has made their choice. You need to make yours. Learn from the guy (he has held the same job for 7 years he must be doing something right) or move on.

  • 3
    "...but you can a totally awesome product with totally crap code." It won't stay awesome for long. – jpmc26 Jun 4 '18 at 9:46
  • The company still will have to face the issue with the bus factor in the long run. Should Mr. A disappear for good next time, the company might not be ready to deal with the ramifications of that event. It is possible though that the company is already planning to do something, or a least set up a fund to be able to deal with the eventual fallout of this situation. – MauganRa Jun 4 '18 at 12:18
  • @jpmc26 if the only definition of "awesome" is "profitable", it MIGHT stay awesome for long enough, especially if a product/service exists alone in a market with high barriers to entry. – kmarsh Jun 4 '18 at 13:38
  • 4
    @jpmc26 in my experience, poor quality code, has very little to do with how well a product is received. As an example, I give you Civ 5. You could actually see the horrid nested giant for loop as the map rendered. Yet it was still profitable. – coteyr Jun 4 '18 at 17:16
  • 4
    AFAIK most of the major banks are still running on mainframes with 30+ year old COBOL handling accounts etc. -- I would say that multi-billion dollar profit reports are pretty awesome? – jkf Jun 4 '18 at 20:03
12

For better or worse, neither nothing nor no one is irreplaceable. (some may be more than others, but still ) If people know the solution, they can start reverse engineering it.

I was in your shoes more than a couple of times in the past. In the most similar situation to yours, "Mr. A" was long gone, however, I had a monolithic solution working for the back end of a cable company where I had no source code for the local developed libraries, and to top if off I was a rookie in that industry.

I pretty much attacked it on a modular approach, having a look at the existing code, reverse engineering where it lacked and writing modules that replaced on turn with better functionality and speed core tasks. I perfected each module before going to the next. In a few months I had killed the former application.

Being irreplaceable is overstated.

Dealing also with legacy stuff is not an easy task, maybe your coworker does it out of inertia or because he does not know any better.

  • Well it's always hard when managers are pressing hard for delivering new features. Devs know that what's really needed is to stop and refactor the codebase. But how to convince the manager? – ferit Jun 5 '18 at 19:49
  • Sometimes, and in the example, above, with regular crashes each closing trimester, there are not really alternatives. It was definitively for the better. – Rui F Ribeiro Jun 5 '18 at 21:03
10

From a business perspective, there are basically two solutions:

  1. Transition incrementally, in other words, take a small piece of the functionality, re-implement it to a high standard, and put it in production, and continue doing this piece by piece until the old system can be totally decommissioned.
  2. Fully build a new system, and then transition to it, either all at once or gradually, e.g. starting new clients on it and gradually moving old clients to it. The new system doesn't have to be as featureful as the old one initially; its selling point can be a lower price, quicker support, etc.

The advantages of option 1 are that it does not need a large amount of upfront investment, your new code gets field-tested gradually instead of all at once, and you start to see benefits early (because the business spends less time and money maintaining the parts of the old system that are no longer active).

However, the disadvantages are that the new system's structure may be strongly influenced by the old system's structure, and in some cases it's really difficult to replace parts of a very messy system. Sometimes a bit of creativity is necessary—if all else fails, your new code can simulate key presses and button clicks on the old system! But interfacing with the old system may generate so much work that it's better to just go with option 2.

Either way, there is something that can be done. The question is, how much will it cost and how much will it save? Trying to estimate that for each of the above options will help determine which option to choose (if any). It will depend on the functional requirements, the structure of the existing system, and the willingness of the business to spend money/use credit upfront for future savings.

  • I wouldn't worry too much about your first disadvantage. Due to Conway's Law, the application is likely already built to reflect the company's structure, whether Mr. A intended it that way or not. The OP's solution likely would, too, even built from scratch. Of course, if it reflects his team (of 1) structure, it might compile to a monolith, in which case option 1 might be an untenable approach, anyway. And all that said, this is still something the OP can't do on his own, at least not through a direct approach. He needs management buy-in. – Bloodgain Jun 5 '18 at 22:54
4

As a member of the software team your job is not to manage the company and its employees.

Your job is to write good software. So worry about the software not the people.

You see the problems with the software, and from your question, it sounds like you have some ideas to fix the problems. Bring these ideas to your manager and fight for them.

"Manger, this software is not tested and its current implementation is not testable. I suggest we work on reimplementing in a language that is more testable, using design patterns that allow for easier testing."

Now, it's very likely that:

A. This is a large undertaking that the company is not willing to invest in, because they don't see the need

B. Mr. A will argue against this stuff and will be listened to since he is more senior

If that happens then you tried to do your job well and were stifled. Time to look for a new job.

If this pans out and they listen to you, you signed up for a ton of work.

  • You are either a junior, or have never worked with such people. – BЈовић Jun 6 '18 at 6:49
  • 1
    @BЈовић Your opinion would be a lot more meaningful --- to me, at least --- if you shared something specific to back it up. Opinions without facts are not actionable. – employee-X Jun 7 '18 at 3:04
  • 1
    @employee-X I was unfortunate enough to actually work with a person similar to the person described in the question. He was always the smartest, everyone else stupid. He was also irreplaceable, cause had to maintain his ball of mud. My manager also said he couldn't do anything. But instead of going along, rest of team made this guy hell. It was so good when he left :) – BЈовић Jun 7 '18 at 5:50
4

You can't assume intention behind the situation.

Purposefully kept away from modern software development frameworks

He might just be most comfortable with what he knows best?

I worked at big companies whose pipeline was a patchwork of ancient and brand new code and where certain pieces of code "just worked" and were never touched. On top the programmers who wrote it were long gone and nobody really understood how they worked, so they just added new functions to accommodate for changing requirements. Their pipeline was always live, so any major rollout could have disastrous(i.e. very expensive work and delivery interruption) consequences, so they shied away from touching it too much.

It is bad habit and makes for unoptimized infrastructure but actually has its merit.

What you could do, especially if you're to work on a lot or all of the code: If there is no proper documentation, either propose to create one (if you know you're qualified and have the time) or ask if there should one be made to speed up the work.

You should (as you already do) talk about adapting new technologies or improvements with your manager but expect that nothing might come of it, even if he agrees with you.

Chances are that nobody higher up sees the benefit of spending resources on this and your attempt might be looked as ah, the new blood thinks can change the world and teach us about the error of our ways.

Unfortunately company structures resist sudden change and modernizations are a matter of gradually wearing down the resistance or implementing it bit by bit unless you can show your "revolutionary changes will bring upon a golden age of financial profit without risks".

Even if he is maliciously doing this to keep his job, I don't see it being your responsibility to blow the lid off, especially if your immediate superior is informed about it.

What would you do? talk with your manager or without him to higher management or with the colleague in question?

Seeing as your manager decided not to pursue the matter he probably won't want to talk to his supervisors with or without you, or he already did and met a blockade there.

If you talk to his supervisors without him you risk alienating him.

Talking to the colleague will most certainly make for a very bad work environment afterwards and he surely will deny any accusation.

3

The only way to fix this is to take away Mr. A control.

First get management approval.

Make a brand new git.

  1. Import a good, agreed upon starting point.

  2. Start from there, and bring the code forward.

  3. Don't give Mr. A access at all(don't even tell him it exists)

  4. Port Mr A. changes to your own repo, or rewrite.

  5. Let Mr A. Do whatever he wants to his repo, as you won't be using it.

  6. Make fake code changes to make it seem active if necessary to hide your new repo from him.

  7. Eventually code will diverge far enough he will become automatically obsolete.

You need to make a critical decision to keep or abandon the modules strategy. Modules are not necessarily a bad them, you need to keep them sync'ed and in working order.

This is going to be a lot of work as you have to re-write a lot of existing code.

An additional benefit, is the code will eventually diverge far enough that he will no longer be able to claim its his code. It will be completely foreign to him.

  • The words "code freeze" leap to mind. If Mr. A keeps introducing new changes then your project will always be behind. To make it possible to replace it management need to tell Mr. A to freeze his code beyond minimal and urgent bugs fixes. That's the missing "point zero" in your list. Then when your project is ready to roll out you can replace Mr. A's existing code base entirely. – StephenG Jun 6 '18 at 22:58
  • 4
    Also note one thing : if Mr. A thinks his control is being taken away and is as devious as described, Mr. A will simply resign or threaten to. This plan needs a contingency. Again, code freeze and minimal bug fix until a replacement is ready is required. – StephenG Jun 6 '18 at 23:00
  • 1
    @cybernard: "good, agreed-upon starting point". The problem is, Mr A has had time and experience making sure no one else knows what this starting point would be and even to make sure no such point exists, if he would not want it to exist. – mathreadler Jun 9 '18 at 11:36
  • 1
    @mathreadler Then you just have to pick a point and deal with. Start with yesterdays code. Your only other option is to start over from scratch. Either option will be brutal but then, long term, it will be fixed. – cybernard Jun 9 '18 at 13:56
  • 1
    @mathreadler Any audit to monitor the changes Mr. A is making to see if he isn't adding code to allow these "urgent" situations to occur? Again this is where a brand new repo comes in. He can't sabotage it any further, and you can start to unwind the spaghetti. I personally think Mr. A. has been planning months in advanced so he can watch you spin your wheels and laugh all the way to the bank. You should have let "smart developer" take over, fired Mr. A, and picked up the pieces afterwards. I bet your problems would have been solved by now. Otherwise your a permanent hostage. – cybernard Jun 9 '18 at 14:56
2

@MightyPizza - You are wasting your time. Go work for another company that has a brighter and better future. You should only spend this much energy on solving this problem if this company is your last stop.

This is the absolute best answer you are going to get. Prep yourself for the jump.

"I want to make the software modern, maintainable and stable."

The best way to do this is on Day One, first line of code, not with someone else's hot pile of garbage.

  • 12
    If the best way is on Day One, then your software will probably stop being " modern, maintainable and stable" starting on Day Two. – Erik Jun 4 '18 at 5:23
  • 3
    @mark you're right it's easier but we all have legacy cod. To rewrite it? It may cost a HUGE amount of money (without an immediate/reasonably short term ROI) and and even bigger amount of time (which you may not have). You may also need more resources than you have. Final result? A product which is less mature, less stable and possibly less feature-rich than the existing one. Sorry but to rewrite isn't, more often than not, an option. – Adriano Repetti Jun 4 '18 at 6:55
  • 4
    Oh and to handle such pile of legacy s** shaping a new modern product, step by step, without interruptions, rewriting when/what required and refactoring all the way is...well, what every (professional) developer does every single day (even with his own legacy code he wrote just two years before) – Adriano Repetti Jun 4 '18 at 7:04
  • 3
    That reminds me of: Things You Should Never Do ... – Daniel Jun 4 '18 at 10:13
  • 5
    The downvotes (which I did not) are probably in part due to this fairly arrogant statement: This is the absolute best answer you are going go get.... it's not constructive, and it diminishes the efforts of the other answers here.... – cale_b Jun 4 '18 at 17:22
2

Be aware that as a non-manager, this does not HAVE to be your problem to solve (apart from following instructions to do things that have been decided on as part of a solution).

Also, never blindly interpret "We want problem XY solved" into a statement "...because of problem XY".

Also be aware that once you drive any initiative, even with approval, to change the situation, you are involving yourself in company politics. In company politics, there is no such thing as "an innocent innovator with no personal agenda, that we can't hold responsible for unintended consequences".

If you do it, do it with a personal agenda, and own the consequences either way.

1

For him to have stayed in the position for so long shows the business is consciously or unconsciously applying Hanlon's razor:

Never attribute to malice that which is adequately explained by stupidity.

It sounds like it's very difficult to find actual cases of malice, just things that have worked out not so well over time. In fact, it looks like Mr A has let people try to change things and then heroically fixed them when they have gone wrong.

You should also consider that Mr A may be suffering from the Dunning–Kruger effect, in that he is unable to see that what he is doing is wrong.

It could be that he was once good, but stopped learning and stopped trying. Who knows. What you have to take as a first principle, is that no matter how exasperating he is, you have to deal with him as though he is not a malicious agent.

Your question seems to take the assumption that he is just after job security and refuses to update things to embed this. You may be right, but you can't deal with him on this basis.

I'm sure some of your bright ideas come up with the resistance of "we tried it once and it didn't work due to X." From a business perspective this makes sense, they took a risk something didn't work so why should they try it again.

You state as your goal.

I want to make the software modern, maintainable and stable.

Lets break this down by business need

1. Modern

There is no implicit business value in modern. It is arguable in any case. You say Java 8, others would say Rust or Golag are modern. So long as a system is supported there is no great value in updating it.

2. Maintainable

This could be difficult to argue as well. Mr A will argue that it is maintainable, as he maintains it. You would have to argue that the maintenance cost would go down drastically to justify a wholesale change of framework.

3. Stable

You say that you want to make the system HA, but you also say that it

the software monitors events, does some processing on them and then takes appropriate actions.

Which does not sound like it needs HA.

So what to do?

Your system sounds like a big ball of mud defined by

A Big Ball of Mud is a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair.

Tackling this head on, with an uncooperative developer managing this will be very frustrating, and probably fruitless. It sounds like many others have tried and failed.

What you can do is start to circumvent it:

  • If you can't make the system HA, then use a message bus system or such like to capture the messages that way if the system goes down you don't lose data.
  • If you can't write unit tests, write system tests. These are not as neat but they will allow for testing to be done on the system
  • The next time some action is required start to build that into an independent subsystem which can listen to the message bus and perform it's own action.
  • If you can start to affect some downstream effects (for example encapsulating the actions which are taken) then put these in a subsystem.

There is some argument to support the majectic monolith over microservices, but it assumes that the monolith is well engineered. In your case, the system may be fine as a well-engineered monolith but if it is not well engineered you need to split things out.

This looks like a good article to start you with:

A more common approach is to start with a monolith and gradually peel off microservices at the edges. Such an approach can leave a substantial monolith at the heart of the microservices architecture, but with most new development occurring in the microservices while the monolith is relatively quiescent.

However it is important to play along with Mr A, do not tell him you are replacing the system, but that you are "improving reliability" or adding redundancy. If you attack what he has done he will make it harder for you to achieve your goals.

You may get flak that you are making the system more complex. It is true, but necessary if you are to move forward long term.

1

Why does everyone want to fire Mr. A so much? I think he did nothing wrong here. Software development is not about making new things using cool tools, or a framework. It is about making a stable thing that just "works." Your company loves Mr. A and is satisfied with his work.

Purposefully kept away from modern software development frameworks

Your system is current in stable condition, so why do the team even need to use modern software development framework like Scrum or agile?

Written core business logic in languages that can't be tested

Is there a requirement that the core business logic has to be automatically tested?

Re-architected software components into 30 modules to add complexity and version certification issues

How much more did this implementation cost your company?

Designed it in a non-scalable fashion, ensuring there is no HA (high availability) capabilities

Still the same question - how much more did this implementation cost your company?

In my opinion, all the things you write down here is just your complaint against him. If he did a lot of "terrible" things the system itself would have melted down in the first year, and he couldn't have had stayed for that long in your company.

  • Is there any requirement that the core business logic has to be automatically test Yes. Always. It verifies any changes in code don't break existing use cases. It verifies the code works as intended and fails as intended. It's your roadmap on the trip to chaos that is enterprise software development. – rath Jun 25 '18 at 9:38
  • (...cont) The question makes it clear there are version incompatibilities often deployed in prod, and that no one know what versions are compatible with each other. Unit testing would make that easily discoverable – rath Jun 25 '18 at 10:08
  • 1
    The important thing is the question didn't give me a clear number about "how much damage have Mr.A done". All the details in this question can only lead to "Mr A slowed down the development process". If MrA's work satisfied all the cost, time, quality requirement then it's fine. – gachiemchiep Jun 26 '18 at 0:10
  • 1
    Erm.... I'm not sure if you read the entire question. The main issue here is that the software is highly unstable. We get continuous outages, he has to be called to save the day every time. New hires can make sense if the software due to the non standard why things have been implemented so they leave after 6-12 months. It is not scalable either, it's slow, requires a lot more memory than it should. Using modern frameworks would've have resolved all these issues. – MightyPizza Jun 26 '18 at 9:29
  • 1
    I understand that Mr.A did make your system with very low quality standard. But my point is: even when Mr.A work's quality is very bad, if he DO NOT create bugs which cost your company tons of money to fix then it's totally fine. All the manager out there would love to add more server or Ram if that could solve their problem. So i don't think the "require more ram to work" is problem at all, sorry about that. – gachiemchiep Jun 27 '18 at 0:24
0

Simple, the business has to offer him a generous raise to document the current state of software and all the knowledge that's not being documented then they fire him. Or alternatively, they hire a tech consulting firm to document everything and he'll see the writing on the wall and eventually leave.

protected by Masked Man Jun 4 '18 at 9:34

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.