How can I convince my manager that we need to reduce technical debt? [closed]

Question

I work as a developer on a 15-year-old software product, which has been continuously under development for all that time. The code base is enormous. We have over 500mb of code, which I suspect is multiple millions of lines of code. That is not including images or the database with over 500 tables. The demand to expand the code is high. Customers constantly want to see new features. That is not good for the technical debt, of course. The programmers would often joke that something that would take an hour to create in a normal code base takes a week in ours. A new programmer is expected to take over a year to get familiar with a module. Upper management wants us to improve the performance of the software and has instructed lower management to do so, but this is not directly visible in the ticket system.

I work on one of the development teams, with four other programmers and a manager. Three of us believe lots refactoring is necessary, at least in the parts that are used often and developed on often. One of the other programmers, who has been working on the program since the dawn of it and the manager are opposed to it. They are opposed to it for different reasons, though. The manager does not believe code-refactoring or otherwise reducing technical debt helps customers or our team's numbers (which may influence his paycheck), while the other programmer believes much of the code is already perfect and variables and functions are evil and should be avoided. The other programmer is not senior to the rest of us, but has been working here longest. There is one senior developer working on another module, not managed by this manager, who we may be able to recruit to our cause.

Over the years, customer expectations have changed too. While they were fine with a module taking 5 minutes to start 15 years ago, they are not happy with that time right now. I think a lot can be gained. I've spent an hour refactoring a single function that I was expanding anyway a while back, which reduced the loading time of one process from 10 minutes to 10 seconds. Other teams do work on reducing the technical debt, which increases performance, reduces bugs and increases the ability to expand. Our team is the only team that doesn't. We are the 'best' team on the ticket dashboard, as we have most tickets resolved, yet to upper management, performance is the most important issue at the moment. Part of their solution is moving the code to a newer language, so it can actually be (unit) tested, as lots of the old code has never been tested and is failing with customers upgrading to 64 bit systems.

How can I (and the other two programmers) convince our manager (and the other programmer) that we need to work on technical debt?

I'd like to emphasise that upper management does believe that we need to work on performance, but has not published certain targets. My manager on the other hand focuses on numbers directly visible on the ticket dashboard. Performance is mainly bad because queries are all over the place, often being executed many more times than necessary. I've suggested to tackle queries and move them to seperate functions or classes, but the main issue is the other developer not believing in classes or functions and the manager trusting him.

Answer 1

The situation you are in is not unique, and it's not unreasonable. It's actually pretty standard in production-based software development. There are a few points to take great consideration of:

1. Technical debt is immeasurable. You can't examine a function and "quantify" how much technical debt there is, or how much money it doesn't get you. You can try, but in the end the numbers won't make sense, or even matter. Much like you can't look at a function and tell "how many bugs" there are (which is a literal question I've been asked recently). The reason businesses prioritize "features over fixes" is because customers tell them "we'll start using you instead of the other people if you add such-and-such." Technical debt doesn't do anything to help that, so you need to find a way to sell it.
2. Businesses run on revenue. Being in IT / software dev we often misunderstand or forget this, but a business needs to make money to survive. That means, it has to find a way to cut costs, or raise profits. A large refactoring (where you stop working on new features) does not do that. What you need to do, is prove that you can refactor and save or increase profits.
3. Refactoring (thus, removing debt) is part of the regular process anyway. When working on a function, you should always be refactoring and reducing the technical debt there as you are in there anyway, not as an after-thought. The problem is that no one did such things when building it the first time, so you're what you might call "between a rock and a hard place"—as a result, you think the only option is to do a large refactor, but it isn't the only one. Anytime you touch a function you should do a little refactoring. It's just part of the development process.

---

So, what can you do?

I'm going to give you advice based on my experience, which allowed me to convince a financial services firm to give me the entirety of Q3 (July, Aug, Sept) to refactor our web application. We're dealing with a 20-year-old application, that has gigs and gigs of code (most recent line-count estimates are in the 200+ million). I convinced them to give me three months (let the other teams do their thing, but give mine 3 months) to rebuild what's necessary. It wasn't easy, and yours won't be, but it taught me a few valuable lessons.

First off, as mentioned in another answer, ROI analysis. Come up with ways to estimate and prove what advantages we might come across by doing dedicated refactoring periods. They don't have to be huge, if you can show that 1 hour of development time can shave 9 minutes off a load every time it's loaded, then that's a good start, especially if it's a high-traffic function / module / whatever. (I.e., use that function you refactored already as a Proof-of-Concept.)

Second, reword your request so that you indicate the business advantages. I hate to say it, but if you're not working for a company that is one of the "startups" as-of-late, your boss won't care how you feel about it. Your boss will care how to make money from it. Find out (maybe from Sales, Customer Service, etc.) what load-times are bugging the customers, and describe how we can fix it to drive customer satisfaction up. (Not sure how specialized your software is, but some of the software I build is specific to our industry, so "word-of-mouth" is huge—thus, if you can make your customers like it better, maybe they'll tell their friends.)

Third, don't "complain" about the current status-quo. Don't "whine", don't "get upset", speak factually. Your managers are much more likely to be receptive if you speak to them without emotion, and just lay down facts in a positive way. Not "this code is crap", but, "here's where we can gain some tactical advantages." Don't talk about how so-and-so writes bad code. Don't talk about how so-and-so is opposed to the idea. Talk about the advantages. Keep the conversation positive. (I know how tough that is, but it's often a good thing to "bite your tongue".)

Fourth, speak to the developer-time that will be saved, and how that can be applied to other projects / issues. What does your backlog look like? Is it constantly growing? Are you pushing things off for more-and-more time? Speak to how this can (and will) affect that. If the business gets value from new features, talk about how this will give you time to create the new features more quickly. Talk about how this will speed-up developers.

Fifth, and this may sound like me being a "jerk", but don't talk about how "it will make life easier on the developers."¹ I hate to be the bearer of bad-news, but your managers don't often care how the developers feel. We're a "production" role in most organizations, that is, we do things strictly to make money, similar to warehouse workers for logistics companies. Unless, as mentioned before, you're working for a company that is radically different, management won't often put too much effort into "developer happiness". Drop that thought entirely, and talk about what and why the business gains from your idea.

And finally, understand that, if this is an old application and remodels are already in-the-works, the best option might be to just "suck it up" and hope things get better. If they're working on a new version already, your team won't get the resources or time you want. They're looking at your team as a "triage" team. You exist to make sure the current one stays standing long enough to get the new one built. Yeah, it sucks, but it's how things work. I've been on many triage teams, and most of them have pretty low happiness and high-turnaround, but it's life. The business doesn't care, because your job is to keep the current one running as cheaply as possible. It's sad but almost advantageous to have high-turnaround, because that means salaries are much lower.

¹: some areas of the world are currently in periods of "developer shortages", meaning they do not have enough spare developers to fill the current roles, not to mention new ones. In some of these cases the companies are taking developer satisfaction seriously. That has not been my experience, and I am writing this answer from my experience. So, your mileage may vary (sometimes substantially) on this part of the topic.

Answer 2

Of course, there are pros and cons.

You view this as technical debt because there's lots of code and lots of data.

On the other side of the coin, it doesn't look as though there's anything necessarily wrong with the system the way that it works right now. It's doing the job that it's supposed to - supporting the business in creating profit.

As has been pointed out, putting a program in place to refactor the code in itself is creating technical debt in terms of analysis, refactoring, retesting, and redeploying a system that already works. The larger the system being reworked, the more risk there is to the business.

The compromise is to refactor only the parts that are actively being worked on, within fairly tight boundaries - so the analysis and testing are constrained within each project.

Otherwise, you're going tie people into man-years worth of effort that the business doesn't really see the benefit in.

I know this isn't the answer that you're asking for, but it's being realistic and this (I feel) makes economic sense.

Right now, it appears as though there are some work-streams that are reacting to customer demand in terms of performance. You've not had any of the same kinds of requests come your way yet, so all you can really do is wait and make your assigned work-tasks as efficient as they can be.

You could run your own analysis and propose areas to improve, but this can only be approved by managers according to what priorities the business has for the system as a whole.

Answer 3

Short answer: you can't do it.

Long answer: the management is interested in business value, you can't sell tech debt. However, you can sell reduction of QA costs through test automation or modularization when this leads to each module sold separately. Just speak their language!

Note to you: a manager is a professional lier. What to you is a 15 year pile of garbage, he presents as as a 15 year track record of industry standard quality delivery. When a manager say something vague like "believe that we need to work on performance" its just a BS to make you happy and conformant, when a manager needs smth done, the is a due date and a person responsible for delivery. You say, your team is "the best", that mean that any change into your teams workflow is against company's interest and will be fought against vigorously and since it influences managers paycheck you would be considered not only the enemy of the company but also personal enemy with dare consequences.

It's worth noting that management aren't interested if you do small pieces or refactoring when fixing bugs and adding new features. They're interested in KPI, that they show quarterly or annually in their presentations.

So if you want to make your life as a developer easy by improving the codebase, just do it secretly with out notifying the management.

Answer 4

I can speak from experience on this.

Years ago, I was the lead programmer on a well-known mainframe product (Comparex). The code in the base program was tangled spaghetti, and any attempt to modify it would break three other things.

Release cycles were about a year apart. There was a big backlog of requested changes, but only a few were getting done each year. So I went through the entire program and converted it to structured format.

This was a big job. I actually had to develop tools to analyze the code and help document which routines called which. I finally eliminated all but 3 nonstructured branches.

Now most of the development cycle was gone by the time I finished this. So I then implemented some changes from the request queue, and they went in so fast I was able to do a "normal" amount of upgrades that year, despite spending so much time on restructuring the program.

The next year, I was still able to implement changes quickly, and I completely eliminated the request backlog.

Answer 5

One solution is that technical debt tickets should also be on the dashboard. Another solution is that performance should be a top-level metric that all teams are measured against.

I do agree that refactoring is of limited use and probably actively dangerous if you don't have unit testing. Adding unit tests could be easier to sell because that increases reliability for everybody.

But in general, if this is the attitude of your manager, that will be very slow to change and the process will likely be frustrating. Maybe you would do well to switch teams?

Answer 6

You need to write up the business case and show the Return on Investment (ROI); most business decisions boil down to money. You need to show that, by refactoring, it will speed up development by XX% forever in the future or it removes the need for an additional developer or both, but it will decrease productivity XX% for XX months. If you can do this, they might get on board with refactoring. BUT,

1. Without unit tests, it becomes difficult to not break existing functionality.
2. Refactoring better deliver on the improved productivity/quality

Never forget that the purpose of software development at almost all companies is to generate revenue or reduce expenses. Make sure your proposal covers those two points and it shouldn't be terribly hard to make a case. BUT, your management, may still not want to do it if they are concerned that the projections are significantly off as that could cost them their job.

Answer 7

We're all in that boat to some degree. Complaining is unlikely to help--you'll just be categorized as someone who doesn't understand the need to get things done quickly and your advice will be ignored.

The thing is, most programmers don't think they have "Permission" to do a fix correctly if it isn't the most direct way to solve the problem (for instance, build a tool to check database consistency or edit system data) Anything that takes time repeatedly and can be automated should be automated--you're a programmer, right?

You not only have permission to do things right but are required to, this is the nature of your job and your manager may not be able to understand the factors, but if you just take the time to do it right and don't complain they probably won't bother you. If managers or stubborn programmers can't understand this approach and actively fight you, start looking for another job.

You can also help your teammates understand that they can implement changes the correct way. It can become a culture with subtle but continual pushing.

When it comes to the bigger fixes like a 5 minute startup time, submit that as a bug and then when you work on the bug, refactor the code to improve it while you are in there.

Over time you should become the team that delivers solutions fastest (If not, you aren't doing the right refactors, as you said a 1 week job should take a couple hours once you've cleaned up a bit) and then you will get more leeway and the ability to make more sweeping changes.

I recommend not ever saying we have to re-write everything. Huge projects like that nearly always go much longer than estimated and/or fail, and they can cost quite a few jobs or the entire company when they do.

Answer 8

Simple answer, you cannot abandon and re-write such a large codebase from scratch; that would be a suicidal business decision.

What you can do however is come up with a plan for writing better code moving forward.

You and the other developers need to come up with a plan of coding structure which all of you will abide by.

You need to envision this unicorn-utopian environment and strive towards it with every ticket that comes your way.

Yes, it will need to co-exist with the current "abomination" but as you are given tasks and tickets then keep working them into the utopia.

Once the new code is verified and working to spec then you can work on removing obsolete pieces of code.

Think of it this way; when a client hires a painter to paint a door the client does not care if the painter uses a 2-inch brush, a 3-inch brush, or a small roller and brush combo. If the paint makes its way onto the door then it's a success.

Answer 9

Adding one more argument to prepare your pitch.

There is a question on this stack How to attract people to work on very old and outdated technologies? asked very recently. I would suggest that maybe not now but without changing the processes the company may in 5 years find itself in a place when they will have to pay a large salaries to people who understand their code and are willing to even look at it.

If a new developer needs a year to familiarize himself with the code base, it means that should 10% of the staff quits, the company will need few months to find replacements and a year to make them as productive as the lost staff. That could be even a year and a half of 10% lost of productiveness. If the company is not big and have currently 40 devs working on the code that 10% is merely 4 people.

Answer 10

I'm thinking this can be a slow process. Ask if new projects can be written with unit testing, and modernization, and explain that it will ensure functionality. Eventually management will see that it is beneficial since codes can be changed easily and might suggest other parts of the application to be brought "up to date."

Right now, you're fighting complacency. The software is working and development takes weeks but it gets done. Anyone can argue that software won't last forever, so the non working 64 bit can be easily dismissed and "work in progress." If you can show that instead of weeks it can be done in days or that you can easily port code for 64 bit, then it'll be helpful.

Answer 11

1. Build a "business case" - a proof that reducing technical debt will bring a $$$ value to your organization. Use explicit data points to prove your case.
2. Communicate the proposal. Make your message short, managers have no time for "TL;DR" texts. If you could pitch your idea in 5 seconds, and if it makes sense and is aligned with organizational goals -- manager will ask for details.

I hope this helps.

Answer 12

My recommendation: Whenever you need to fix a bug, or add a feature, and you are working in some area of the code, you leave that area in a better state than it is now. If you are spending a lot more time for any change than you should because of the poor quality of your code, then doing this doesn't add noticable time to your work and helps the next time in that area.

And I would give your boss one bit of advice: If your code doesn't run on a 64-bit system, then you are in for a lot of pain soon. Here's what happened in my area: For the last two years you couldn't publish iOS apps that didn't support 64 bits. If your users use iOS 11 (the current version for a few months), 32 bit apps don't run. MacOS is slightly behind, but users get warnings right now (which will get you a lot of expensive support calls), and in the next MacOS release 32 bit support will be gone.

Windows is a bit slower, but there are very sound technical reasons for disallowing 32 bit apps. And when that happens, and you don't have a working app, your boss is in trouble (you are not, because they will need you). Try telling your customers they can't upgrade to Windows 12. Worse, they can't replace broken computers when new ones are shipped with Windows 12.