Background: I am working on a legacy code base. It is huge. I did not write it. I have done a lot of work on it, but more work remains. There are still unknown parts in there for me.
Scenario that happened recently:
- Sales requested a change to a product line. Namely, update percentage figures based on new coefficients. Engineering gave me a cheat sheet with what percentages should be, and what the coefficients are.
- Assuming that my work will be to replace coefficients and update percentage, I thought I will do just that.
- I estimated 1-2 days to complete that, thinking that it's 2-4 hours work, but knowing that something usually slows it down.
- It took me 2 weeks, plus there is more work to be done, because knowing of my initial estimate I took a lot of shortcuts that are still there to be cleaned up
- There is probably at least a week of work to be done. Maybe 2, to get things where they should be. So much for "2-4 hours of work" as originally thought.
What happened?
Scope just kept increasing. I didn't know this, but I kept learning of this as I did more work. I didn't know it, Sales didn't know it. Engineering didn't know it. Code was written by someone else. Engineering didn't know of different algorithms that had to be replaced. Sales didn't know code. I mean in retrospect I could have realized that "both algorithms needed to be replaced", but I couldn't have, since "replacing coefficients" would work as a drop-in replacement, without having to touch anything else, I thought.
It was not possible to simply update coefficients as I originally thought, because new coefficients had entirely new algorithm attached to them, incompatible with old ones. There were two algorithms in the code using those coefficients. One simple that always assumed percentage at 100% and thus a few shortcuts could be taken and formulas simplified, and one that varied percentage - no shortcuts would work, you'd have to write the complex formulas. Engineering originally gave me the cheat sheet for 100%, and we both did not consider the hard complete version. That slowed me down a lot.
So main things that slowed me down:
- not realizing that it's not a drop-in replacement of coefficients. I need to update the algorithm
- Wow, this algorithm is HUGE and I keep learning that there is more and more to it (cheat sheet, then excel sheet, then multiple excel sheets, new formulas, etc)
- once I did, I started updating one algorithm, without realizing there is another one
- once I realized there was another algorithm, made shortcuts for it to work as well
- after updating percentage and coefficients everything else kind of needed to be updated as well. Other formulas, other variables (essentially what I refer to as "the algorithm")
Basically, scope creep and I don't see reasonable ways to uncover all this without a round-table discussion of what the change actually is, and insight into codebase and the project. I guess if someone had enough insight to reasonable foresee all this, they should really be paid big money for their talent. With hindsight 20/20 I can make up an imaginary scenario where there is an estimation meeting and people go
- "what are the changes",
- "what are the complete changes, in the worst case that have to be done" (i.e. rewritten)
- "are there more places where this change needs to be done",
- "what is the hardest thing that needs to be done"
etc. Maybe in this case at least in part, this scope creep could be mitigated.
This happens a lot to me with this particular code base and this particular project. How can I mitigate this or how can I estimate better? Or how can I communicate better that scope keeps increasing as I keep learning of new things. How do I communicate with bosses when they try to hold me to original estimates ?