How might I avoid significantly overestimating the time required to deliver a potential client's software project?

Question

Spoke with a potential client about delivering a computer vision app. I estimated this would take one month, erring on the side of a conservative estimate to account for any complexities that might unexpectedly bubble up. However, I didn't realize just how conservative it was compared to another iOS engineer's estimate of 60-80 hours when he told me. He also said that a number of his contacts agreed that 60-80 hours sounds right.

How might I avoid such a discrepancy of 2-3x in the future, and potentially navigate situations like this?

Answer 1

I estimated this would take one month,

At this point into your text I was fully prepared to tell you how ridiculously short that timeframe is to develop an app on your own that does this. But it seems it was good I read your post till the end.

He also said that a number of his contacts agreed that 60-80 hours sounds right.

Well, then let them do it on that estimate. When the 60-80 hour app isn't done in a few months and they are still paying by the hour, maybe they will remember you.

Your estimate was your estimate. If you feel like it was the truth then it doesn't matter what other people estimate. My former boss used to say "I could do this in VB over the weekend". Until one meeting I just got up and said "that's great, that way the development team can work on that other issue, just send it over on monday". I don't think he mentioned that stupid line ever again.

The only point you need to care about: could you have delivered on your estimate. Don't listen to other people. If other people estimate less, they should do it for/in less. You will find the willingness to estimate shorter will go up in smoke once you tell them that they will actually have to do it, not just talk about it.

So if someone says "but your competitor will do it for half", just say "that's great, I'm happy for you. Let me know if you have anything else you need help with."

Answer 2

1 month is about 150 hours, so you've estimated twice what another developer estimated. That is within the normal margin of error for software projects.

Unless you both independently develop this project you have no idea whose estimate will prove to be correct.

You can get a more accurate estimate by developing a more detailed spec, and then estimating the effort required for each part of the spec. However, this is likely to take a significant amount of effort in itself.

Answer 3

I didn't realize just how conservative it was compared to another iOS engineer's estimate of 60-80 hours when he told me

It's a common tactic to try and get estimates lessened to say someone else said it would take less. Yet they never seem to hire the person who said so. Because quite frequently they just pulled that estimate out of their backside or asked someone who pulled it from a similar place.

If you have the experience and knowledge to be sure of your estimates, stick to them. Otherwise you look greedy or incompetent.

potentially navigate situations like this?

I would wish them good luck with their project and move forwards from their response if any.

Answer 4

You can't avoid over-estimating or under-estimating the work, because there is no right answer. Your estimate must take into account a lot of variables that you can't know in advance, and other quotes may include assumptions that you aren't comfortable with, or have a really low price because someone is desperate to get the work.

The key is to break the project down into sections, and assign time to each section. Then if the customer challenges your timings, you can justify the value, showing the elements that your competitors may have omitted. It also provides a framework for a bit of horse-trading, for example "As you can see, I've allocated 5 days for acceptance testing, but if you think that is too long, I can reduce it". Then if the testing takes more than the allotted time, you have some leverage to get a price increase.

The customer should get the impression that you have a standard system for calculating time & costs, and any variations you offer are carefully thought through. However, the reality is that most software project estimates include quite a bit of guesswork, so there is no right or wrong answer.

Answer 5

To avoid overestimation, here are my suggestions:

1. Get input from experienced software engineers before giving the customers the estimated costs

   If you work in a company with many experienced software engineers, then before you give the estimates to the customers, you can contact these experienced software engineers who have worked on similar projects to get some inputs or general guidelines.

2. Who will actually work on the projects: experienced engineers or beginners ?

   Please keep in mind that these experienced software engineers likely to have significant skills and therefore, it may take them much less time to complete the projects than the beginners.

   So, you will have to take into account whether the experienced engineers or beginners will actually write code for the project to get a better time estimate.

3. The hiring costs : experienced engineers vs beginners:

   You can also inform the customers about the costs. There is a tradeoff.

   It will cost more to hire an experienced engineer to complete a project in a short amount of time. The project may have less bugs and the design may look better.

   It may cost less to hire a beginner who may take longer to complete a project.

Answer 6

If you can properly justify your estimate with sound reasoning, and show this to your client(s), then this is what matters most.

What follows is a particular experience I had which, I hope, will help you understand the statement I just made - which is my answer to your question.

A few years ago, at a former employer, I had a team "lead", with a background in mechanical engineering, not software engineering, pitch his "solution" to upper management about how a problem could be solved with a "Simple System"™. The Simple System™ had to manage user accounts, track hardware inventory, notify specific users when certain component quantities were below a user-given threshold, allow users to place orders for more parts, produce reports, etc. - you know, the works. The estimate that he came up with for his "Simple System"™ was 3 months and I would be the one to do it. I said I thought it'd take longer than that.

I was basically asked to do a full project estimate - waterfall style[0]. So, this is what I did:

1. I spent about 2 weeks (that I could've spent working on the "Simple System"™) breaking down high-level feature requests all the way down to stories (with point estimates) and associated tasks.
2. I looked at my individual velocity per iteration in unrelated projects and used that as the basis for my assumed velocity on this estimate.
3. I did not estimate time - I estimated effort based on task size and then derived a time estimate from it (using velocity points per iteration, etc)
4. Documented a margin of error for the estimate. (This came from empirical data in the book Agile Estimation and Planning; it shows an error margin between -40% and +60% relative to the original estimation value.)

I explicitly documented the following assumptions:

1. estimates are estimates, not commitments;
2. waterfall-style estimates (which they asked for) are less reliable, need to be revisited at least every 6 months, and should be considered completely invalid if at least that much time has passed without a revision;
3. my degree of familiarity with the technologies that would be used in the project (i.e., I didn't have to spend time learning them from scratch).
4. I'm the only dev working on the system;
5. I have other responsibilities that will sometimes require that I stop working on "Simple System"™ (i.e., I'm not working on "Simple System"™ 100% of my time without interruptions)
6. That velocity-based estimates were based on unrelated projects and may be different in practice. (In reality, velocity is a measure for a team, not individuals)
7. The estimate was only as good as the documented assumptions - break the assumptions, and you invalidate the estimate.

Long story short, my estimate came to 15 months to implement all the features, with the -40%+60% margin being down to 9 months (best case) and up to 24 months (worst case). (NOTE: This was 15 months to complete all features, not 15 months before we could have a portion of the system up and running in production while making incremental improvements.) They didn't like my estimate, so what was their solution? They delegated the project to an inexperienced intern and invalidated the whole estimate instead. After all, the intern was expected to be around for up to a year and they knew for a Fact™ that building Simple System™ wasn't going to take more than 3 months, so they could go overtime 4x over without it being a problem, right? GENIUS! (No, I'm not joking.)

The first Git commit in the Simple System™ repo was April 29th, 2014. For whatever reason, my brain committed that snapshot into its own memory and remembers. I had made that commit.

Well, fast forward 7 months and it seems reality is starting to rear its ugly head back at them. My manager, the very person who made the decision to hand it off to an intern, made a comment saying "Well, no one's talking about the Simple System™ project estimates anymore..."[1]. I just gave her my I-tried-to-tell-you-shrug and kept going. It was their own self-inflicted problem, not mine.

Fast forward again. It's been a year. The intern left for another team. They brought a new inexperienced intern to work on the previous intern's codebase[2].

Fast forward yet again. It's been at least 1.5 years since work on Simple System™ had started. It's June 30th, 2016 and it's my last day at the company. Simple System™ still had zero working releases in production - not even a partial one.

A friend close to the project within the same team at the company told me it took them about 7-9 more months to make a first release - and there were still lots of missing features.

If you're doing the work, then it's your estimate, based on your knowledge, experience, and ability. If you can show sound reasoning to your clients on why you're estimating something in a particular way, then that may or may not help. You could explain that you're trying to account for some unforeseen events and that, if things go smoothly, you'd expect things to take less time overall, and so on.

Keep in mind that some tend to provide their "best-case" estimates, assuming nothing goes terribly wrong - since it might be unlikely - and then, as soon as something worth mentioning does happen, they inform the client of the delay and adjust the estimate. At the end of the day, client assumes that particular risk.

Also, keep in mind that you have no idea about how others have justified their own estimates - if at all? Perhaps they've solved that kind of problem so many times they already have enough experience to know how much work that is "by heart". Perhaps they have more people. Perhaps they're full of crap. It's up to the client to weigh in those factors.

---

[0] I argued against this, but it was a waste of time.

[1] While this is speculation, I took this to mean that Mr "lead" and Mrs manager had been, basically, trashing my original estimate behind my back in their pRoGrEsS mEeTiNgS, and how I (in their view) had no idea what I was saying/doing/etc, prior to that point or whatever. (Note that no one ever bothered to challenge the reasoning and justifications behind my estimate, but that's probably beside the point. As far as I could tell, they just "felt" Simple System™ was Simple Enough™ to take 3 months because they kept referring to Simple System™ as "simple".)

[2] When I say inexperienced, neither knew Python, SQL for databases, how client-server systems worked, Django, Git, Apache, etc. They were computer engineering majors, not computer science majors, so I don't blame them.

Answer 7

Adding this as a new answer, though it sort of just refutes the original assumption. You said,

How might I avoid such a discrepancy of 2-3x in the future, and potentially navigate situations like this?

How do you know you have a discrepancy? You are not the same as your developer buddies. If someone asks me how long it will take to do a project, and the project is highly similar to one that I already have a code-base to handle, then the project could conceivably be done in days, not weeks or months. If you asked me how long to do a project that isn't similar to other work I've done, then it might take me a month while others could do it in half a month.

Having a large code-base to pull code snippets from is half of the benefit of having experience. The other half is in knowing pitfalls before you hit them, so that you can discuss them with the client before it causes your development to grind to a halt.

Answer 8

Software always takes longer than estimated. I recommend an estimating technique an experienced IT person explained to me a long time ago: (1) calculate your best estimate based on your experience and carefully researching the problem, (2) double your estimate, (3) present it with plus/minus 25% to cover contingencies. For example, if after careful consideration you think a project will take 100 hours, predict 200 hours plus/minus 50 hours. If someone estimates less, wish them the best and move on. And if your estimate is accepted, always announce as soon as possible when the schedule has to change because something new (typically a spec change) has been introduced.

Answer 9

I think the big problem is that they appear to be tying hours of work directly to days in a week which isn't accurate.

However, I didn't realize just how conservative it was compared to another iOS engineer's estimate of 60-80 hours when he told me. He also said that a number of his contacts agreed that 60-80 hours sounds right.

If you break that down it would be 7.5 to 10 days of work. However that would only be the case if you are working on this 8 hours a day with nothing else involved. If you factor in things such as emails or meetings with the client the amount of time you can focus on the work decreases. If you factor in any other work that might come up during that time it will reduce the work you can get done even further.

When doing an estimate you shouldn't just focus on the amount of time it will take to do the code but also how much time you will be able to devote to it on a given day. From my perspective saying 60-80 hours of work and 1 month of time to deliver the product isn't that far off as you can easily spend those hours over 3 or 4 weeks instead of the 2 that the hours alone would suggest.

Not to mention a common thing that non developers tend to forget is that 2 different people can write the same program in a vastly different length of time based on their experience with what they are doing.

Answer 10

However, I didn't realize just how conservative it was compared to another iOS engineer's estimate of 60-80 hours when he told me.

I'm sorry, but this question sounds naive. Imagine that you're about to run a marathon for the very first time, and you think you can complete it in 5 hours. You don't suddenly change your estimate because you hear someone else can complete it in 2 hours. That would be insane.

He also said that a number of his contacts agreed that 60-80 hours sounds right.

Next time, ask questions. Ask to see the paper trail. If there was no paper trail to speak of, and no written estimate, assume that the spec was full of holes and that the calculation was for a barebones prototype.

In other words, you should be multiplying that estimate by a factor of 9 (according to Fred Brooks) instead of reducing it.

But most importantly, you should be asking many-many questions. How come was the other estimate so low? Were they going to use a commercial SDK? Is a commercial SDK allowed? Is there a licensing budget for that? Did they do that kind of project before? Does the company have a reputation for delivering on their estimates? How many other projects have they done for them?

And if you still don't understand why their estimate was so low, assume the worst. Because whatever magic sauce they do have, you don't have it yourself. After all, not everyone can run a marathon in two hours, and that's ok.