It was fixed-price contract to develop Python scraper to scrape 2 websites.

I always charge ~$200 per website. We agreed $200 per website. I agreed $200 blindly without analyzing those websites because I was busy in other things.

Usually it takes a full day or upto 3 days to write and test code to entirely scrape a website.

Today, I studied those 2 websites in question and found a trick to scrape entire website using a single link, in short it will take me max 30 minutes to write scraper for both websites.

Should I still charge my client $400 in total? Or tell him that I figured a loop-hole that make it easy for me.

  • 1
    Comments are not for extended discussion; this conversation has been moved to chat. – Lilienthal Aug 21 '17 at 6:47
  • 12
    "in short it will take me max 30 minutes" - it sounds like you're not finished yet; what if there are unexpected surprises when scraping the Web sites? One estimate is that debugging is twice as much effort, so an adjusted estimate should be 2-3 hours. – Brandin Aug 22 '17 at 8:02
  • 3
  • 1
    I think you know you should charge him the agreed amount, why wouldn't you? – this.lau_ Aug 22 '17 at 8:08
  • 2
    @Fattie I'd say that it depends on the location. – Federico Aug 23 '17 at 13:30

13 Answers 13

up vote 452 down vote accepted

You and your client agreed on a per-site rate. If it had turned out to be way harder, would you have been allowed to charge $1000? I expect not.

I faced this in real life when my staff would insist if we estimated 10 and they got it done in 8, we should only bill 8, otherwise we were "stealing" from the client. Yet when we estimated 10 and took 15, we should still only bill 10, otherwise we were stealing.

You can see how this does not ever lead to a profit for the company. You can also see where rules like "don't let developers discuss billing with the clients" come from.

Charge as agreed and enjoy your bingo. They don't come along very often.

  • 46
    If the website proved to be way harder to scrape, he would never pay me extra money. THANKS @KateGregory – Umair Aug 18 '17 at 17:20
  • 32
    Your real-life example is actually a bit of a distraction/side-issue, since that deals with something that seems to be based on billable hours vs estimated billable, while, as the rest of your answer points out, they agreed on a per-job/site rate. – PoloHoleSet Aug 18 '17 at 17:29
  • 35
    I disagree. We handle it this way: We estimate times. If we do it in time, we charge, what it took. If we need more time, we talk to the customer when we reach 75% and no end in sight. Estimates are no fix prices. But I totally agree with Kates point: You talked about a fix price, not estimates, therefore charge 400$ :-) – Fabian Blechschmidt Aug 18 '17 at 21:38
  • 27
    "If it had turned out to be way harder, would you have been allowed to charge $1000? I expect not." That is why i have accepted your answer... – Umair Aug 19 '17 at 7:14
  • 20
    Consider this - they're paying for your expertise, not your time. Your time is just a way for you to measure the cost of your expertise. Your expertise managed to get a $200 job done in 30 minutes instead of 3 days, but that's because you understood how to get it done in such a short time, where someone with less expertise wouldn't. So you deserve that $200. – Zibbobz Aug 21 '17 at 13:35

You agreed to perform a task for a fixed amount. You fulfilled your obligation.

I once worked with a group of top-notch people who could fix things in minutes when it would take others days to do the same thing.

I would always tell them that "It's easy for YOU not them, if it were, they wouldn't bring their machines in here."

Or, as the old joke goes,

A client calls up a consultant to ask him to locate a problem they had. He says that he will do so for $50,000. After about 15 minutes on the site, he walks to a particular machine, takes out a piece of chalk, draws an X* on it, says "replace this part". submits his bill, and leaves.

The client is unhappy and demands an itemized bill, so he sends them this:

Time on site: 15 minutes at rate of 400/hr----100.00

1 piece of chalk---------------------------------------- 0.25

knowing where to put the "X"----------------49,899.75

That's what paying by the job means

  • 4
    that's not a joke, that's a true story about Charles Proteus Steinmetz. – ed.hank Aug 18 '17 at 23:13
  • 33
    @ed.hank not quite snopes.com/business/genius/where.asp – Richard U Aug 19 '17 at 13:18
  • 2
    shame on the smithsonian then, that was where i read about it - smithsonianmag.com/history/… – ed.hank Aug 19 '17 at 13:47
  • 3
    @ed.hank yes, shame on them. No date, no time, no citation, nothing to back it up whatsoever. – Richard U Aug 19 '17 at 14:11
  • 25
    He draws an X, says "replace this part", then leaves. That's kinda important. Otherwise I doubt the machine was fixed by his drawing an X on it. – immibis Aug 21 '17 at 0:54

You have answered your question yourself.

I always charge $200 per website. We agreed $200 per website.

The client agreed to pay you $200 for getting the job done, regardless of whether you use any "clever" tricks. Just charge the agreed amount and be done with it.

Your question reminds me of this story,

One day a woman spotted Picasso in the market and pulled out a piece of paper. "Mr. Picasso," she said excitedly. "I’m a big fan. Please, could you do a little drawing for me?" Picasso happily complied and quickly etched out a piece of art for her on the paper provided. He then smiled as he handed it back to her. "That will be a million dollars," "But Mr. Picasso, "the woman replied, flustered.”It only took you thirty seconds to do this little masterpiece." "My good woman," Picasso laughed, "It took me thirty years to do that masterpiece in thirty seconds."

Source

"found a trick to scrape entire website"

Finding the trick is your skill, so i think you deserve that money.

  • 13
    (Except Picasso didn't use Dollars) – deviantfan Aug 20 '17 at 4:33
  • 1
    Your story absolutely demands this counter-argument: youtu.be/Dw5kme5Q_Yo – Radu Murzea Aug 21 '17 at 14:20
  • 2
    @deviantfan, and except he didn't set the price in advance and get agreement on it. And except the story probably never happened with Picasso. – Wildcard Aug 23 '17 at 6:23

Your agreement with them was what's known as a firm-fixed-price (FFP) agreement. You accepted all the risk. They could have counter-offered with what's known as a cost-plus-fixed-fee (CPFF) agreement, if they wanted to accept the risks. If they think you charged too much you can point them to any number of websites that show how uncertain software cost estimates are. You might tell them that you'll give them some price consideration on future jobs. FFP and CPFF contracts are discussed in great detail on the web.

  • This is spot on. The contract specifies who assumes the risk, which is you here. Unless there is something special, that party gets the benefit if the risk does not occur. Your pricing should cover the case where the sites are harder to scrape than you expect. Hold the "outrageous profit" for the time when the risk does occur and you take a loss on the job. – Ross Millikan Aug 19 '17 at 3:41
  • 1
    Thank you - not sure why this isn't higher. Fixed price work is common, though most trades aren't a fan - for the customer it gives them a fixed budget which they can get sign off and agreement on. For the person delivering, they take almost all the risk on the project being as expected - most customers realise this and understand that there will be some padding in the quote and sometimes they'll 'lose out'. OP has an extreme example, but ultimately the work is the work. – Dan Aug 19 '17 at 11:21
  • 2
    No such thing as "outrageous profit" as long as all parties enter voluntarily. – Matthew Whited Aug 21 '17 at 14:42

Contract was to develop Python scraper to scrape 2 websites.

I think you have already answered your question. You had a task, you quoted a price, they agreed. You did it.

Now, you have a 2nd issue, you may want future business if it is available. If so, them maybe you consider offering a discount on future work because this job proved more straight forward than expected. I personally would only consider that if you think the chances of additional contracts exist, or referrals, or if you think the trick you did here can be duplicated in the future. Offering such a discount is tricky though as yes, this job proved easier than expected, but the next might be more difficult and you will not be paid extra on that one. You should price yourself on average jobs and the easy ones balance the tough ones.

You are absolutely entitled to charge the full amount but it is worth considering a discount if you believe it will benefit client relations for future work.

You quoted a fixed amount, your client agreed, and you delivered the work. This means you are completely at liberty to charge the full amount with a clear conscience. But, even so, you may find it better to cut your client a break in order to build a good relationship for future work. Ultimately, your value as a contractor depends on how prospective customers perceive your value and so you may consider that giving them a reduced rate on this case is worth it for a bump in perceived value for future work.

You should always deliver what you promise to. That means that where it takes longer to build something that you promise for a flat rate, you charge what you promise.

This means that when you find a more efficient way to do something, charge the same and consider spending some time providing additional value for the customer.

It is also wise to wait the time that you expected to finish as much of consulting is anchored by the maxim VALUE PERCEIVED IS VALUE RECEIVED.

Unless you are positively amazing, you will find that this model averages out in the end.

You gave an honest estimate based on your experience which took a long time to get.

Quotes are fixed but estimates can be changed by agreement, however you should charge what was agreed, especially as you had to understand the situation to spot the shortcut. It's not everyone who will see that shortcut.

Accept it - the next job you may not be as accurate- also your clients pay you for results unless you charge yourself out by the hour...

There are two considerations here which lead to "yes" and one which leads to "no".

First, you are not charging your client for your insight, but you don't account for the fact that your insight is all your are really saying. Otherwise, you are just charging them for typing. And they wouldn't pay you to type a scraper. They pay you 99% of what they pay you because you have spent time to acquire expertise to scrape and 1% to type.

Second, they are paying for the result and not for the work done. If the result is worth $200 dollars to them, that makes your time more valuable. It doesn't make the result of your work less valuable to them.

Third, if you appear to be "nice" this may invite future business. But don't sell yourself too short. If you got a windfall and you want to share it with the client, give them a discount (and tell them you figured out a way to make it happen faster). Charge them $150 if you want, but don't drop it to $25 or so. If you discount too much this time, they'll expect much cheaper price next time. If you admit to a windfall and share it, they'll expect quality work next time. If you really have a lot of clients, I don't need to tell you which one would work out better for you.

This is only a matter of the contract/agreement between you and your client.

If you contracted to charge per hour to an estimated amount of, say, 80 hours, do not charge more than 80 hours even if it takes 2 months (and in such a case you might want to have some real good explanation why your initial estimate proved to be so much off...) and if you find a trick to resolve it in 30 minutes, round it up to that 1 hour and bill that.

If you contracted 200 USD a piece, charge 200 USD a piece regardless of the time spent (again unless it proves inordinately long as above).

However as indicated in one of the answers above, if you find solution in 30 minutes where you expected it to take up to 6 days, you may consider offering the client a discount if you believe it will bring you additional business (from them, from the word of mouth, whatever) but that is not a question of billing/finance or professionalism/ethics but a question of your marketing.

Somewhere along the way you probably mentioned to the client that scraping both sites would take X hours (or days, or whatever), and the client also blindly accepted your judgement and professionalism. If it took you a LOT less (it's a huge deviation you're talking about, not just a few hours), then you should charge less. If instead of taking 1~2 days for both sites you ended up realizing that it would take you 1 month (another huge deviation) would you still charge $200?

  • 2
    As the OP mentions, the already had an agreement that specified that price. Your answer assumes that you know beforehand the total time the project will take. – DarkCygnus Aug 21 '17 at 15:29
  • @GrayCygnus, of course it was an estimation, but the deviation is just too big to be ignored. And the agreement was established on information (that it usually takes 1 to 3 days per site) that turned out to be erroneous. If your mechanic says it will need 3 days to fix your car and you find out he did all the work in 30 minutes, but charged the initially agreed value, will you still think he deserves the money? – EPMS Devteam Aug 21 '17 at 15:36
  • I don't believe that example is comparable to this case, if my mechanic did that I would be happier as I got my car earlier in any case. The OP indicates he charged per website in a fixed price. Delivering that product in one second or two months does not affect the fact that it is still worth that value. Perhaps in the future the OP could foresee this kind of situation and give a better pricing to it. – DarkCygnus Aug 21 '17 at 15:43
  • @GrayCygnus, are you sure the OP delivered the product after 30 minutes? Or waited 1 or 2 days until doing so (hence deceiving the client)? – EPMS Devteam Aug 21 '17 at 15:53

You should seek clarification of the deliverables before billing

You should examine the contract to determine if you were tasked to develop a scraper or to just scrape the 2 websites.

If the contract was to develop a scraper as a deliverable, then, according to what you have said, you haven't fulfilled that requirement. In other words, it may be that the client wanted the scraper for future projects and the current 'scrape these two websites' were just the initial project. In that case, the answer would be a flat "No" because you haven't done the work they wanted to pay for.

If you did develop a scraper, but you have a huge dependency on a third-party site that you have no control over, what happens in a month or a year when that tool goes away? In that case, you should confirm with the client / contract as well as the terms of service for the tool you are leveraging (ie - can it be used for commercial purposes without a license?)

If the work requested was simply to scrape the two websites, and the requirements to do so were overestimated, there are other quality answers to choose from both ethical and business perspectives.

If it were me, I'd be very clear on what both the scope of work and required deliverables are before billing, otherwise you are risking your reputation with the client.

  • 4
    This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review – mcknz Aug 18 '17 at 20:11
  • 1
    I've edited it to be more clear. Since the question is a subjective one, I believe my answer to not only be valid, but possibly one that won't lose the OP business or reputation @mcknz – NKCampbell Aug 18 '17 at 20:20
  • 1
    What do you expect the OP to do if the sites shut down entirely? He may have promised to maintain it, but unless he's a total idiot he didn't promise it would work next week. It works now, that is all anyone can guarantee. – jmoreno Aug 18 '17 at 20:23
  • 2
    someone flagged this answer, probably because it was asking a question -- if you have a question of the OP, best to ask in a comment rather than an answer. Then use the OP's response to direct your answer. – mcknz Aug 18 '17 at 20:25
  • 3
    Down-voted this response because according to OP's verbiage, he/she is in the business of scraping sites ("usually it takes...", "always charge..."). Your response seems to assume that OP is unfamiliar with the business where they appear to have plenty of experience. – Kent A. Aug 19 '17 at 11:55

protected by enderland Aug 21 '17 at 15:39

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.