-6

I have a question about the going rate for freelance Python programming.

Here's the situation: I work full-time in marketing but I have been a freelance web developer since 2005. My marketing company is getting a new website and our previous development company wants to charge nearly $4000 to access our old database to migrate the data over to the new site. However, I know that I can build an html scraper with Python to get all of the necessary data into a spreadsheet, to be uploaded to the new site. (I built a prototype in about 90 minutes and it works.)

My problem is that this is far outside my job description and I want my company to contract me to hand over this code. I don't want to charge by the hour because the fact that I was able to do this quickly is based on 15+ years of experience (a common issue with hourly rates in web development, I know). I want to save them money—so I want to charge less than the $4000 that the old development company is charging—but I also want to be fair to the work that I've done and to industry standards.

What seems like a reasonable price for this?

2
  • 3
    Did you write this python code on work time or using work equipment? Commented Oct 23, 2019 at 22:52
  • No, I used my own computer and did it on my own time.
    – Kate
    Commented Oct 24, 2019 at 14:24

5 Answers 5

4

A reasonable price for that would be the salary that they are already paying you to do your job. Any other demands for money are likely to go down very badly with your management.

1
  • Thanks for this, but this is far outside of my job description. I am not expected in any way to be writing code for them. And there is a precedent for doing extra work for extra money at my job.
    – Kate
    Commented Oct 24, 2019 at 14:25
1

Looking at it from your employer's viewpoint; For an unspecified amount (lets say $1K), you can scrape the site to get the data.

  1. There's no guarantee that you'll get all the data. You can only scrape the site to get data which is visible on the website. There may be data in the database which is not shown, but legally required to be stored somewhere (depends on jurisdiction, of course).
  2. There's no guarantee that your code will work. They don't know you as a programmer, and they don't know the quality of your work. A 90 minute prototype is not the same as a fully tested solution.
  3. They would have to set you up as a supplier on their accounts system - that will cause extra work for the finance people, and they may have problems with you being an employee and a supplier at the same time.
  4. The company obviously don't have access to their own data - the question then is, do they even own it? If not, then from a legal viewpoint, stealing it back is still stealing. That's one for the attorneys, and a full legal answer would cost more than $4K...

If they let the existing developers do the work, they'll pay about $3K extra. But... they'll have guarantees that the data will be there, and accurate. If not, they will have no issues or qualms about suing the company. The company is already set up as a supplier, so there are no extra hoops to run through. And while they are working on the extract, you can be doing your normal day job and earning money for the company. This - even though it looks expensive to you - is the cost-effective way of obtaining this data (the issue of why your company didn't have access to it's own data is another question, and one that they should address before commissioning a new site...)

8
  • I hear you. What I didn't mention in my original post is that no one at my company wants to pay the $4000 and the alternative is to get the data manually—that is, visit each page (over 1000 pages), and manually copy/paste the data into a spreadsheet. What we are really interested in is the search engine rankings, so keeping whatever data is already visible to the search engines. We don't need the database itself—just the data in it. There will have to be some massaging of the spreadsheet data in any case, because the old database doesn't map easily to the new one.
    – Kate
    Commented Oct 24, 2019 at 14:27
  • The legal question is a good one, but from what I can tell, it doesn't apply here. We still own the data, but it's just that the old company wants to charge us to put a Drupal module on the old site for us to even obtain the old data. The data is still ours.
    – Kate
    Commented Oct 24, 2019 at 14:28
  • Drupal is Open Source Software, and there are many (many!) free backup modules out there for it. If your company has Admin access to the backend, installing your own backup module would be pretty trivial.
    – PeteCon
    Commented Oct 24, 2019 at 15:26
  • We don’t have access to the backend—the old company is essentially holding it hostage. That’s the issue, and that’s why we are building our own website that we do have full control over. The old company is price gouging us because they know we don’t have a choice.
    – Kate
    Commented Oct 24, 2019 at 16:13
  • 1
    Just because you own the data doesn't mean you own the servers it's on. Accessing them against the terms of service may be illegal (like automatically scrapping them). The site may be hosted on a multi tenant system and your actions could cause problems for other unrelated sites. A firm that's scummy enough to withhold your data is probably scummy enough to go after you if for this. Commented Oct 25, 2019 at 14:00
1

Contract with your previous web development company to do the work for them, so they can collect the $4,000, and arrange for them to give you half. Then you make money and they make money and your employer never needs to know.

But that's not an entirely serious suggestion. My serious suggestion is just to do it as part of your normal job, show your company what you can do, make a note of it later when you have your year-end review and use it to leverage a raise.

This is what I did. My company was paying $500,000 USD per year to a data vendor to buy their data. I was able to scrape the data for free (also using Python). I thought about having my friend, who is also a developer, set up an LLC that could scrape the data using my code, collect it, and sell it back to my company at a small discount compared to the vendor. My company would have gone for it. They'd never have known that I was in cahootz with the LLC.

Unfortunately, or fortunately, I'm not sure, I was too honest to do that and I ended up just leveraging the raise.

1
  • 1
    Thanks, notmySOaccount. I do have an LLC of my own that I have considered for this. Not sure I'll go that direction, but it's helpful to hear your perspective on it.
    – Kate
    Commented Oct 24, 2019 at 14:35
1

Whilst I admire your initiative, unfortunately I think your approach is a bit naive. I'll explain why I think that:

  • Firstly if you're migrating data from one site to another, it's typically not just a collection of HTML, CSS and JavaScript files but includes database(s) as well.
  • If the website's content is stored in a CMS (e.g. Drupal, WordPress or similar) then it might have quite a lot of metadata associated with it (e.g. categorisation etc), that is quite important. Not just the raw html.
  • If the website is using a CMS then there also might be a number of users, roles and content moderation workflows that need to be set up as well.
  • You've mentioned that the $4k is to provide access to the database not static web data. Do you have a copy of the database schema? Do you know that it only contains scrapable content?
  • Assuming for a moment that you enjoy your job and you're being fairly paid, why would you want to charge your existing employer a fee to migrate their own data? I doubt many employers would enter into a "contract supplier" relationship with an existing employee. If they felt you were capable then I'd expect that your employer would expect you to do this within the scope of your role.

In short, I think that you might be looking at the data migration exercise from an overly simplistic and/or superficial perspective. Btw $4k is very cheap in my experience to do this type of work. Especially if it includes any degree of data cleansing.

If you're adament that your skills and experience qualifies you to challenge the supplier's approach on the data migration then you could always ask to be involved in the planning conversation with the supplier. But if you challenge them on their approach you might impact your company's relationship with that supplier from a trust perspective.

1
  • Thanks, ChrisFNZ. All we need is the HTML that is visible on the page—we don't need the CSS, JS, schema, or metadata. We are migrating to a new schema, so even the $4K solution would involve a lot of massaging of the data dump. The alternative to paying the $4K is that I have to manually copy/paste the data from each HTML page into a spreadsheet—I figure if I can do it by hand, there must be a way to do it programmatically. Since coding is way outside of my job description, I want to be paid for it, especially to make up for the many hours it would have taken to do this manually.
    – Kate
    Commented Oct 24, 2019 at 14:34
1

You might offer to do the job without any additional charge (during work time). Of course this has the disadvantage that you do not get extra money. But career wise it may be a boost at your current company, you can make a very good impression and you can prove that you have skills outside of your current responsibilities. In the long run this could pay off.

2
  • Thanks, simplemind. Usually I would be in this mindset but I’ve been in a situation where my regular work is undervalued and underpaid, so I’m not inclined to give away this kind of skill for free. I won’t get into the details here, but this is where this question is coming from.
    – Kate
    Commented Oct 24, 2019 at 16:27
  • @Kate You have been in such a situation at your current company? And is this still the case?
    – simplemind
    Commented Oct 24, 2019 at 16:29

Not the answer you're looking for? Browse other questions tagged .