0

Background: I work as an algorithm developer in a small research/development team at a SaaS startup. Our whole team consists of juniors (including myself), and the manager is also relatively inexperienced (but more experienced than the team members).

I have research experience from academia, however my industry experience is limited (this is my second industry job), therefore I'm not sure if the situation I'm in is normal and perhaps I'm in the wrong here.

The situation: Despite our team working mainly on research projects (there are no specific deadlines for integration of most features into the product), my manager (who is also the tech lead) prioritizes "quick and dirty" results (he expects to see a new meaningful result every day), and sometimes requests to implement solutions which work decently on some data, but it's easy to come up with realistic scenarios where they won't work.

For example, an algorithm is correct under the assumption that all images have a white background, but the actual data has different backgrounds. If I tell him that we should try fixing the algorithm so that it doesn't assume any background color (and I'll gladly do it myself), the manager would say that the data analyst looked at the results of the algorithm and they were satisfactory, therefore there is no need to change anything. However, I'm not sure how exactly they tested it and how good their methodology was.

There have been a few occasions where previously "satisfactory" algorithms turned out to perform badly on new data, and I suspect this is due to situations like the above (I was not involved in developing said algorithms, therefore it's hard for me to judge what the actual issue is).

Am I in the wrong and suffering from "premature generalization"? Is it better to use "quick and dirty" solutions until they are proven to fail? What should I do, assuming my goal is to improve as an algorithm developer, and provide good results?

2
  • "There have been a few occasions where previously "satisfactory" algorithms turned out to perform badly on new data" - important to know: what happened then? How dd your manager react to that?
    – puck
    Aug 8, 2021 at 7:40
  • 1
    Have you asked your manager why they place more importance on the quantity of results than the quality? There may be some factor you don’t understand, probably funding related. I had a manager once that recognized that me trying to make my code bullet-proof was hurting the perception of how much value I was producing and constantly pushed me to declare things “finished” before I was entirely comfortable. It helped me stay out of rabbit holes.
    – ColleenV
    Aug 9, 2021 at 20:20

4 Answers 4

12

You're working for a startup. These generally have a limited timeframe in which to get results before funding runs out. In scenarios like this, a good enough solution today is better than the perfect solution next week. You are up against the clock.

The priority of the company, and therefore your priority, is to find the shortest path to the minimum viable product.

You need to lay out your concerns to the manager and let them decide what to do. Give them a menu of options and anticipated timelines, and let them decide what to do.

6

Am I in the wrong and suffering from "premature generalization"? Is it better to use "quick and dirty" solutions until they are proven to fail? What should I do, assuming my goal is to improve as an algorithm developer, and provide good results?

Sometimes in business, as in life, "good enough" is more important than "correct methodology". Learning how good "good enough" must be in your particular business context often takes time and patience. It's different in every industry. It's different in every company. It's different when the company is a startup.

Talk to your boss, peers, and others. Seek understanding. Don't worry about "correct methodology".

Remember: "Perfect is the enemy of good".

4
  • 2
    It's truly shocking how many programmers are not aware of this. Aug 8, 2021 at 14:13
  • 2
    @GregoryCurrie Well, I'd say it depends. Too many programmers also just accept any manager wish without speaking up. If architects would do what many programmers do in that regard they'd all be in jail and responsible for some deaths. It always depends on the potential consequences. If they could get people hurt, physically, financially etc. you might be liable (sometimes your company, sometimes you individually). Even if it just lands your company in problems, by e.g. ignoring black people when you build your automatic hand dryer, it might be worth to speak up one level higher. Aug 8, 2021 at 18:26
  • btw. regarding the answer, the first paragraph is totally supportable to me, but "Don't worry about "correct methodology"" sounds too strong, perhaps to calm OP. Imho worrying about it is totally correct and part of OP's job! But they need to learn to assess what are critical issues that one cannot with good conscience compromise on and what is just good enough for what the company needs now. You are totally right that in some perspective one will barely ever build "perfect". Obviously feel free to adjust or not, just giving some perception feedback. (and why I won't upvote) Aug 8, 2021 at 18:30
  • @FrankHopkins I get what you're saying, though probably structural engineer more than architects. Structural engineers are mandated by law, at least where I am, to not cut corners, even if they are told by the managers too. Software people working in startups live in a different world. It's just not really relatable. Aug 9, 2021 at 4:51
3

Never underestimate the tech debt, it will come back to haunt you.

That said, I don't like your leader approach, in this case you have, I think, at least, 3 options:

  • give feedback, let him understand what the team is feeling, maybe he is being pressured from "above", and doesn't handle it well.
  • with each delivery write a list of cases that the product covers, and, if you know it, which don't.
  • propose well made solutions that cover less cases and iterate on that to improve it, this will need a roadmap and kpis to assure that each iteration will give some value, not only quality projects.

That's my list of "friendly" approaches, mix and match.

3
  • 1
    Maybe also compute the extra cost to fix it now as compared to estimating the cumulative cost of fixing it later or living with a continuously larger set of errors. Aug 7, 2021 at 15:17
  • 3
    @CaptainEmacs the problem that startups is that unless they hit milestones and can show enough progress to get to the next funding round, there Mya not be a "later". Aug 7, 2021 at 17:37
  • @user1666620 Yes, that needs to be factored in. Aug 7, 2021 at 18:15
-2

Is it better to use "quick and dirty" solutions until they are proven to fail?

No, "quick and dirty" is never ok. "quick and dirty" becomes "slow and awful" in less time than you expect.

Quick and limited is ok though. That algorithm that works with white backgrounds, how difficult will it be to change when it needs to work with blue backgrounds?

Or with cats instead of dogs? Or sounds instead of images? Or, in fact, we just bought another company and they have a ready made solution that works with all of them, just plug it in will you.

The point here is that you don't actually know what will be needed until you get there, so make sure the options are available but don't try to pave every road.

he expects to see a new meaningful result every day

This is actually fairly normal; and desirable even.

The customer isn't an expert in what they need, the product owner isn't an expert in how to program software and you aren't an expert in what the customer does.

The goal is to close the loop and get feedback quickly so that everyone in that loop can see progress and ensure that it is in the right direction.

a few occasions where previously "satisfactory" algorithms turned out to perform badly on new data

Again, normal, expected and fine. If everything worked correctly already it would be called "developed" not "development". :)

The big question is, when something goes wrong, does it take long to fix it?

That "quick and dirty" solution haunts you here, it won't just take weeks to fix, you also won't be able to estimate how much time it takes.

juniors (including myself), and the manager is also relatively inexperienced

This is the only thing that is worrying, but it is also an opportunity.

Look up "continous delivery", "agile development" and "Scrum". Don't bother with learning the methodology, but try to understand why you, the developer, wants a fast cycle with small changes and instant feedback first.

Once you understand the benefits, you can hopefully see what your manager is after and can give him that while keeping the code clean.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .