2

I have a disagreement with my boss to do with asserting floating point values in unit tests. It seems to me that he does not have a real understanding of the problem.

His position: Tests should always assert exact values. His reasoning: code under test should always return the same value with same inputs, so a tolerance it not necessary.

My position: Imagine this test:

@IsTest
static void passes() {
    Double a1 = 0.1;
    Double a2 = 0.2;
    Double a3 = a1+a2;
    System.assertEquals(0.30000000000000004, a3);
}

Clearly the assertion is mathematically wrong, but the test passes anyway because of floating point inaccuracy. The above simple example clearly proves to me that my position is correct, yet my boss will simply not accept it.

How can I convince him?

  • 3
    did you show this test to him? Just find a real case where your test case fail to detect an error or give an error when the data was correct and propose a test who can handle those cases. – Juan Carlos Oropeza Nov 12 at 15:16
  • 4
    I've elected to flag your question as off-topic. That may be because of a misunderstanding on my part of what you are trying to achieve. If so, I apologise. I read your question as a request for additional arguments that you may use to convince your boss in this specific technical discussion. If you are instead asking how to deal with technical disagreements in general, my only recommendation would be to not focus on right vs wrong. Construct an example (or preferably, select one from the current codebase), so that you boss may himself conclude that his approach has disadvantages. – TvZ Nov 12 at 15:23
  • 15
    Your example is NOT evidence against your boss' position. To a great extent it's evidence in favour of his position. The test is not "mathematically wrong", it's mathematically correct within the domain of floating point arithmetic. – DJClayworth Nov 12 at 17:57
  • 3
    Be careful with tolerance thresholds. Seemingly insignificant rounding errors can become greatly magnified when high order calculations are involved. en.m.wikipedia.org/wiki/Chaos_theory – AffableAmbler Nov 12 at 21:54
  • 3
    I would argue that when your application relies on 0.1 + 0.2 == 0.3 then using floats is just simply wrong. This is a perfect example of why not use floats when dealing with currencies. On the other hand when you are aware of the floating-point inaccuracy then a test like round(0.1 + 0.2, decimals: 2) == 0.3 might be better – spickermann Nov 16 at 10:11
41

It might be you're trying to win a technical point while your boss is making a business point. You have to understand what his priorities are.

Code under test should always return the same value with same inputs

It all boils down to one thing: Why are you fighting this battle?

Is it because you know you're correct, and are just arguing for the sake of argument? Or is your product at risk from bad tests?

If you're arguing for argument's sake, drop it. No one cares. Your boss has asked you to do something, and you should use your Arguing Capital for something important. Pick a better hill to die on, in other words.

edited to add: When I first started as a developer I used to argue on everything that could be argued. I wanted to show my lead I've got the chops. I didn't realise I was annoying my team, and after a while they learned to ignore me because my arguments were not adding value. So, the few times I did have something important to point out, I had to fight against my past impression to them.

If the product is at risk, then you're right to bring this to the attention of your boss. You (plural, The Team) must decide if floating-point precision is good enough for your project (ie. tracking temperatures to the 2nd decimal digit), then the issue is acceptable. As long as it's documented.

If you require arbitrary precision, use an arbitrary precision library and ditch the primitive types (and this argument) all together.

  • 4
    +1 Great answer, the first sentence is a good summary. Developers often lose sight of the reason we write code, which is to get paid by a customer. – Dave Gremlin Nov 12 at 20:09
  • @rah made a good point, but still, I don't fully agree with him or her. These kind of things happen all the time in tech companies, someone is not very strict on the tech side. This kind of test is small, but what if it is a more important situation? Do you trust your boss to do better? I don't think so. As a manager level, I think it is great to have someone who CARES! Anyway, I feel it might a signal for you to find another team or other opportunities. But be subtle. – Bill Chen Nov 12 at 22:18
  • +1 for 'Arguing Capital' and a solid answer – Kilisi Nov 15 at 16:29
11

Your test does not prove your boss is wrong. Also, for some applications I've worked with, regression on the level of numerical accuracy is unacceptable, I once worked with assertion based on file hashes (where tolerance made no sense), and I've even seen people use "tolerance" to claim a failing test was passing (I'm unsure if the person was oblivious to this fact, or if it was a deliberate measure to show progress).

So, unless more information is provided, I would side with your boss in this discussion.

Nonetheless, it seems like both of you might have a communication problem, which can be addressed by making sure both of you can answer the following questions:

  1. How often and how much extra work for you is it to not use a tolerance?
  2. Can you actually find tolerance values that you know will continue appropriate in the future?
  3. Wouldn't that be extra work as well?
  4. Are there other parts of the code where indeed a tolerance isn't simply "unnecessary" but actually "unacceptable"?

When I say "both of you", I mean at least: Can you answer these questions and justify "why" your answer is? Have your boss heard you explaining these questions? Can you tell what would he answer for these questions and what is his reasoning? Can you understand his reasoning?

Often people think someone else "is just wrong", when actually they've failed to understand this someone else. And even if he/she is wrong, unless you understand the person's reasoning, you won't be able to point out actual mistakes ("oh you thought we only worked with double precision? Some tests use an emulated hardware with fixed point precision, so the same code yields different results in different tests") or misconceptions ("Oh, you thought it was easier for me not to not use a tolerance? Actually I'm having more work due to not using it...").

  • Interesting points here. My position is best explained by looking at the example unit test in my original question, specifically the asserted value 0.30000000000000004. The 4 on the end is outside our control, and is possibly subject to change in a different version of the language which could handle floating point arithmetic differently. Such changes outside our control should not break tests. – NickJ Nov 13 at 14:38
  • I suspect that in real life you are not testing if 0.1+0.2 is either 0.3 or 0.30000000000000004. Yes the 0.00000000000000004 difference is caused by numerical accuracy. – Mefitico Nov 13 at 16:24
  • But if for instance you were supplying a library for mathematical operations, some users know that sin(pi) is 0 in theory, yet for some applications, the users might require results that match exactly with legacy software, so they actually want sin(pi) to yield 1.22464...e-16. They're not okay with that yielding 0 (which should normally be preferred) nor are they okay with anything between -1.5e-16 and +1.5e-16 (which besides being worse, is what you are proposing). So case in point: Your test (even in my practical experience) does not settle the argument. – Mefitico Nov 13 at 16:24
  • Hence, more context is needed to decide what is okay or not for your system/requirements, and that should be the focus of the discussion with your boss. – Mefitico Nov 13 at 16:25
7

Show, Don't tell.

Come up with different set of test cases, one according to your logic and understanding, and another set with that of your boss.

Execute and capture the results.

According to your logic the tests will pass, but since your boss's logic is not correct, it'll fail for the different inputs. It'll be made clear then.

Also note: Don't make this a personal battle, try to focus on the correct logic, not the person mentioning about the logic.

2

You and your boss are viewing this situation through the lens of different goals.

You: The test should fail if the software does not give a "suitable" answer

Boss: The test should fail if it gives a different answer than it gave in the past

Consider the downsides of each approach:

Tolerance:

  • You have to figure out an appropriate tolerance for each situation
  • If you chose a tolerance not correctly linked to the business need, you might overlook changes in behavior which are problematic for real uses

Exact Sameness:

  • A change of computational platform bringing different floating point methods could suddenly cause a massive number of broken tests requiring developer time to evaluate and address

It sounds like your boss is willing to accept the risk of having tests fail in the case of a sufficient platform change, in return for the benefit of having his or her attention explicitly called to the fact that something has changed - ie, they want to know about the change and have a chance to consider its relevance.

In the example you gave, the boss wins, both in their role and their reasoning.

Now, if you could demonstrate that developer Macs routinely gave different answers than Linux production servers, you'd have a point. But that's probably not the case here - even things like mobile apps aren't as distinct in their behavior as they once were. Though if you can find a platform the company uses or is likely to that does give a different result, that would be a great thing to bring to attention.

  • I would argue that the expected results should be calculated initially not by the code under test, but by another means (by hand, a calculator) so we know what the correct answer is. If we use the code under test to produce the expected results, then any bugs will mean asserting wrong values. If we calculate by hand, there's no way of predicting what the inaccuracies will be. – NickJ Nov 16 at 14:22
2

Since you asked on workplace.stackexchange.com and not floatingpoint.stackexchange.com: You are fighting a pointless fight. You know you're right, and you know the boss is the boss, and you know that as long as everything passes everything is fine. so write these unit tests the way your boss wants to write them.

You're not doing it because he is right and you are wrong, but because he is the boss. Your particular example is really not something you want to fight about. Save your breath for arguments that are worth arguing about.

PS. He's the boss, so suck it up if you are arguing about things that don't matter. And this one doesn't matter.

  • And if ever you run in situations where actual tests keep failing because it is not using tolerance ranges, you can show it do your boss and re-iterate you point. Or you just silently change it. – Helena Nov 12 at 23:56
  • 1
    I cannot agree with the "he's the boss, suck it up" approach at all. Bosses aren't infallible, they sometimes make mistakes, and I wouldn't be much of a professional if I didn't argue for what I know to be right. – NickJ Nov 13 at 14:33
1

I think @rath is correct in telling you to let it go. However, the one thing that could possibly change your manager's mind and cause him to relent is for him to hear the same complaint from additional people.

While I don't think a coding standard for floating point evaluation in unit tests are not worth dying on your sword over, your manager probably wouldn't die for it either. It's just that he's got higher ground at the moment.

I've lost plenty of mini-battles by having my feedback rejected. Then that same person hears similar feedback again from an additional source - sometimes weeks or months later - and suddenly they relent. (There's got to be a common name for this principal).

If you have anyone else on the team, especially a Senior Engineer, annoyed that unit tests need to evaluate exact precision of floating point values, have them raise the issue with your manager as well. But don't orchestrate a campaign on this issue, otherwise it becomes obvious what you are doing. One advocate is enough who can broach the issue with your manager at an appropriate time.

0

How can I convince my developer that ...

  1. The level of precision is not relevant to what we do

  2. No sane developer will type 0.3000000000004 on a unit test on their IDE as a result of 0.2 + 0.1

  3. We have bigger fish to fry and we can definitely use his help on those things

-His Boss

What I’m saying here is that you’re coming in from a “doing things right” mindset and arguing from that perspective while your boss is coming from a “doing the right things”.

Ultimately, both of you have only so much time in a day so do you focus your efforts on “doing things right” or “doing the right things”.

  • I don't understand what you are trying to say, although point 2 is true, hence my question. – NickJ Nov 18 at 11:38
  • I don't understand why people downvoting this answer. @Goose responded from the point of view of the OP's boss. It's not an obvious way, but it clearly shows a different angle of view on the same situation. – Andrei Suvorkov Nov 20 at 5:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.