Technical disagreements with my boss

Question

I have a disagreement with my boss to do with asserting floating point values in unit tests. It seems to me that he does not have a real understanding of the problem.

His position: Tests should always assert exact values. His reasoning: code under test should always return the same value with same inputs, so a tolerance it not necessary.

My position: Imagine this test:

@IsTest
static void passes() {
    Double a1 = 0.1;
    Double a2 = 0.2;
    Double a3 = a1+a2;
    System.assertEquals(0.30000000000000004, a3);
}

Clearly the assertion is mathematically wrong, but the test passes anyway because of floating point inaccuracy. The above simple example clearly proves to me that my position is correct, yet my boss will simply not accept it.

How can I convince him?

Answer 1

Your test does not prove your boss is wrong. Also, for some applications I've worked with, regression on the level of numerical accuracy is unacceptable, I once worked with assertion based on file hashes (where tolerance made no sense), and I've even seen people use "tolerance" to claim a failing test was passing (I'm unsure if the person was oblivious to this fact, or if it was a deliberate measure to show progress).

So, unless more information is provided, I would side with your boss in this discussion.

Nonetheless, it seems like both of you might have a communication problem, which can be addressed by making sure both of you can answer the following questions:

1. How often and how much extra work for you is it to not use a tolerance?
2. Can you actually find tolerance values that you know will continue appropriate in the future?
3. Wouldn't that be extra work as well?
4. Are there other parts of the code where indeed a tolerance isn't simply "unnecessary" but actually "unacceptable"?

When I say "both of you", I mean at least: Can you answer these questions and justify "why" your answer is? Have your boss heard you explaining these questions? Can you tell what would he answer for these questions and what is his reasoning? Can you understand his reasoning?

Often people think someone else "is just wrong", when actually they've failed to understand this someone else. And even if he/she is wrong, unless you understand the person's reasoning, you won't be able to point out actual mistakes ("oh you thought we only worked with double precision? Some tests use an emulated hardware with fixed point precision, so the same code yields different results in different tests") or misconceptions ("Oh, you thought it was easier for me not to not use a tolerance? Actually I'm having more work due to not using it...").

Answer 2

Show, Don't tell.

Come up with different set of test cases, one according to your logic and understanding, and another set with that of your boss.

Execute and capture the results.

According to your logic the tests will pass, but since your boss's logic is not correct, it'll fail for the different inputs. It'll be made clear then.

Also note: Don't make this a personal battle, try to focus on the correct logic, not the person mentioning about the logic.

Answer 3

You and your boss are viewing this situation through the lens of different goals.

You: The test should fail if the software does not give a "suitable" answer

Boss: The test should fail if it gives a different answer than it gave in the past

Consider the downsides of each approach:

Tolerance:

• You have to figure out an appropriate tolerance for each situation
• If you chose a tolerance not correctly linked to the business need, you might overlook changes in behavior which are problematic for real uses

Exact Sameness:

• A change of computational platform bringing different floating point methods could suddenly cause a massive number of broken tests requiring developer time to evaluate and address

It sounds like your boss is willing to accept the risk of having tests fail in the case of a sufficient platform change, in return for the benefit of having his or her attention explicitly called to the fact that something has changed - ie, they want to know about the change and have a chance to consider its relevance.

In the example you gave, the boss wins, both in their role and their reasoning.

Now, if you could demonstrate that developer Macs routinely gave different answers than Linux production servers, you'd have a point. But that's probably not the case here - even things like mobile apps aren't as distinct in their behavior as they once were. Though if you can find a platform the company uses or is likely to that does give a different result, that would be a great thing to bring to attention.

Answer 4

Since you asked on workplace.stackexchange.com and not floatingpoint.stackexchange.com: You are fighting a pointless fight. You know you're right, and you know the boss is the boss, and you know that as long as everything passes everything is fine. so write these unit tests the way your boss wants to write them.

You're not doing it because he is right and you are wrong, but because he is the boss. Your particular example is really not something you want to fight about. Save your breath for arguments that are worth arguing about.

PS. He's the boss, so suck it up if you are arguing about things that don't matter. And this one doesn't matter.

Answer 5

How can I convince my developer that ...

1. The level of precision is not relevant to what we do

2. No sane developer will type 0.3000000000004 on a unit test on their IDE as a result of 0.2 + 0.1

3. We have bigger fish to fry and we can definitely use his help on those things

-His Boss

What I’m saying here is that you’re coming in from a “doing things right” mindset and arguing from that perspective while your boss is coming from a “doing the right things”.

Ultimately, both of you have only so much time in a day so do you focus your efforts on “doing things right” or “doing the right things”.

Answer 6

I think @rath is correct in telling you to let it go. However, the one thing that could possibly change your manager's mind and cause him to relent is for him to hear the same complaint from additional people.

While I don't think a coding standard for floating point evaluation in unit tests are not worth dying on your sword over, your manager probably wouldn't die for it either. It's just that he's got higher ground at the moment.

I've lost plenty of mini-battles by having my feedback rejected. Then that same person hears similar feedback again from an additional source - sometimes weeks or months later - and suddenly they relent. (There's got to be a common name for this principal).

If you have anyone else on the team, especially a Senior Engineer, annoyed that unit tests need to evaluate exact precision of floating point values, have them raise the issue with your manager as well. But don't orchestrate a campaign on this issue, otherwise it becomes obvious what you are doing. One advocate is enough who can broach the issue with your manager at an appropriate time.