4

We're reviewing a code change that one of our users has returned to us, claiming that they've found a bug in the code, where the total of some numbers doesn't add up correctly on a legal form. At first I was unable to reproduce the problem, which I told the user so that they were aware of the progress(in hindsight, telling them I could not reproduce the problem before knowing why may have been short-sighted, but I wanted feedback from them).

After reviewing it further, I've found a discrepancy in how the user put together their failure example. The order in which they are added up is based not on the order in which they're entered, but on the date they were received, which the user puts in when they enter the data. This means that the way the user put the data together, not a bug in the application, led to the discrepancy they saw, and I found that this is the way the application has always worked.

Now I am in a bit of a bind - it is important that the application remain consistent, and I need to tell the user about this discrepancy without offending them. They aren't known for being aggressive or getting highly upset, but I don't want them to base their decision off of being defensive about the test case they made being 'correct'.

How can I bring this up to the user without offending them? The users own this application, so for better or worse what they say goes, goes. I can show them the logic behind how this 'error' appears, and why it is programmatically correct, and I really do think it is also legally correct (and my co-workers agree), but if they decide we need to 'fix' it, we won't be able to stop them, so I would much rather talk them down the right path without being confrontational.

Just to be clear on how much power this user has:

In our specific case, the user is one of two heads of the department that uses this application, and specifically in charge of the part that this change is part of - essentially he is the biggest "stakeholder" in this part of our application

  • Possibly describe it as a workflow issue, and ask if this workflow is likely to occur again. If so, offer to discuss ways to mitigate (through training? error message? etc) the problem. – mkennedy Jun 11 '15 at 16:39
  • @mkennedy It's not actually a workflow issue - it's how numbers are put together for a legal form. I've edited my question to make this more clear. – Zibbobz Jun 11 '15 at 16:43
  • 3
    "Wrong" is not a word that you should use for a situation like this, it makes people defensive and that's not something you want to encourage in critical stakeholders. – teego1967 Jun 12 '15 at 13:10
  • This is a problem very specific to software development. It would fit much better on programmers.stackexchange.com. – Philipp Jun 14 '15 at 11:16
  • 2
    Even if there isn't a bug you've learned that your UI clearly isn't foolproof. – Nathan Cooper Jun 14 '15 at 12:49
16

If I'm understanding the situation correctly, this user is essentially your customer and has full decision-making rights over your software, so that's how I am approaching my answer.

Go to your user, fully explain the situation, and give them options and recommendations. Tell them what the code does and expects from the user, and why you set it up to behave that way. Make sure to specify the difference between what they did and what was expected, and why they got the result they did. Don't point blame; just state facts and cause-effect.

You can then give them options of how to proceed. If they want to change the software so the erroneous setup is actually correct, explain the impact, cost, and time it will take to do so. Also give them the option to leave it as is and offer better training or clearer instructions in the manual. Depending on how your software works, there may also be some middle-ground available, perhaps by adding warnings during data entry.

By leaving the final decision with the user, you give them the power to decide if they were right or wrong, lessening the risk that you will offend them. It sounds like you haven't had issues with this user before, so as long as you approach the discussion from what is best for the program and not who's right or wrong, you should hopefully not have any problems.

@keshlam gives a good example, simplified from IBM's PMR handling process:

We certainly understand why you didn't expect that result. Here's why it happened: _____ You can work around the problem by reorganizing the data so _____; that's the fastest way to get back on the air, and is what you'll have to do until we have a fix that tolerates both versions. Now, given that workaround, how critical is it that you have a "real" fix? Can you get by with the workaround, do you need the fix ASAP, or is it something we ought to fix but that can wait for us to get to it without interrupting other work?

  • 4
    "We certainly understand why you didn't expect that result. Here's why it happened: _____ You can work around the problem by reorganizing the data so _____; that's the fastest way to get back on the air, and is what you'll have to do until we have a fix that tolerates both versions. Now, given that workaround, how critical is it that you have a "real" fix? Can you get by with the workaround, do you need the fix ASAP, or is it something we ought to fix but that can wait for us to get to it without interrupting other work?" (That's a simplified version of IBM's PMR handling process.) – keshlam Jun 11 '15 at 20:01
5

Just because this is how it has always happened is not sufficent to say the user is wrong. It could be the code has had a bug in the calculation all along. I have seen this happen many times.

What you have is a differing intrepretation of the requirements and in this case, it is the stakeholders who need to make the final determination, not you and not just this one user unless she is the only important stakeholder. The more critical to the sytem this calculation is, the more important it is that all the major stakeholders become aware of the issue and can do the analysis they need to do to determine what is actually needed.

So you present the findings to the decision makers. You emphasize that the user who reported it is correct using her calculation but that is not the calculation your software uses. You then ask them to determine which calulation is correct.

I cannot emphasize too strongly that this is a purely business decision. You can (and should) make recommendations and provide information about how much effort it is to fix, but in no way can you determine that the user is incorrect here or make the choice based on your preference. This is NOT a development decision.

So I would present to them the information that this part of the code was not changed and how the calculation is currently designed. Show them the code calcuation results and the user's calculation results in a spreadsheet. Make sure to show them the intial requirement if you have one.

Explain the impact of changing this calculation on not only that page but any reporting and especially and emphasize that means historical reports were incorrect and might need to be regenerated. If it would affect other systems such as data warehouses or file exports that too would need to be mentioned.

Explain how difficult this would be to change and give an estimate of the hours. Explain the risks associated with making a calculation change or with not making a change.

In everything you present, make sure to give the user credit for the correctness of identifying an issue when her calculations showed a problem. Emphasize that you agree that there seems to be an issue if you use her caluation method. However, make your case for why you think the current calculation makes more business sense. Frankly once the stake holders start looking at the calculation, they may dertermine that both methods are flawed and are not giving them the information they thought they were getting.

Once a decision has been made, then do your best to implement it even if you don't agree with it. Again, this is not your call, only the business stakeholders have the right to determine what to do in a case like this.

  • In our specific case, the user is one of two heads of the department that uses this application, and specifically in charge of the part that this change is part of - essentially he is the biggest "stakeholder" in this part of our application. – Zibbobz Jun 11 '15 at 17:42
  • In that case, you get both department heads together and tell them that this issue was raised and that using this calculation methodology it is a problem, but using the other calcuation it is not. Then say you didn't want to change something that critical without everyone being on the same page and confirming which methodology is correct and what the risks of changing it are. At this point you are just doing due dilgence, reseaching a raised issue, presenting what you found and letting the people who are supposed to make the decison make it with all the information they need at hand. – HLGEM Jun 11 '15 at 17:50
  • And be sure to raise the legal issue. – HLGEM Jun 11 '15 at 17:51
  • Also specific to this problem - the two heads work on entirely different parts of the application that don't intersect in this regard at all. Bringing the other head in for this wouldn't make a lot of sense, because they wouldn't even know what we're talking about, and it wouldn't have any impact on their part of the application. I didn't bring that up before because I didn't want this to get too narrow in regards to our own application structure. – Zibbobz Jun 11 '15 at 17:53
  • 1
    I make these kinds of presentations roughly severla times every week. Present the problem identified, the research into the issue and then the possible solutions and costs and then let the stakeholders decide. The more you do this, the more they will trust your judgement when you tell them what possible solution you think works best because you are showing you understand their business needs. – HLGEM Jun 11 '15 at 17:55
4

Just tell them straight up the calculation is based on entered date not entered order and that is how it has always worked. That would be a change to the code logic. And it could cause consistencies as other analysis and calculations are also based on enter date.

If they decide it must be 'fixed' then yikes. I would get your boss involved so if it does escalate he/she will not be surprised.

From a UX perspective a caution message. Entry order was not in the same order as Entry Date - calculation is based on Date.

  • I agree. Just be straight forward and clear. No need to tiptoe around anything. – NotMe Jun 11 '15 at 17:28
1

1 - Informing the user of error

On this one, I'd go with the strategy - "maybe you didn't know about this tricky feature, it took me some time to figure it out too..." and then explain why this unusual way of doing addition in a not-commutative fashion is actually the only way the software can function accurately due to the cases it's got to handle.

2 - Posing the option of software change

You may manage to explain this so well that your high power stakeholder feels enlightened and says "no problem - can you add XYZ to the documentation?" and the problem is solved. In which case, skip this step. Don't propose changes when all sides agree that no-change is good. That sounds obvious, but often the visceral reaction is to try to go above and beyond to fix a problem.

If the user is still saying "I don't like it, change it" - then ask the user to engage with you in figuring out a better way to handle the problem. Enunciate each case that the software is currently handling - from a business perspective and not a technical one - and what outcome will be produced in each case. This guy is the expert on what cases are important and what the outcomes should be - so he may just find a flaw in the logic. He also may be in a position to say "you are absolutely right about the outcome of case # N, but we can afford to not care about that one. On the rare times it happens, we just do it manually anyway." In which case you may end up writing a new feature to correlate the manual work, but at least he'll be picking which priorities are important and he'll be very aware of the ramifications of his decisions - which is about all that any leader can ask for.

3 - Covering yourself

Especially if this is your first time with such a tricky user, cover yourself. After a meeting for #2, write the user and your immediate supervisor a summary memo - cover the explanation rendered in #1 in a short summary, and the conclusions reached in #2 and what actions you'll be taking now that you've finished discussing it. Include a time estimate and either give a deadline or suggest that you need help setting a deadline amid other conflicting priorities.

This gives you a couple of valuable things:

  • Your boss knows what is taking your time
  • The user gets to review any decisions he made with you and correct any miscommunication before you waste time working
  • Your boss knows that you are solving problems for a high stakes user and can directly see the outcome of your communication and the decisions - so if a long time later the user claims that you DIDN'T do what he asked, your boss can say - "hey, you got an email on this way back then - why didn't you let my employee or I know that you didn't like what we agreed on??" - VERY powerful.
  • You give yourself some time to really consider the work involved in making any changes. It's often the case that with someone staring at you, you won't have the time to really research the risks of what you are changing and give a good estimate. Giving a follow up email lets you check with coworkers, do some research and feel confident in your estimate.

If I were your boss, or the tricky user - I'd be thrilled with a mail like this, particularly if the summary of the explanation was under a paragraph long, and the outcome actions were clearly bulleted, very short and with easy to read deadlines. Avoid the "wall of text" effect - where you send one big missive with lots of words - go for short and pithy, rather like the US News news paper (less like the New York Times).

-Beth

0

First, it's not a user. It's a paying customer. It's actually a customer paying you for creating a product. That's a very, very, very different situation.

If a customer is wrong and offended by the fact that you are doing your job and finding they are wrong, that cannot be helped. There are people like that, and nothing to be done about that.

In your case, the seemingly same calculation can get different results. I'd find that very suspicious. If this happens on a legal form, I'd find it even more suspicious. If this is due to rounding errors, I wouldn't want to tell a customer "you'll get different results depending on how you enter the data, live with it". I think before telling the customer that they are wrong, you should really consider that maybe they are right, and maybe adding up numbers should always give the same result, no matter how the numbers are entered. (How to achieve that: Ask stackoverflow.com)

0

You don't tell the customer they are wrong, you tell them how the program operates and how long it has been doing it that way. If the behavior has changed, tell them what it was before and when the change took place.

The customers desired outcome is never "wrong" - it can be unethical, uninformed, unconsidered, dangerous, expensive, impossible or any of a dozen other things, but not wrong. They want what they want. If you want something else, present it as what you think is a better option.

  • The customer is not always right (nor are we). The customer is always the one with the money. Try to help the customer be right. But sometimes you have to decide between being right and being paid. If you must do something wrong, consider providing the customer with a special option so you don't break the behavior for everyone else – keshlam Jun 14 '15 at 3:13
  • @keshlam: saying, "you are wrong, you don't want X you want Y" is not the right thing to say, whether it's a customer or not, unless you are sure they are lying to you. – jmoreno Jun 14 '15 at 3:28
  • Not advocating you be that blunt -- but when they want something really unreasonable, that usually means they asked the wrong question earlier, and you can sometimes work back and help them correct that. Which is better for everyone. – keshlam Jun 14 '15 at 4:43
  • @keshlam: I am trying not to make this software specific. When customer expectations don't match your service, you explain your service, you don't say they are wrong to want a different result. You might try to adjust their expectations or you might adjust your service. The misunderstanding can be your fault or theirs, but it is more important to correct the misunderstanding than to apply blame. – jmoreno Jun 14 '15 at 13:01
  • We're in agreement. – keshlam Jun 14 '15 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.