10

I have a user, from an internal department, who requests many changes or points out bugs in our company website. We have very close and strong ties, but recently they have been getting upset with having to test these changes. They consider them a waste of time. A recent email I received even had this little line:

When you take a car to a garage to be serviced, you aren't then asked to test to make sure the car is working

The problem is, we follow a model that unless a change/bugfix is an emergency or using standard written procedures (E.g. A standardised parameter change), we require user testing/confirmation before we can release the change or bug fix.

What is the best way to manage this user's expectations and still get the tests completed or confirmed by them?

Improve this question
  • 1
    @JoeStrazzere Sadly, we don't have a QA team, instead we use key users to examine and confirm the system is working as expected. This user is also one of those key users. I've tried education several times, however they are ones who like to cut any red tape they find – Draken Aug 18 '20 at 10:26
  • 1
    Often when you get your car back it still has issues so it's advisable to drive it around the garage a few times to check! – user Aug 18 '20 at 14:29
  • 2
    I don't know about your colleague, but I definitely test that the car works as expected after getting it back from the garage. If it doesn't, I don't pay and return it. Also, acceptance tests are common in a lot of business transactions - ever built a house? If you have an electrician over, do you pay them without testing that the sockets they have installed actually work? I don't think so. Validating that what you get is working es expected is something people do every day. – Polygnome Aug 18 '20 at 17:35
  • 5
    From reading some of the answers, it seems people have different understandings of what the user is actually being asked to do. Can you clarify whether the user is doing full blown QA where they're trying to make sure nothing is broken or are you just asking them to confirm that the changes made are indeed what was requested? – BSMP Aug 18 '20 at 17:36
  • 2
    Getting custom internal software is more like going to a tailor than a car mechanic. Any tailor I've gone to made it very clear that they are not responsible if I leave their shop before trying stuff on first! – Seth R Aug 18 '20 at 18:46
10

There is no universal answer. Let's say the user reports a spelling mistake on a web page. "It says teh where it should say the". You would not hold up deployment of this single fix waiting for the user to agree that you fixed the typo.

Now, let's say the user asked (as one of mine once did) for a button to be "two shades lighter of blue." It makes sense to ask them to confirm that you have guessed correctly what on earth that means. Wording is key here, though: "please test the fix and approve it" implies that you're not sure you did it right, and can sound a little "it's on you if anything goes wrong later". A friendlier "can you let me know if you like the new colour" or "make sure that's what you wanted" or even "make sure my developer understood you correctly" is not only more precise, it contains the explanation of why developers are asking for work from a client.

You also need to be sure that the client understands you do test your work and that you have coded what you meant to code. What the client is testing is that you understood the request. So they ask you to add a column to a report and you forget to ask where they wanted it: when you deliver you say "I added it at the end" or "I added it next to order date" and then ask them to confirm that is ok.

Sure, all this is "testing". Look at it and make sure you like it. To use their analogy, if you took your car to be painted blue you wouldn't drive off in a bright red car without saying anything. So, for everything except absolute emergencies, you do the fix or add the new stuff, you put it on staging for them to look at and confirm it's good, then you put it live. Using the phrasing "confirm it's good" does much of what you're looking for:

  • it contains the assumption that you have done the right thing. "Look for mistakes" contains the assumption there are mistakes
  • it doesn't use a verb that the client is paying you to do like "test"
  • it doesn't use a verb that make the client nervous about taking on liability, like "acceptance"

We often add specific details to our test requests. Confirm the formatting on the new column is what you wanted. Confirm the developer made the right choice about font size. Basically any decision you had to make yourselves because it wasn't specified, and for some reason you didn't ask before you did it, tell them that and ask them to confirm. Anything that was ambiguous and you didn't clarify before coding, point it out now.

Improve this answer
  • 8
    That's true. But that's not what the user is in this situation. We used to talk of verification and validation. One tests if you've met the spec (and if you've broken anything else) and the other is "Was the spec what we actually should have done?" and typically comes from users. My choice of words is deliberate and meaningful. – Kate Gregory Aug 18 '20 at 13:43
  • @JoeStrazzere Is the end user a professional QAer? Are they going to look extensively for problems before it goes into production? – user253751 Aug 18 '20 at 17:44
  • 1
    Completely agree with this. We try to figure out what they actually need before we even start, but sometimes a request looks reasonable and specific and then it only turns out after they see it that it wasn't what they needed. If there was a specific person who wanted something then I always want them to look at it before I declare that it's done-forever. – user3067860 Aug 18 '20 at 19:21
18

What is the best way to manage this user's expectations

Thoroughly understand the problem before you fix it and then test professionally.

You shouldn't trust an end user for the testing, it should be done by a professional who has a vested interest in making sure it's error free before release.

Improve this answer
8

You need to present the user acceptance step as something that benefits the customer, not you.

You have been talking about the benefits to your company ("we require user testing to confirm the change/bug is working as the user expects", "we are not allowed to release live without user validation"). It's not surprising that the customer feels like they are doing work for you. This, after (in their mind) you failed to address their needs in the first place, by releasing something that wasn't what they wanted!

You can start by explaining that this is a key part of the process that benefits them. Imagine how much worse it would be if you released a "fix" and it still wasn't fixed, or working in the way they wanted! Imagine if you'd accidentally made it worse!

There was clearly a breakdown in communication that caused the bugs in the first place. If you had perfectly understood what they wanted, you would have perfectly tested against those expectations yourself, and released something that was... perfect. They found a bug. That means that whatever process you have for them to request a feature and you to provide it, missed something. It isn't sensible to assume that the process to report the bug and get it fixed, can't possibly miss something too. The sensible thing to do is to double check: "we think we've now done it as you wanted, is this right?".

This is especially true if they are paying for any of these changes or fixes. This is an opportunity for them to ensure they are getting their money's worth. If you release something, and it's still not what they want, you'll have to go through the whole process all over again, and charge them again. Wouldn't it just be better for them to confirm the fix before releasing, so they only pay once?

In any case, it needs to be explained to them as a positive opportunity for them to have their input, not a drain and demand on their time for no benefit.

FWIW, I think it's a bad idea to argue by analogy, as other answers do, because you end up talking about something that isn't actually the problem you should be talking about. Even so, if I took my car in to fix a flat tyre, I'd have a look at the tyre before driving off, to make sure it really did seem to have been fixed. Otherwise, if I just started driving and noticed it was flat a way down the road, the garage might say they did their job right and the new tyre must have gone flat after I left.

Improve this answer
  • Quite true about the garage. Funny story... had my car in for some work, stopped by the garage, picked it up, drove away. 30 seconds later, I get a call "hey come back, we forgot to reinstall X". It wasn't a safety issue but ever since I look at the car pretty carefully before I drive off. – DaveG Aug 18 '20 at 13:40
5

When you take a car to a garage to be serviced, you aren't then asked to test to make sure the car is working

Ah, funny. When I take a car to a REPAIR (and website changes are not servicing), which may include stuff found at servicing, such as the ball bearings on a tire being not totally round...

...every time that happened when I picked up the car I actually was asked on a test drive with a representative of the mechanic (representative as: front line worker, as not super small shots separate the mechanics from the client).

Your client seems to think that website changes are servicing. This is NOT the case. Servicing would be patching libraries. Changes are akin to repairs or upgrades on a car. And the assortment that no test drive is done in those cases is flatly wrong or indicates the use of a low end mechanics shop, in my experience.

You should talk to the client about that. Also about the fact that upgrades to a website UI flow may not actually work out the way the client expected them. Not that you make an error - but I can not recount the number of times a change request was followed by "ah, that does not work as I expected" although the change was exactly what was ordered. Catching this early is part of a workflow.

I suggest you talk to them and explain that their comparison is a fallacy that more points towards the use of a low end mechanics shop ;)

Improve this answer
1

I'm of the opinion that you should not be telling the user to test the bugs they pointed out to you, or the features they've requested that you / your company chose to implement, unless you're unable to recreate the bug on your end for whatever reason.

When they report a bug to you, you should first be recreating the bug before you go and implement a fix, so you know that you actually fixed the bug...Then when that bug fix is deployed, you can communicate to the user that a fix has been deployed.

If they request a new feature, you / your company should be getting all requirements / expectations up front before any work goes towards actually implementing the feature. Not blindly guessing what they want, implementing it, and then asking them to test it for you.

Improve this answer
  • 4
    The user end testing is the final step in the validation process. We do all of the requirements / expectations gathering before hand and/or checking how the bug works before fixing it. We are still not allowed to release live without a final user validation. – Draken Aug 18 '20 at 9:49
  • If that is the model that your company chooses to follow, then your company simply follows a model that is going to irritate and frustrate users that have issue with being the ones responsible for testing bug fixes and new features. I know that if I was a user of a website, and the website owners expected me to test every bug that I reported to them, there's nothing you can say or do that would make me not be annoyed and frustrated by your company's process. – Garrison Becker Aug 18 '20 at 9:54
  • Either your company needs to change their process, or just accept that some users are going to be annoyed / frustrated by it. – Garrison Becker Aug 18 '20 at 9:55
  • 1
    Every software company that delivers software to clients expect clients to test it. The company I'm in has contracts with a lot of clients, and for some of them (when the client doesn't want to pay for QA) the software's warranty is voided if they didn't do a proper test campaign. Lots of other software companies work like this, because it's expected that the client will test it. Agile methods involves users the more it can in order to maximize quality - so basically what you're saying is the opposite of the good practices. – LP154 Aug 18 '20 at 13:41
  • And yes, you were 100% straight up wrong when you said "Every software company that delivers software to clients expect clients to test it". And it is a known fact that not EVERY software company expects clients to test their code. That means you were flat out wrong, like I've stated. – Garrison Becker Aug 19 '20 at 7:21
0

You said:

The problem is, we follow a model that unless a change/bugfix is an emergency or using standard written procedures (E.g. A standardised parameter change), we require user testing/confirmation before we can release the change or bug fix.

Why do you follow that model? What requirements/goals is this model intended to achieve? Who instituted it?

If the idea is to provide an opportunity for stakeholders to verify the solution, put a timer on the stakeholder verification step and if the stakeholder hasn't verified a solution (that you have tested) in 2 weeks then close the issue/work item as "complete".

If the idea is to delegate testing to stakeholders, well, I imagine you'd need to escalate this up the org chart since this particular stakeholder doesn't seem to believe that their job duties include performing the testing you want them to do.

If you generally believe that you are on the same page with problem reporters as to what problems they are reporting, the timeout strategy seems like the way to go.

Improve this answer
-1

When you fix a bug (fix the car) YOU should check if the fix was done correctly. So you should replicate the issue noted by client before committing to fix.

Simple: client notes "I hear clinging noise from engine", mechanic's solution "More noise insulation was added so the client won't hear anything from engine bay" is kind of a bad solution. If the mechanic replicates when and how the client hears the noise he can pinpoint what needs to be changed/fixed, and confirm the fix removed the issue. He then can document the fix and avoid further accusations from clients if he changed the oil and 3 days later the AC stopped working.

When a mechanic MODS (changes) the client's car the shop should make sure the client is aware of what the mods do and how they affect the driving experience. Then, before giving the car back, they should make sure the car is working as intended (there's no point in adding turbo if you remove the fuel line). If the change is interfering with existing architecture the client should be aware of it before the shop starts working: "Hey, the only turbo that fit needs a hole in the hood/bonnet, do you want us to do it?"

The client expects a "package". And trust. So when they go for a service they don't expect they need to check if the oil was changed. Or if the wheels are balanced after tire change. The shop might tell him he need an alignment.

Treat fixing a bug like a recall. The client comes in and expects you to change a compromised part to one that will work as intended. Not to discover that you changed the control arm that wear the bushings prematurely but not the bushings.

Improve this answer
-1

What is the best way to manage this user's expectations and still get the tests completed or confirmed by them?

You need to set the correct expectations for your users. If the company policy is as you stated:

we require user testing/confirmation before we can release the change or bug fix.

Then you need to make it clear to the user that their testing is a requirement before you even begin to work on their issue. If there is any documentation indicating this policy then you should present it to the user so they don't think that you are just being lazy and do not want to test yourself.

The fundamental issue seems to be with the policy. I would speak to your manager, or whomever has the proper authority so that this policy can be re-evaluated and improved to better address bug fixes/changes.

Improve this answer
-2

When you take a car to a garage to be serviced, you aren't then asked to test to make sure the car is working

That's actually true, a client can take your software and say it's good and put it on production, and after the guarantee time, if there is one, they will have to pay for everything they haven't checked in testing. And it's the same with a garage or a TV. If you have a problem with it and didn't test it when you received it or within the guarantee terms, you'll pay for it.

I'm not telling to answer like this to your client, but you could make him remember the terms of your contract, hoping you (or your team) didn't mess up the maintenance contract.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .