QA mark issues which were not marked for any other project before (even those that were shipped to production). What should I do? [closed]

Question

I am a software developer. I will try to be as less technical as possible. I made a SOAP service for a project. This service accepts a request from a user, which is like this:

<request>
  <x>123</x>
  <y>456</y>
</request>

Now QA enters another tag in the request like below:

<request>
  <x>123</x>
  <y>456</y>
  <z>789</z>
</request>

He says that the service should throw an error on this and marks it as a bug. Then he further adds repeated tags which are non-list items, then says that error should be thrown, then marks that as a bug. Then he gives String value in tags whose data type is Boolean or Number. The service does not show error on those as well so he marks it as a bug also. Last one is high priority bug and the other two bugs are marked as low priority. The problem is, the tool which I use to generate the webservice does not handle these scenarios, however it uses JAXB for conversion. Now I shouldn't be asking questions about using JAXB in this site. Apart from that, how should I handle this situation? Are these really bugs? If not, then what should I do? Furthermore, other projects have been shipped upto production and passed UAT and SIT with this behaviour (developed by myself and other teams), but no issue was marked in their releases. Should those projects be changed as well?

Edit: After reading the answers, I think I should add that this service gets consumed by another third party system, after which it goes to the end user. So this third party system already accepted previous releases of this and other projects with the same scenarios. Furthermore, adding validations will also impact this third-party system and that is why it is a high risk fix (if that's even a term). Thanks for all the answers. They are very valuable :)

Answer 1

As a developer myself, all those sound like they could be issues, or could not be. "Does not throw an error" is meaningless without context (it could be bad or it could be good). The key question is "What is the impact if we release it like this?"

If the impact is (for example) that the extra field can be processed by the application and do something unintended, then that is a defect that absolutely needs to be addressed. If the impact is that the field is ignored then it's probably not an issue, but you'll need tests (preferably automated) to prove that.

It's really frustrating when something you're working on is held up to extra scrutiny, but nothing you've said there sounds like an unreasonable thing for a tester to raise, and the way to convince everyone that it's not an issue is to prove it with automated tests (and if the other party is not convinced then to request further automated tests from them that demonstrate the vulnerability. Note that the vulnerability is not "accepts a string instead of an integer", it is "After a user has entered a string instead of an integer then their information is corrupted when they save the form" for example).

This is a good thing, and if you work together (rather than as opponents), then it will increase the robustness of both the code and the tests, both now and going forward for new features.

Answer 2

As a QA tester myself, that last bug you mentioned about String values in Boolean or Number fields is absolutely a legitimate thing that probably needs to be caught and handled. Just because everyone involved KNOWS that a certain field is supposed to be a integer only won't stop some zealous computer system from forgetting to cast a decimal and send those extra digits. Nevermind some dumb user who literally writes out 'Two' in a client system that tries to cast values before sending in a SOAP payload. I'm not saying you need to cast these values on the fly, but there should probably be some failsafe so there aren't cascade failures elsewhere. I'd assume thats what the QA was really worried about.

As for the others .... eh, that is hard to say. I'm inclined to agree with Keith's answer, this QA tester was quite zealous. I'd check with your Project Manager on if you need to worry about them, but I'd assume you're probably fine. If the client and other projects know the inputs, part of their job is to actually provide the correct number of inputs.

Stack Exchange tech aside, back to Workplace

Again as a QA tester, in my experience its always better to err on the side of caution, especially starting out. Part of that learning process was I would occasionally pass up 'bugs' that turned out to either be specifically designed that way as a feature, or were a conscious design ask by the client we just got stuck with. When that happened, dev would just write back in the ticket that 'No, this is working as intended' and maybe give me a few insights into what knowledge I was lacking. I'd then re-QA the item, confirm there was no actual bug, and pass the item.

Is your QA relatively new? It could honestly just be the same thing where he is trying to learn a bit and has never used SOAP before. If your team is comfortable that these are unnecessary fixes, mention why that's so and help provide some knowledge.

Alternatively, he did flag the issues properly as 'Low Priority' meaning this isn't something critical. It could also be possible that QA is sick of seeing failures upstream with clients where silly things CAN occur and are sick of constant training. It sounds like this is everywhere since other projects have this as well, so he might just be trying to get any live ticket into the system so some enterprising dev might fix this in down time. Who knows?

You're Dev, not a PM. Fix the one bug, comment on the others, kick the tickets up the chain. If they come back down, you'll have guidance as to what to do. Good luck!

Answer 3

If what your QA tester is reporting as a bug is "Does not throw an error", then I agree with him. Silent failure is a productivity killer.

If I should misunderstand the requirements of your SOAP format and pass in more fields than it expects, and I don't get an error, then it's reasonable for me to assume that all the fields get saved or processed. Then when later requests show that the fields aren't there, now I don't know whether it was a bad request, or a bad response, or a misfire in the database somewhere, or...?

Be strict about the format you accept, and throw an error on any deviation. Strange as it may seem, that's the most user-friendly way to create an API of this sort. I wouldn't worry about giving specific errors; something like "bad request format" is just fine if your expected format is well-documented.

Answer 4

It's time to throw an UNDEFINED_POLICY_EXCEPTION to your mutual boss

This is ultimately an Architectural Strategy Question that needs to be resolved by project leadership, not something that can remain at the level of a dispute between an individual coder and QA tester - though it would be entirely appropriate for both of you to present your thinking to those tasked with setting policy for this.

It's certainly a valid concern and philosophy to argue that components should appropriately handle bad input.

But at the same time, you cannot pursue that philosophy all the way down; at some level, there has to be a trust that inputs provided by an earlier stage are valid.

Where that level is, is a very nuanced question - it depends on access restrictions and preceding stages, and it depends on the consequences of failure. History is full of examples where unanticipated responses to bad inputs have caused painful failures and security issues. It is also (commonly if quietly) full of cases where the error and exception mechanisms themselves have propagated as bad inputs or return values to code unprepared to handle them. Many times, it takes quite a bit of strategic thought to decide how a complex system can appropriately respond to failure of an internal stage.

You need to escalate this issue for a determination of what the project's policy is. . Until you have a requirement, you cannot produce code that meets it, and QA cannot test code against it. Don't wing this; ask for a meeting, present your cases, debate it as a team, and let leadership set policy.

Answer 5

Here is the deal, you develop what you are ask to develop, if those "bugs" where described as requirements, the QA person is just doing it's work.

If they are not in the requirements, then you have to talk to you manager or product owner and ask why, maybe that is not important or needed and the QA person misunderstood, maybe he forgot to write those on the requirements he gave you.

Is in this part where the problem may be, if your product owner (technical lead, manager or whoever is in charge of the project) gives development and QA different requirements, this is what happen.

As per the other projects, the keyword here is "requirements", the software is constructed in a way that fulfills those requirements, read them, understand them, if you not have them, ask for them.

Answer 6

The job of the whole team is to create a product that is good enough to ship, verify that it is good enough to sip, and ship it.

Job of your management is to define what “good enough” means. And someone ‘s job is to define what issues need fixing, and which ones don’t, while staying good enough.

If you think that fixing the issue that was reported is not a good use of your time, then report it to someone who makes decisions. They should either tell you that the issue needs to be resolved, or they decide that it doesn’t need resolving and tell the QA person not to report issues like this one, since it just creates unnecessary work.

Answer 7

Sounds like an overzealous QA tester.

That should have been clarified during requirements gathering. Technically, that's the fault of the analyst, or PM, or whomever. But passing blame doesn't get the job done. I'd try to fix it as much as possible, but make it known that those things should have been clarified prior to getting to you.