16

I work in the software industry in a junior capacity and have been unable to reproduce a bug.

This issue has apparently been seen < 3 times at a customer site, and a senior member of the team have heard of it, however they have not been able reproduce or isolate the cause. Unfortunately I have not been provided any logs.

We normally close off such a bug with the stated reason "unable to reproduce". I have spent more than 2 days digging through code, comparing versions, attempting to reproduce this bug by following the given steps to reproduce. Unfortunately with no more additional useful information at hand, I don't see or know of anything else useful I can really do.

My manager has suggested I try different things than what was given in the steps to reproduce, to try and reproduce the bugs behavior I have to, no avail. My manager at the least wanted an answer to what possibly cause it. (This makes me somewhat uncomfortable as what "might cause it" when that can be definitively tested quickly, it either will or won't)

I would like to provide a satisfying answer to my manager, and frankly I don't have one. My manager believes that if it's happened twice on a customer system (but no useful logs pulled or available to me) that it is bound to happen again (I can agree with that however without additional information I don't know what to try), however I have been unable to reproduce this bug.

My work on the bug has the same conclusion as the senior member who has heard of this behavior (couldn't reproduce or isolate the cause).

I don't want to spend anymore time spinning my wheels on this as I feel I've given it every reasonable effort with the information I've been given. Is there a good way to sell this to my manager?

Clarifications:

The system at hand is a deployment of our companies software solution for a customer. I don't wish to go into anymore detail into this, my apologies.

By no logs I mean that no logs were provided from when these scenario(s) happened. Our code logs errors, warnings, etc. We have in my opinion decent logging.

In my opinion the bug's severity should be low to medium. The client has to wait longer to see the resolution of an action, however I still defer to those around me who know better than me, still want to see this addressed.

| improve this question | |
  • 11
    A valid answer to "My manager at the least wanted an answer to what possibly cause it." is "I cannot reproduce it on our test systems, it may well be an environmental issue at the customer site and we have no logs (or access to logs)." Without logs, you will continue to chase your tail, so ensure logging is turned out at the client site and watch for another incident. – Jane S Jul 2 '17 at 22:36
  • Isn't this really a pretty technical software question, and not for this site? In answer to your question, you've just described ...... all software :) Simply follow standard procedure: totally reengineer the segment in question, in particular, adding massively more reporting and logging. (You glibly mention you have no logs or other information - that's your fault :) ) Software is tough. You describe the normal situation. it's a fantastic chance to be proactive. – Fattie Jul 2 '17 at 23:50
  • 1
    OP, in very general terms what is the "system" at hand, just to make it easier to discuss? (ie, is it "in your embedded code" "something to do with a WWW interface" "a problem in a game") – Fattie Jul 2 '17 at 23:54
  • the thing is @JaneS, you don't just sit around waiting for someone to implement logs. For example, if you're an (android) app dev, you wouldn't just cross fingers a user out there sends in a good report or dump. You'd instead (and of course all volume android apps do this) just use one of the many tools available that proactively enlightens you on crashes in the field. – Fattie Jul 3 '17 at 0:00
  • 3
    How severe is the bug for the clients who encounter it? Does it stop one or more people from doing their job? Does it have a financial impact? Are there regulatory or compliance issues? Safety issues? How its dealt with ultimately is a cost/benefit issue. How you can sell this to your manager depends somewhat on the client impact. – Laconic Droid Jul 3 '17 at 2:09
29

You have three entirely separate problems: the technical thing of finding the bug, the political thing of keeping your manager happy, and the different political thing of keeping the client happy. (For example, your boss may want your other work to still get done, but the client doesn't care about that. They care about other stuff.)

You want your boss and the client to know that you care and want to fix the bug. But you don't want to give the impression that you're lost and don't know how to find it. The longer you work on it, the higher the chance people will think you're not good at this. But refusing to work on it is bad too: you have to care and to want to fix it.

So. They have told you that something bad happens. An error message appears, or a wrong value appears somewhere. And you've looked for that error message or for bits of the code that affect that value, but you can't tell which of those ran. When you do what they say to do, it doesn't happen, but you can't see a significant reason why it "works on my machine." Make sure that your boss understands both of these things. That you are thinking logically and you have looked into the possible causes from looking through the code, and that you have tried to repro with their instructions.

Next (check this with your boss if need be) go to the client and get someone to agree to be your go-between on this bug. The person has to agree to put in some time for you so that you can solve it. Follow steps like these:

  1. the person or you gets someone in the client org to agree to give you logs on request.
  2. the person triggers the bug. They take screenshots for you. If there's an exception dialog, they click for More Details and they copy and paste everything they get. If there's a report or other save-able exportable asset that demonstrates the problem (eg the bad number) they save or export it and give it to you.
  3. the person or you asks for logs covering the time that just happened.
  4. you investigate armed with this info.

If you solve the bug, yay. If not, you create an instrumented build. Say a particular error message occurs in several spots in your code. You change it so each of those spots leaves a unique signature - in the logs, in the message, somehow. Maybe you also turn the logging up if that's a thing you can do. You get that code to your person. Then steps 2 through 4 again.

Still not found? You check that your person is ok with still being your person. You check with your boss that it's ok to still be looking into it. You add more instrumentation, more logging, more capturing. Your boss sees that you're not just thrashing, but you're logically working your way towards narrowing it down. Your client, who is interacting with you a lot, sees too that you're working on this and working intelligently. And you are likely to solve it this way, too. But along the way you will be sending the right message to those who are watching you.

Still not found? There are things you can install on the client's machine that enable "historical debugging" or "replay debugging" -- if the client refuses to install it you can honestly say "well I won't be able to solve it if you don't co-operate" and if they do install it, well, again you're sending the right message and you are that much more likely to solve it.

The key is to know what you need (more information) and how to get it. Then set to work getting it and make sure everyone knows you have a plan and are following it.

| improve this answer | |
  • 1
    The beauty of this answer is that you provide a suggested course of action to your manager, instead of a can't reproduce it report. – user8036 Jul 3 '17 at 12:38
  • I like this answer. I would however place the emphasis more on implementing better diagnostics for the problem, rather than having somebody to liaise with. – TonyK Jul 3 '17 at 16:55
  • @TonyK if they had not written any logging code, I would agree. But they have written logging code and the bug-reporters are not giving them the logs. That has to change. – Kate Gregory Jul 3 '17 at 21:38
  • Thank you for your answer, in particular I like the breakdown of the 3 problems I have tangled together. – user1821961 Jul 4 '17 at 0:10
8

The answer depends from the severity of the problem :

High Severity

If the problems is severe and creates real problems with high impact on customers then you have to continue working on the problem until you find a solution or a reasonable workaround that will minimize the consequences of the bug. You may try the following:

  • Add additional logging
  • Work on the machine that produced the problem
  • Talk with the person who produced the problem
  • Make your code more robust

Low severity

If the consequences of the bug are minor, then you have to convince your boss that the cost of fixing the problem is much greater than the benefits. Then the manager will have to handle the customer.

If you cannot convince your manager then say to him, that you will work on the problem for one hour every day, and that you cannot estimate when this problem will be solved. This will give you three benefits :

  • You will work on your other tasks
  • Not working continuously on the problem may help you to find the bug more easily
  • After one month your boss will see things differently and probably will be easier to convince him to stop working on the bug
| improve this answer | |
3

If you can't reproduce the bug, how will you know if you fixed it? All bugs should have a test script (set of instructions) that will reproduce the bug. The test script is run before and after your bug fix is installed. This will tell that you fixed the problem. Enforcing this is as simple as giving bugs that don't have a test script a low priority.

| improve this answer | |
  • Your answer addresses a part of my struggle which I have failed to express in my question and I agree with you, I can't know if I've fixed the bug if I can't reproduce it and test my fix against it. To clarify the "test script" or steps to reproduce the bug don't reproduce the bug. My manager was concerned about this bug, thus it has become my highest priority currently to find a satisfying answer to this (unreproducible) bug. – user1821961 Jul 3 '17 at 1:52
  • 2
    This answer fails to address the very real concerns of the customer. – TonyK Jul 3 '17 at 16:56
  • He is supposed to develop the test scropt, but has so far failed to do so. – jmoreno Jul 4 '17 at 1:44
  • Sometimes bugs can't be reproduced in-house no matter what. I suggest you delete and get the disciplined badge. – Joshua Jul 16 '18 at 15:49
2

Talk to your manager. Tell him that this bug is hard to reproduce (and you haven't managed at all so far), and that you have no useful information to help you. And that you have several choices: Go on trying to reproduce the bug, examine the code that is involved and try finding bugs there, or modifying the code to make it easier to find the bug, through logging and whatever suitable measures. Or of course do nothing - mark the bug as "cannot reproduce" and hope it doesn't come back.

Your manager should then decide which path he wants you to go. Fattie's answer might help fixing the problem, but might get you into trouble if your manager has totally different priorities.

PS. A colleague of mine had a bug that would only strike after 18:12 in the evening (just for fun, try to figure out what's special about 18:12:16 in the evening). Everything worked absolutely fine before that. From nine to five, that bug was absolutely not reproducible.

PS. There is one reasonably common bug that is repeated by developers again and again and results in software malfunctioning in up to the first three days of the year, displaying the wrong year. But not every year. On January 4th that bug disappears.

| improve this answer | |
  • 18:12:16 - who would choose to store a time as a 16-bit number without checking how many seconds there are in a day? – André Werlang Mar 7 '19 at 19:31
  • About the Jan 1st-3rd issue, please tell more about it. It doesn't seem to be a leap year or timezone issue. Maybe the two factors at the same time... – André Werlang Mar 7 '19 at 19:34
0

For arguments sake lets assume somebody's cat died as a result of this bug. This person wants you to find what is responsible to ensure no more cats must perish because of this senseless maniac bug, who is practically a psychopathic serial cat killer, with no conscious.

Now if this was your cat, at what point would you want the detective to stop looking? What if they had exhausted all reasonable leads after 1 day, I think you would still want them to keep looking so you still have a chance of getting justice for your cat.

But the fact is you have no emotional attachment to this cat, and you know deep down your efforts are in vein. So what to do? IMHO an appropriate course of action would be to reassure all parties that you will do what you can to catch this killer, but this may need to enter a long term strategy also to take any preventive measures as a result of your investigation, to ensure cats are as safe as they can be in case the killer were to strike again. Then you will need to get additional work assigned to you whilst you still search for the cat killer. At the beginning this might not be much work and your time will be still mainly on chasing the killer. But slowly you can ramp up your additional tasks till you hardly spend any direct time on the case.

Do keep it in the back of your mind, maybe suggest additional logging changes you could make to help set a trap for this bug if it does come up in the future. That will probably be your best bet in ever really solving the bug as you say currently there is no logs, maybe logging is something lacking in the business which could be improved and you could spend time on that and it would still be under the hood of catching the bug, and then move on to the next task once you convince your boss that if the cat killer strikes again you will catch him then, and minimize any harm done to customers in this process.

| improve this answer | |
0

You said that your software produces adequate logs. Why don't you have these logs?

If it's because you don't have the security permissions to see them or something administrative like that, tell your boss that you need to see the logs to be able to solve the problem. If he can't give you access but he can give somebody else access, then maybe that person should be assigned to work on the problem.

If you don't have the logs because they were deleted between when the bug happened and when you were told about the problem, then you may be limited to saying, Wait until it happens again and this time capture the logs before they get deleted.

If the logs aren't adequate to diagnose the problem, the only solution may be to improve the logging.

You can ask the user to be on the lookout for this to happen again and if and when it does to get screen shots or whatever information you think may be useful.

You can always try poring over the code some more and doing more testing on your own, but you indicate that you've done this and it hasn't solved the problem. The only suggestion I can give on that point is: Programs are often bad at testing their own code or the code of other programmers because they go through the steps as intended, while a real user who doesn't understand the design philosophy may do crazy stuff. Just for example, I recently had a problem where for some reason a user decided to type his entire address -- street, city, state, zip -- into the zip code field. We never considered someone doing this when writing our zip code validation function and we got bizarre results. Not saying that particular craziness, of course, but think outside the box. I have no idea what your app does, so I can't give any specific suggestions.

| improve this answer | |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .