184

I’m relatively new at a job, but have established myself as the “guy” when it comes to a certain web service we use. Our own website integrates with the service. I don’t really work on the site, just with the service.

Today we had something of a show stopping issue that resulted in users of the site not being able to use the service. Naturally I was called in to help, but one of the site developers got snippy with me personally, insinuating I wasn’t doing my job and accusing me of poor communication. Later on in a conference call with at least 8 other people, he passive-aggressively called me out on not having resolved the issue.

After a long day, working late and from home, I finally determined the root of the problem to not be in the service, but in the code on the site and specifically in code that the snippy developer wrote.

TL;DR: the guy who was pointing his finger at me was in fact the cause of the problem

I’m generally an easy going person and have no need to tell this guy off; however, I’m hoping for some suggestions on how to tactfully express to him that I’d prefer he was 100% absolutely sure that it’s not his code before he takes that tone with me again.

Improve this question
1
  • 2
    Comments are not for extended discussion; this conversation has been moved to chat.
    – Jane S
    Mar 15, 2018 at 2:07

11 Answers 11

Reset to default
377

Just send an email to everybody involved saying that, after hours, you found (and fixed) the bug. "It is so and so in AB." Maybe with a link to something. Don't mention any person.

That should be enough. You don't have to point a finger at anybody.

The others will look into this and soon they will find out who was/is responsible for that bug, and next time they will think twice before they mess with you.

Improve this answer
18
  • 47
    I'd do this too assuming this was a once-off incident. If it is part of a pattern then defensive escalation might be warranted. For now I'd just keep the notes of what happened ready so that if this happens a few times I can compile a fairly damning report of just how much time you're spending cleaning up this guys mess.
    – Tim B
    Mar 9, 2018 at 9:46
  • 153
    Agree with this. Be seen as the guy who gets s**t fixed and let others play the blame game if they want - send a neutral email explaining where the problem was and those who know the system can figure out who caused it, you don't need to make that point
    – Will Appleby
    Mar 9, 2018 at 9:47
  • 66
    “Because the others will look into this“ depends, many people, especially higher ups, don't look into resolved issued.
    – DonQuiKong
    Mar 9, 2018 at 11:50
  • 88
    To satisfy the need of vengeance I'd slide a "sorry for all the holdup, the reason it took me so long is that the bug wasn't in the code that I am working on but it turned out to be in this other bit".
    – Džuris
    Mar 9, 2018 at 15:36
  • 51
    All of this was super helpful, thank you everybody for chiming in. I think I was a little worked up over the incident so some level-headed responses are just what I needed. I did exactly what was suggested in this answer (which is echoed in many of the other answers, I'm sorry I can only approve one as "correct"): I sent an email linking to my pull request, so the evidence is there for everyone to see. I feel good about not pointing fingers and letting others, if they care to, figure out where the real problem lies. Thank you again.
    – Ryan
    Mar 9, 2018 at 17:07
122

Avoid the blame game

There's always going to be issues, mistakes happen, circumstances arise that means that service X is no longer happy with service Y.

What everyone needs to concentrate on is solving the issue without getting drawn into the blame game. Let the subject matter expert look into and diagnose the problem and work things out from there.

The problem isn't (officially) with someone else - it's a problem with service Y who has its own subject matter expert to sort the issue out.

After that, there's a "lessons learned" to try and figure out what caused the issue.

Not "who". That's the key thing here. If people want to take the blame, that's the honourable thing to do.

In this case, don't retaliate, don't get drawn into the blame game, be thankful that the issue was identified and sorted out.

Improve this answer
10
  • 49
    That's the problem with the "avoid the blame game" meme. What usually happens is one person blames someone and then when someone says "hey, that's not me", the first person comes back and says "let's not get into the blame game". Lots of people in deep denial with flip-charts say nice and fluffy things about communication and "lessons learned" and stuff, but the reality is that you should do the right thing (not blaming others for your mistakes) because it's the right thing (not in expectation of reward), and suck it up as best you can when others don't behave because work pay you money.
    – user74616
    Mar 9, 2018 at 10:21
  • 36
    @Snow You say its pretty immature playground stuff, but if OP's manager gets it stuck in their head that OP can't be relied on, that affects their career. It's a nice rule in general but if your abilities are publicly questioned I think there needs to be some kind of pushback, otherwise it could become a recurring issue.
    – Jeff Lambert
    Mar 9, 2018 at 15:15
  • 4
    A different way of framing it is what's to "blame" is the process. Either multiple people are "to blame", or you have a process where a mistake by any one person can cause a serious issue in production in which case the whole team is "to blame". Blaming an individual for a mistake isn't going to stop them (or anyone else) from making another mistake in the future. Figuring out how to improve the process so it is harder for the inevitable mistakes to become production failures, however, is both a constructive dialogue and of business value.
    – Derek Elkins left SE
    Mar 9, 2018 at 16:33
  • 4
    @snow My usual approach, if I'm blamed by people in the habit of pointing fingers, is just to say where I think the fault lies, let them say "let's not play the blame game", and then get on with life. That way other people can decide. The problem of letting it pass is that you can end up getting a reputation as a doormat among people looking to pass off their mistakes and, when it reaches critical mass, people who don't know what's happening assume that a piece of work is poor quality even if they don't know you, and it becomes lore. Usually an initial challenge is enough to stop that spiral.
    – user74616
    Mar 9, 2018 at 17:58
  • 9
    @DerekElkins After years of working with someone who used to say that, no. Sometimes it is a person's fault. Because especially in software, there are major design flaws, just plain stupid lines of code, and complete and totally wrong understandings of the code you wrote. While we don't need public shaming of any kind, they do need to acknowledge (at least mentally) they screwed up and learn from it. No process can fix everything. Sometimes you just plain have to have competent people, and people don't get competent without taking responsibility for their bad decisions.
    – jpmc26
    Mar 9, 2018 at 23:21
100

Some people have a tendency to aggressively put the blame on others in order to divert attention from themselves. Since you are also relatively new, it must have been very convenient for him to do what he did.

Here's my advice. Write a mail, describe what the issue was, how you reached the root cause, what was the eventual fix for it and how it can be prevented in the future. Include all the stakeholders in this mail.

At the end of this mail, write something on these lines,

XYZ,

Can you please add these steps in the appropriate documentation or Standard-Operating-Procedures document for this piece of code?

This way you are "not pointing a finger" at him, but you are explicitly calling out that he is the owner of this piece of code and hence responsible for it. Calling out is important because not everyone (especially the higher management) will open any codebase links to see whose commit it was.

A bit harsh, but he deserves it.

Improve this answer
3
  • 5
    Best answer until now IMO. The idea of asking for documentation is quite good.
    – Rui F Ribeiro
    Mar 9, 2018 at 15:33
  • 3
    I like the idea of adding note for "XYZ....."
    – user77808
    Mar 10, 2018 at 5:25
  • In most places I've worked, someone does indeed look at the code, especially if there were actual user issues / outages. If not, they would tend to ask someone who has. I don't think calling out is needed, and I'd prefer not to do it, unless it comes up again and I'm getting hassled about it.
    – bytepusher
    Mar 10, 2018 at 13:58
25

I've build a career off of not playing the blame game, but the truth is, humans listen to the loudest voice. However, if you get involved in defending yourself, he's going to pull you down to his level and beat you with experience.

If you can find one, you need a champion. Ideally, this should be your manager, but sometimes, it is another well respected developer. They will go to bat for you to silence the loud one. All you have to do is have a private discussion with them about the facts (not the blame) of what you did to solve the problem and how they would like you to proceed so that, next time something is wrong with the service, the problem can be found and fixed faster. That may involve writing a little testing utility that tests the service directly (without using the other developers code), or some logging or something, so the "us versus them" issue can be very quickly determined. If they know who was actually at fault, they can clear your name without you getting in direct conflict with the loud one.

I've always bent over backwards to avoid putting other developers on the defense. I've said things like "I'm having trouble duplicating the problem, can you please let me know what calls you are making to the service and what you are getting back so I can test the service correctly"? If the developer bothers to give you log traces from the actual calls and responses, ask what they expected to get back. However, most of the time, the developer will just show you their code. In that case, you can sometimes spot the problem. Even if you do, still don't call them out. They have to find the problem themselves. Have them run the code through a debugger, and innocently ask what a certain variable contains at a certain line of code. I could go on, but you get the idea.

Improve this answer
1
  • 5
    Best advice here. Dealing with code is much like dealing with problems with posts here on SE; if you let things get personalized, everybody stops listening to everyone else.
    – T.E.D.
    Mar 12, 2018 at 21:22
18

It's a very fine tradition in the IT industry (and in some others, like aviation) that when someone finds a problem, everyone works together to find a solution, and ideally to find the root cause so that processes can be improved, but no-one gets personal blame or suffers penalties for making the mistake. The result is that people are open and honest about their mistakes, rather than trying to cover them up, which is to everyone's benefit.

It looks as if there are people in your shop who haven't bought in to this culture, and that's something that needs management attention.

Improve this answer
1
  • 15
    OP says there was an 8-people call, with 2 or more developers present. One of topics discussed was poor communication.All that while production was down. Not until the evening was OP able to effectively analyze the problem. Blame game is not the most severe problem of this "culture".
    – kubanczyk
    Mar 9, 2018 at 19:33
12

I believe you should have a talk with your manager on how issues are handled, especially priority issues that impact user experience.

In this case you have 2 systems communicating with each other and suddenly the communication stops working. When communication fails, it is important both parties look at both systems. Everyone put the focus on your service, leading them to not investigate things on their end. The biggest waste of time in resolving an issue is trying to find out what is wrong with a part of it that is actually running as should.

These are, however, learning experiences. I bet you now know perfectly how to troubleshoot your service. Try to set up a troubleshoot plan based on your experience, and make one of the first steps to be sure it's actually your service that is failing ("Does the service work being called from another page?", "Is the service failing partially or entirely?"). You are THE web service guy, you can have a little confidence in your work.

Try to let go of the fact that it was actually his code that failed. Trying to call him out on that is a bit petty. He shouldn't have put the focus on you so much from the begin with. See it as a general issue within the company with troubleshooting an issue and handle it as such with your manager. You also don't know if it was actually his code, he could have also copied it from another section written by a colleague.

Improve this answer
3
  • 1
    Yup, in an ideal world this web service would be covered by a test suite and running that suite would be your first port of call. If the tests all pass, then you're looking at (a) client error, (b) service error not caught by tests, i.e., a testing failure, (c) service working as intended but not as expected by the client, i.e., a documentation failure.
    – Ed Daniel
    Mar 9, 2018 at 12:38
  • 1
    That's a good call, I'll have to look into some kind of test rig for the service. It's a pretty popular and robust service, so honestly I don't think it's ever an issue of bugs with them, more of a is-it-configured-properly thing. Regardless, it's my domain so it would be great to cover my bases for the next time this happens. But on the plus side I did learn more about that domain, along with a bunch of "AB's" code that interacts with my domain, so I'm better off for the experience.
    – Ryan
    Mar 9, 2018 at 17:09
  • Copying code you don't understand is inappropriate, even if it is not buggy. And if it is in the same project, copying it instead of calling it is inappropriate.
    – WGroleau
    Mar 17, 2018 at 4:41
4

I think the suggestion to talk with your manager is a good one. Your manager needs to know that you were blamed unfairly.

But on top of that, you are being tested. Your initial instinct is right; you have to respond, or it will get worse. I'd email him directly and let him know that the problem was in his code, and that for this time, you haven't told anyone but your manager, whom you had to tell in order to protect yourself. And lastly let him know that if he does it again, you will go public.

Improve this answer
3
  • 1
    This will just escalate the problem, I disagree with your advice
    – reggaeguitar
    Mar 12, 2018 at 18:37
  • 2
    A private attack is still an attack, and yes it will escalate the situation. Corrective action should follow the chain of command.
    – Phil M
    Mar 12, 2018 at 22:02
  • 1
    More of a "proportionate response". I've had situations like this; if one doesn't respond, it has a tendency to get worse. The chain of command thing might work. Maybe or maybe not. What I can say is that from personal experience, I've seen it fail twice and succeed once. The trouble is that while waiting to find out, things can get quite a bit worse.
    – William Jockusch
    Mar 13, 2018 at 2:09
4

As a software tester, I'm familiar with this situation. And also as a software tester, why wasn't this bug picked up in testing? I'm assuming you're a developer?

Yes, it sucks that you've been blamed by the guy that caused the issue. However, this is NOT what the business is going to care about. All they want is a resolution where no more show-stoppers make it to the live site.

I would wait until the next conference call if you have them regularly, and mention that you've analysed the root cause of the issue and objectively state you findings. If asked who wrote that code say who it was, otherwise do not point your finger at the guy who did it. You should be more mature/professional than that and you will get a lot more respect if you handle the situation in such a way.

As a resolution to stop this happening again, query testing as to how this slipped through in an non-accusatory way, and work with them to implement a new step or test that would have picked the issue up.

Finally, report all this to your manager. Show stopper defects are a big deal and his higher ups will be breathing down his neck too.

Improve this answer
2

I'm a little late to the question here, but alongside the very sound advice in Edgar's answer, there is second part to this.

If you send out a communication that you found the problem and note where it was solved, then the other developers will probably be aware where the issue lies (this is good), but management will probably not.

Doing it like this means that you've done this guy a favour - he publicly called you out, you found the issue, fixed it and didn't drop him in it with management. Building up trust like this with your co-workers will stand you in good stead in the long term.

Slight side note - this obviously depends slightly on the nature of the fault you find in their work. If what you find is so egregiously wrong that it indicates incompetence on their part, you may want to quietly escalate this to your manager - they will want to know and you are building trust with them as well.

Improve this answer
1
  • If you don't expose him, others probably will. Unless they are all his "buddies," in which case you are history in spite of your innocence.
    – WGroleau
    Mar 17, 2018 at 4:44
0

I suggest you sleep on it before responding to the personal aspects of the situation. If you fixed the problem and got things running again then you are still “the man”. After getting some rest it will be easier to be gracious.

Improve this answer
0

While the main issue was already addressed at length in other awswers - the other guy harassing you - I'd like to point out another perspective.

The fix was done in the code the other guy owned, yet more often than not the service could have prevented a bigger issue by being more conservative at handling requests from clients. Does it validate inputs? Does it reports any runtime errors? Is there a testing environment (this applies to consumer as well)? Sometimes an issue in the service was only waiting to surface, so I'd say just because a fix was done in one place it doesn't mean that is the whole story. Also, for issues like this, there's more than just code. There's process as well.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .