52

Does producing such a grave bug like Log4Shell make you effectively unemployable?

The people involved have committed under their real names and, taking into account how prominent the bug is, it's easy to assume that every potential employer googling for a candidate's name will learn that they were involved in this incident.

Can such an error be forgiven and, if so, under which circumstances? I'd expect that making a grave error as junior isn't a no-go because juniors are expected to make errors. However, employers wouldn't expect seniors to make the same mistakes.

For non-IT people: This introduced a critical security vulnerability to log4j2:

Affected commercial services include Amazon Web Services, Cloudflare, iCloud, Minecraft: Java Edition, Steam, Tencent QQ and many others. According to Wiz and EY, the vulnerability affected 93% of enterprise cloud environments.

The exploit allows hackers to gain control of vulnerable devices using Java. Some hackers employ the vulnerability to utilize the capabilities of the victims' devices; uses included cryptocurrency mining, creating botnets, sending spam, establishing backdoors and other illegal activities such as ransomware attacks.

Improve this question
5
  • 2
    Comments are not for extended discussion; this conversation has been moved to chat.
    – Kilisi ♦
    Dec 17, 2021 at 11:20
  • 2
    Hey everybody. Comments have already been moved to chat. Please, use such chat room to continue the discussion. Further comments may be removed without warning...
    – DarkCygnus ♦
    Dec 18, 2021 at 0:11
  • For non-IT people, probably the Wikipedia article is more accessible.
    – Andreas Rejbrand
    Dec 18, 2021 at 22:54
  • 1
    This question is being discussed on Meta.
    – EJoshuaS - Stand with Ukraine
    Dec 20, 2021 at 17:09
  • Norrington: You are without a doubt the worst pirate I’ve ever heard of. - Captain Jack Sparrow: But you have heard of me. - [Pirates of the Caribbean: The Curse of the Black Pearl, 2003]
    – A. I. Breveleri
    Dec 25, 2021 at 21:30

15 Answers 15

Reset to default
19

Does producing such a grave bug like Log4Shell make you effectively unemployable?

Absolutely not.

I follow the Edward Deming principles. Systems are to blame, not individuals.

I'd expect that making a grave error as junior isn't a no-go because juniors are expected to make errors. From a senior, however, you should expect the ability to spot the most obvious mistakes.

It was 8 years ago. For all I know, the developer was an intern at the time, but even if he wasn't. Eight years is a lifetime in this field.

it's easy to assume that every potential employer googling for candidate's name will learn...

If anything, this could make an interesting interview topic. So the developer just needs to be prepared to talk about what happened, and what he has learned from those mistakes, in case this topic ever comes up during his interviews.

Improve this answer
1
  • 1
    To add to the interview topic level, it's worth noting that how they respond to the issue is fairly relevant - and if they understood what conditions led to the situation that happened. With that much information ,even the "Three Gateway Protocol" that BGP is infamous for being, can be recognized as an achievement even if Pakistan was able to block YouTube.
    – Alexander The 1st
    Dec 17, 2021 at 6:11
155

Your mistake is thinking that this is a bug. It isn't. It's a feature that someone wanted, and the developer implementing it did so without any bugs that we know of. In hindsight, it was a very, very stupid feature. Actually, insanely stupid - but only with the point of view of the year 2021, when every computer on the internet is under constant attack. In 2000, when the feature was created, nobody did think of it. And the feature was in the open, and for 20 years nobody had the bright idea to exploit it.

So no, there is no problem for these developers whatsoever.

Improve this answer
1
  • 4
    Comments are not for extended discussion; this conversation has been moved to chat.
    – Kilisi ♦
    Dec 17, 2021 at 11:19
85
  • No, the bug definitely can not be compared to that.
  • Missing quality assurance, tests and wrong prioritization are never a single persons fault
  • The impact of a bug has very often nothing to do with how stupid it was to make.
  • Seniors who believe only Juniors make mistakes are more dangerous.
Improve this answer
4
  • 7
    FYI: the comparison OP made has been edited out of the question.
    – Lilienthal ♦
    Dec 16, 2021 at 21:30
  • 14
    This is not even a bug. It's a feature that should have never passed security review, if they even had one as part of the process.
    – dbkk
    Dec 17, 2021 at 17:37
  • 2
    Seniors who believe only Juniors make mistakes are more dangerous. - Nobody actually creates perfect code the first time around, except me. But there's only one of me. -- Linus Torvalds
    – hanshenrik
    Dec 17, 2021 at 20:47
  • 4
    It was a feature that someone wanted. I would be sure that it was tested properly, was found working, and was used. There was no bug that testing would have found. What was found years later was that the feature could be abused to create massive damage.
    – gnasher729
    Dec 19, 2021 at 11:30
46

Could producing a grave bug that made headlines make you unemployable?

If all you care about is what bugs/mistakes a candidate has produced then maybe.

Or, if like most normal hiring managers, you look at the totality of their knowledge and experience and allow them to explain any mistakes in their past, it is much less likely that the candidate is "unemployable".

A proper analogy would be any coach/player of a professional sport making public mistakes (usually on live television to audiences of millions). They somehow seem to be employed after making those mistakes.

Improve this answer
5
  • 4
    Stack Overflow itself could be another example. If you hang out on Meta Stack Exchange, you'll sometimes see on bug reports specific employees posting an answer that includes something like, "This was my mistake". (I can't recall if an individual has ever tried to take responsibility for anything that would be considered a larger issue but they don't seem to fear publicly admitting they made an error.)
    – BSMP
    Dec 16, 2021 at 19:53
  • 2
    Also, having made a mistake you're possibly more likely to avoid something similar compared to someone who hasn't.
    – camden_kid
    Dec 17, 2021 at 9:42
  • 3
    To continue the sports analogy: The receiver missing a catch for the game winning touchdown. The pitcher giving up the game winning home run. The goalie not blocking a penalty kick. These are all mistakes that can cost not only a single game but a championship deciding game. The players remain employed under their current contracts and can often get paid more at their next contract negotiation. There was a great Nike commercial with Michael Jordan listing how many game winning shots he'd missed - ended by saying "because I missed them, I am who I am today". [con't]
    – FreeMan
    Dec 17, 2021 at 18:07
  • 2
    Failure isn't the end unless you let it be the end. If an employer is so small minded that they see one failure in your past and totally discount all the good you've done, they're doing you a favor because you wouldn't want to work for them anyway.
    – FreeMan
    Dec 17, 2021 at 18:08
  • 1
    The sports analogy is faulty. In sports, one team is expected to fail. A proper analogy is when one's actions cause significant financial loss or death, e.g. a stock trader who loses hundreds of millions, or a pilot who crashes a plane.
    – user71659
    Dec 17, 2021 at 22:05
27

It's important to note that log4j is NOT a bug. It is a deliberate feature.

The developer didn't make a small error. They didn't make a typo. They didn't forget a corner-case.

The implemented probably the most dangerous feature they could have envisioned, and turned it on by default.

The fact is, it would probably have been better if it was a bug, because at least then we would have known it wasn't intentional.

The person who implemented this feature refers to themselves as a "security software engineer". If they were applying for a job, I would want to understand what that thought process was.

Did they consider the wide ranging implications of their change? Did they consider the risk factors? Or did they just implement the feature as asked, without any consideration to impact on the many users that would be exposed? Did they just consider it not their problem to deal with the fallout? Was there malice involved?

Improve this answer
1
  • 1
    Comments are not for extended discussion; this conversation has been moved to chat.
    – Kilisi ♦
    Dec 18, 2021 at 10:46
24

I'd just figure you'd never make THAT mistake again.

I like to say, "I don't lose, either I win, or I learn"

Truth be told, most people are like that. A mistake, even a high profile mistake is not an overall representation of the person, their skills, or their ethics.

At my company, we would go right to a post-mortem and figure out what went wrong and how to prevent it. It's usually a mistake on several levels, and it is more important to prevent it from happening again, than it is to assign blame.

So, nobody would be unemployable for a mistake, but it would close a few doors, and quite frankly, they are doing you a favor, because a company that would not accept the fact that people make mistakes is not one you really want to work for.

Improve this answer
3
  • 2
    To be fair though, it's possible to learn from other people's mistakes. It's how we move forward as an industry.
    – Gregory Currie
    Dec 16, 2021 at 22:41
  • @GregoryCurrie It's possible to make mistsakes nobody else has? It's how we learn from our mistakes that we move forward as an industry.
    – Old_Lamplighter
    Dec 16, 2021 at 22:52
  • 3
    This is not a mistake that nobody else has made. This is a very obvious problem for anyone that is even slightl knowledgeable about security.
    – Gregory Currie
    Dec 17, 2021 at 0:17
11

No, because it requires that you made something important

In order for a bug you created to hit headlines like, you first have to make something that is used widely enough to be important. Log4j's bug is important, because log4j is used by a large number of important projects who chose it because of its quality.

So, yeah, you take a hit for the bug but that doesn't really measure up against the bigger factor of being someone who is capable of creating such a project in the first place.

Improve this answer
0
6

An extreme example: Kevin Mitnick is employed as a security consultant, and he was jailed for hacking.

So, no: mistakes made do not necessarily affect your employment prospects, as long as you can show that you've grown past them and that you've got the expertise.

Improve this answer
4
  • 3
    What does this have to do with the question? Did Mr. Mitnick introduce a bug? Did he make a mistake? The best burglars make the best testers of burglar defence systems and virtual burgling (hacking) is no exception.
    – gerrit
    Dec 17, 2021 at 11:23
  • 2
    My point is that it's all about reputation and trust.
    – Roger Lipscombe
    Dec 17, 2021 at 13:52
  • 3
    @RogerLipscombe Mitnick does not have reputation of being incompetent.
    – ojs
    Dec 18, 2021 at 12:59
  • Breaking the law is a mistake, though. At least according to most societal framings.
    – Roger Lipscombe
    Dec 18, 2021 at 14:47
6

Let's speak generally and not in the particular Log4j example which is not comparable to the common "grave bug that makes headlines" issue.

We are humans. We make mistakes. Everyone does. Moreover, common software projects involve several people and not just one individual, so even if a single person could be blamed for the bug, the rest share responsibility with it cause they all work in the same codebase. A colleague at a former job I had, would tell me "there is no bug in my code or your code, there is a bug in the code".

Everything boils down to why the bug happened and how it was handled. If there was intention, which is extremely rare, that is quite a red flag for employment for sure.

More commonly, there will be negligence instead. That person not taking enough cautions on its work to ensure the delivered code was of high quality and secure. In this case, it is just a negative annotation to consider in the full person's record, but even then, how was the issue managed? Did that person involve themselves into fixing the bug ASAP? Did they learn a lesson and now are more careful about not making that mistake again? Even if all answers are negative, you have to see it in perspective and consider the environment this person worked on and the chance that this could happen in your own company if you hired that person, so it is really a hard choice to make.

Then, there will be times where the bug was reasonably difficult to find out, and where all due cautions were taken and still the bug got out. You cannot blame the developer for causing the bug in that situation.

Finally, somebody making these headlines could actually be a hint that this person's previous work had a high impact. It's not easy to work at a project that powers 10% of the whole Internet!

Improve this answer
4

I am firmly of the opinion that for any bug to make it to production, then two things must be wrong. People make mistakes. And therefore, if a single person's mistake can cause a disaster, then there's something wrong with the process. Both the testing and the code review processes must have been inadequate, for a bug to slip through to production.

So either three people are responsible (the creator of the bug, the reviewer and the tester), or the development process is broken. In short, the developer by themselves is never to blame.

Having said that, if I were an interviewer who learnt that the person I'm interviewing was involved in the creation of such a howler, one question I would ask them would be what they've learnt from the experience. In other words, what are they doing differently now, from what they were doing when the original problem arose. Their answer to that question alone probably tells me whether or not I want to hire them.

Improve this answer
2
  • 1
    If you expect that there's a single piece of code on this planet that is devoid of bugs, I'd consider you delusional.
    – vidarlo
    Dec 18, 2021 at 20:23
  • 2
    Right, if your security plan is "don't make mistakes", then be glad you don't fly airplanes for a living.
    – Harper - Reinstate Monica
    Dec 18, 2021 at 22:19
2

The way I see it is that any news is good news. While making a critical error is bad, it shows that 1) you have a community that uses your products, and 2) now you can show you're committed to your product by fixing it and releasing an emergency patch.

It's sort of like when Facebook got hacked and all their user information got leaked. At the same time, all your information is already "sold" so the reality is a leak means Facebook just didn't make the revenue when they would sell the same information to advertisers.

Can you get fired? It really depends on the culture and I imagine if the bug can be fixed rapidly. I don't think a normal company would fire someone on the spot as they'd want it fix. So at least you're safe until the bug gets fixed. But after which, sure you might get into trouble. No way to know and it probably depends on how often you normally do such a bad thing and exactly the mood of everyone in the company.

Improve this answer
1

Producing a bug - no, managing such project - maybe

Developers do make mistakes. Someone is having a bad day, lack of sleep, problems at home, simple oversight ... Lots of things get committed.

However ... on any serious project, even if open-source, after that initial commit, there should be code reviews, testing, regressive testing and other QA techniques. Especially if project in question impacts so many people and other applications. Person responsible for organizing all of this, be it a team lead, scrum master, project manager or whatever ... should be taken under scrutiny if something like this appears.

This not necessarily means that this bug is caused by organizational flaw, but it should be examined how did it manage to slip trough cracks.

Improve this answer
19
  • 3
    This "bug" was actually a deliberate feature, and would have passed all QA.
    – Gregory Currie
    Dec 17, 2021 at 6:22
  • 1
    If this were actually a requirement, people would just not contribute to open source projects. Further, if you aren't satisfied with the quality of software as developed for free by someone else, you're perfectly welcome to create your own version on your own time with your own resources.
    – user3067860
    Dec 17, 2021 at 19:12
  • 2
    @rs.29 The feature is actually bug free and is designed to not be secure.
    – Gregory Currie
    Dec 18, 2021 at 2:44
  • 1
    @rs.29 I don't think it's accurate to call this a part of QA process. I've been in security trainings in some past jobs and per my recollection most reasonable approach (both for QA and developers) is to stay away from security matters and instead do only and exactly what professional security experts tell them to do. You are free to call these security professionals "also QA" if you wish, except that they are not. They are security experts and this is a separate full-time profession
    – gnat
    Dec 18, 2021 at 9:24
  • 3
    QA checks if the software does what it is supposed to do. It did. Normally security problems make the software misbehave. Here the exploit was the designed behaviour.
    – gnasher729
    Dec 19, 2021 at 12:12
1

In this particular case there should be no risk. For one, there is no one bug to point to, everything was working as intended, and known to every user (or even just reader of the documentation) from the very start. Yes, one person asked for the feature; one person implemented it, but many, many persons witnessed it and said nothing.

But more importantly, there is another point of view: as an employee, I would not want to be employed in a company (or below a manager) without a modern/agile error culture. IT in general is very complex, and having a positive error culture is one of the hardest, but also most productive features of an organization.

So, no, there is no risk in committing open source with your real name in this regard - assuming most open source commits are from people who do what they love to do, this is always a great way to demonstrate their knowledge and interests to possible employers. If employees select against that, then this is a useful filter against a combination that maybe would not have worked out anyways.

Improve this answer
1

producing such a grave bug

Software development is a team effort. In my opinion, the developer didn't produce a bug; they implemented a documented feature instead.

Usually, in a product development organisation, when a developer is finished with an implementation, they send it off to QA for review. So, instead of blaming the developer, I would actually point fingers at the quality assurance process -- or perhaps more appropriately at the product owner. Whose idea was it that a logging framework should support automatic object deserialization via JNDI? What does that have to do with logging?

Every time you implement a marginally useful feature, you introduce a possibility for bugs due to the increased complexity. Keep things simple, YAGNI!

Improve this answer
1
  • Being able to look up strings in JNDI is a simple thing. They just missed the ldap feature.
    – Thorbjørn Ravn Andersen
    Dec 19, 2021 at 12:04
-1

Quality Assurance is usually the responsibility of the company or maybe the team. Very rarely the individual who causes the bug (if it can even be traced to a specific person).

Also, they would definitely not be unemployable. There is a lot of demand for bugs these days.

Improve this answer

Not the answer you're looking for? Browse other questions tagged .