8

I work as a software engineer at a company, that has an outdated build infrastructure. Things like continuous integration are practically impossible. We don't use Git BTW.

There is a company with a huge code-base and anyone can deliver to the repository regardless of how broken the build is. As you might imagine, when it's time to create a labeled build, it usually takes several days. Trying to fix the build as other people are breaking it at the same time.

Right now, there is one brave person, that tackles this problem by crawling through the build logs and then calls the people responsible and asks them to fix their errors. He basically does what continuous integration does. This can take the whole day of that person.

I was asked to be this person. To me, this seems like something that isn't and shouldn't be a part of my job. Skip the fact, that it is a very annoying job. It is basically managing other programmers and telling them to fix their errors. It feels like it has nothing to do with software development process.

Finally my question: Do you think, that this is a reasonable thing to ask from a software engineer? Have you ever come across something similar?

Edited question: As many people correctly mentioned, my original question is opinion-based. Therefore, I will alter my question to make it answerable. If I strictly didn't want to accept this role, is there any legal way of me refusing without resigning?

  • 2
    @sf02 I would continue with other work. This activity can easily take up all day though. Either way, I don't think, that this is relevant. – FMx Jan 6 at 18:48
  • 5
    We don't use Git either. We use SVN, which is fine for us. And I introduced nightly builds here gradually, one step at a time. When I started, we had no coherent build and deploy process. Now we have. Before setting up a CI server environment, start by standardizing the (local) build process. – Doc Brown Jan 6 at 19:08
  • 3
    Voting to close as the asker's insistence of receiving an opinion on this description combined with their rejection of suggestions describing how a software engineer faced with such a need could improve the situation make this not something that can be practically addressed, but rather purely a prohibited question of opinion. Solving the problem (and doing it the hard manual way as that evolves) is productive, arguing about who should be stuck with its unsolved version is not. – Chris Stratton Jan 6 at 19:31
  • 2
    @ChrisStratton I think it can also be easily salvaged if OP asks what I think he really wants to ask - whether he has to do the job, and is there a way out of it, and going back to what he considers software dev tasks. – Tymoteusz Paul Jan 6 at 19:33
  • 7
    As a frame challenge to your question, I would suggest that while your attitude seems to be that this is work that is beneath you, perhaps you can instead see it as an opportunity. Being the one that "manages other programmers and tells them to fix their errors" can be a very powerful position, and (ironically) it sounds more like it's giving you a high degree of control and ownership over the final product, versus simply being busywork. Not to mention the value you can add by helping others fix things, and the problems you will learn to avoid in your own code by seeing other people make them. – dwizum Jan 6 at 20:10
11

Reasonable in general? It is neccesary. Something you want to take on? Questionable.

This is a combination of manual QA, DevOps, admin work, and trying to change a culture where devs commit with reckless abandon. Nobody cares about quality to the point where they can't even generate a build so they have designated you as digital janitor to clean it up.

It would be one thing if this is a proving task to see if you can manage the team. If you are up for a promotion in the company, then go and do it. But you are being used as a substitute for management, tools, and basic testing. Numerous people don’t give a damn so here is a mop.

It feels like it has nothing to do with software development process.

It does, only because the company is using people to do what machines have long done.

Here is what you need to decide:

If you are staying long term, do the task and try to do CI, even if it doesn't really work. It is difficult to avoid if you don't plan to leave within a few months.

If you can leave easily with no consequences, just be incompetent at it and someone else will be assigned. Expertise in it will mean that they have you do it more and you will spend even less time on development. Spend the time cleaning up the good old resume.

| improve this answer | |
  • The sad truth. The better you get at cleaning up other people's crap, the more they'll get you to do it, because other people can't even clean up their own crap. – Nelson Jan 7 at 8:16
  • This is like some sad, outdated, horribly inefficient version of code review. Companies like this scare me, especially if they are holding sensitive data. – Josh Jan 7 at 16:22
  • @Nelson unless you want to be a highly paid project rescue consultant, that seems like a dismal existence. – Matthew Gaiser Jan 8 at 4:18
  • @MatthewGaiser if a company is too cheap to higher QA staff, they won't be paying for the consultant anyways. They'll just go bankrupt. – Nelson Jan 8 at 4:49
37

Do you think, that this is a reasonable thing to ask from a software engineer?

Yes. You've been asked to drive and maintain quality of the software, it's perfectly in line of what is expected from a developer, even if thankless.

Have you ever come across something similar?

Many times. And what I did was take ownership of the quality and implement automated checks that will yell at the owner of broken code. This is your chance to shine and instead of complaining about lack of CI, actually implement it.

Things like continuous integration are practically impossible.

I don't buy this for a second. If a person can browse logs and throw emails at people, so can a script. Start small, with something extremely basic like linting every commit, or just checking if it builds, and build up from there. It's a process, and apparently you are now IT to start it.

| improve this answer | |
  • 2
    We use a very exotic versioning system which makes it really hard to do what you say. There is very little documentation and support. Either way, nobody is going to allow me to do this, because it would mean, that I would have to do DevOps fulltime. – FMx Jan 6 at 19:16
  • 7
    @FilipMinx we are already down from impossible to really hard, that's infinite progress in the span of 28 minutes. I am not sure what answer are you looking here, after reading some of your comments if it's on how to do the actual integration, it's off-topic here. If it's on "how can I get out of this assignment" then amend your question and ask that. – Tymoteusz Paul Jan 6 at 19:19
  • 5
    I never asked how to do any integration or how to implement CI. I only asked for an opinion on whether this is a reasonable thing to ask from a software engineer. – FMx Jan 6 at 19:27
  • 2
    @FilipMinx Opinion, as helpful it would be, is off-topic on this site. So, you would need to really be clear which question you are asking - Tymoteusz offers two options, but perhaps your question could be different; but not for opinions. – Captain Emacs Jan 6 at 20:58
  • 2
    @FilipMinx What do you consider "very exotic"? Something home grown? Do you just mean non-git? If its commercial, I think there is a very good chance someone here has experience with it whether its just old (rcs, cms, svn), commercial (AccuRev, Perforce), or both (ClearQuest). If its got a command-line interface, it can be scripted. – bluegreen Jan 7 at 17:57
5

It's perfectly reasonable to ask you or any other dev to be that person. It doesn't matter, if this task is annoying or frustrating, there are always tasks at work which fall into that category.

It's also your chance to start implementing a CI environment. My suggestions:

  • start with setting up a build server; if there isn't a actual build machine, you can start on your own machine. My recommendation is www.jenkins.io as there are plugins for everything and you find a tutorial or howto for almost every task

  • gradually add more features to the build server, e.g. unit or integration tests, code coverage, code analysis, packaging/deployment, etc.

  • a migration to Git is not always necessary, it could help, but that's probably the biggest change many teams may want to avoid

| improve this answer | |
  • We actually have all that. Except we use a very exotic versioning system. None you have ever heard of. All of this doesn't stop people from delivering breaking changes. The company is actually working on migration to git but this takes a long time. I have a very specific role and nobody is going to put me in charge of DevOps. – FMx Jan 6 at 19:15
  • @FilipMinix - I bet I have heard of it. – Donald Jan 7 at 5:49
  • @Filip Minx: Serena Dimensions/PVCS or ClearCase? – Simon Jan 8 at 6:02
3

Firstly, in theory, this is reasonable to ask. Which is to say ensuring code quality needs to be the responsibility of someone. Ideally, of course, it needs to be the responsibility of everyone, but it seems like your company is screwed up, so they'll settle with "someone". In this case, you are "someone" and that's why you got chosen.

My question is, why is CI an impossibility for your company? At the very least, what if you write (or tell others to write) unit tests for their code to make sure their code is correct, and to auto-run those tests before a deployment? Do you have a single person (or O(1) sized group of people relative to your O(n) sized organization) responsible for deployment? Those people can be instructed to run the unit tests before deployment, and reject the build if the tests fail. That's kind of like doing CI manually, and is better than nothing (it's definitely far from ideal but it's better than what you currently have).

How much power do you have in this company, and how open to listening to dev concerns is management? If you went to management and said "this is unsustainable, there are too many problems and nobody is fixing them, our software is constantly breaking", what would they say? Could you get leeway to prioritize, at the very least, fixing the existing problems and adding unit tests at least for those fixes to make sure they don't present in regression? And could you get management on board with penalizing developers who refuse to adhere to those practices? You should have a meeting with management (or at the very least with the person asking you to take on this role) and ask them these questions.

As for code management, you said you don't use Git but you do have a repository. There are plenty of companies which use alternatives to Git, like Subversion, and work just fine. If you don't have a good reason why to use specifically Git, that's not a particular thing to be concerned with.

| improve this answer | |
  • The reason why we have to settle with "someone" is that "everyone" does not care. Engineers often don't check if their commits broke anything. I don't even blame them. We work on many streams simultaneously and sometimes you just forget. That is why you need an automated tool to alert you. We use RTC as a repository, which has zero integration with modern tools. I have zero power. The company understands the need for this and is actively working to make things better but because of the huge system, it is very hard. – FMx Jan 6 at 19:06
  • @FilipMinx In a very large system it is difficult to categorically identify one by one if your commit broke anything. That's why you unit and integration test, because your testing should cover your common use cases and if those aren't broken then your commit is probably fine. If you unit and integration test your code, the it should be the responsibility, at the absolute least, of the original developer, the code reviewer, and the deployment specialist to make sure those tests do not error. If you don't test now, start doing so ASAP. – Ertai87 Jan 6 at 21:14
1

Are you the new hire? If so, it could be a rite of passage or form of hazing, so to speak. You get the bottom job and you build your way up if you stick around.

My thought: this isn't something great. I'd just say it's probably a high turn over position if they stick you with this task. Most likely you're in this position because the other guy wants to do the "cool stuff."

What I think you should do: quit now, while you're ahead. I'm not sure why people of this forum tells folks to stick around and attempt to change things. We're not working for NASA solving world problems or medical devices. There are an abundance of other jobs that offer challenging roles that don't involve convincing people to do things right. Ask any AA person what's the first step to solving an alcoholic condition and they'll tell you that you should admit you have a problem and need to solve it. If your workplace is unwilling to admit they have an issue and stubborn to see it, then you certainly aren't going to be "that guy" who did it. You don't need to work on the latest, super cool guy framework nor do you need some great CI process to release products. People have been doing it since the 80s with great success and doesn't involve days to prep a release figuring out who did what. There were nightly builds around for decades before GIT or anything else we see today. A simple branch or tagging will do just fine with most SVN releases. If your workplace is so disorganized that it can't do that, I'd say it's not a good idea to stick around.

| improve this answer | |
  • 2
    A decent case can be made for heroics if they are going for a promotion, have a stake in the company or if this were a one time extraordinary situation, but this is regular scheduled janitorial work which is clearly unwanted as someone had to be asked to do it. – Matthew Gaiser Jan 6 at 19:50
  • 1
    @MatthewGaiser Not only that it sounds like coworkers regularly commit broken code into the SVN. So if you svn update, you might be spending an entire day figuring out what broke and why then going to that person. Makes working tough and harsh than what the OP listed. I imagine as your coworkers do other tasks, they might ignore you for days or even weeks before coming back to fix whatever. Then when you release, you'll again suffer broken code and who knows what. – Dan Jan 6 at 19:53
  • The committing of broken code makes this a management level issue. Evidently they can't be bothered. – Matthew Gaiser Jan 6 at 20:10
0

You're unlikely to gain much sympathy from this group. Many of us started before CI was a thing. Our "CI" used to be two guys manually running tests for 3 weeks. Many of us have slowly automated these sorts of problems away over the course of years. It can be done, and it can be done at the grassroots level, but it's a slow process.

Our small team set up our own CI server before we had an "official" one. Our success with the experiment along with other teams is how we were able to convince management to fund an official one.

I had my own personal git repo on top of our "exotic" VCS when we were having problems with constant broken builds similar to what you describe. That allowed me to hold broken changes out of my stable branch when I needed to get work done. My success and the success of others on those sorts of experiments is one of the reasons we use git officially today.

Things are a lot better for us now, but it's trickier changes like dependency updates and flaky tests that break the build now, and people still don't always realize when it's their fault. I still regularly and voluntarily take a day to be the "build police" to help my team have a clean build to finish our features on top of, even as I look for ways to improve the automation so the policing isn't as necessary. Most everyone with any seniority does. We don't think it's beneath us, and neither should you.

| improve this answer | |
0
  • Yes, I have come across something similar
  • I assume you use another version control than git
  • My tip: Change it gradually. Deal with the worst things first. Typically people can be convinced to change
| improve this answer | |
  • The problem with switching from "a very exotic versioning system", as the OP describes it in a comment, is that they are likely to lose their commit history and be unable to recreate previous builds. (that said, I still feel strongly that they should do it - move to SVN (git, if you must), and introduce Jenkins. Otherwise, get someone less qualified and teach them, how (I can remember when "build master" was a job); but don't waste a develoepr's time on it on a regular basis. – Mawg says reinstate Monica Jan 7 at 7:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .