56

I'm on a team with a senior developer who has been with the company for a long time (maybe 15 years) and works exclusively on the backend architecture of our 25-year old product. Recently, as part of a modernization effort, our team was tasked with updating the product so that web applications can talk to the product using JSON. To solve this problem, this developer wrote a custom translation layer that transformed messages from web clients over HTTP (JSON) into messages the product can speak and vice-versa.

In the beginning, this layer was fairly simple - it just translated messages back and forth between the web clients and the product. However, over time, this layer has become more and more complex; this layer now can transform messages, run message validation, make database queries, make network requests to 3rd party systems, loop over data, execute complex conditionals, and more. It has grown to the point that I think it's accurate to call it a full-fledged programming language.

We are the point where all new business logic is written in this translation layer. I feel that this is a Bad Idea™ for a number of reasons:

  • There's very little documentation on how to write code in this translation layer
  • As a result of #1, there's now a very steep learning curve for anyone joining the team
  • Developing new business logic almost always requires assistance from this developer
    • In fact, each sprint we give him "API support" tasks on our scrum board that are dedicated to helping other team members through issues while developing in the translation layer.
  • The development experience is poor, since all our development tools need to be written by this developer

Perhaps the most infuriating part is that we also have a Node server in front of this translation layer. However, writing business logic in this layer is discouraged by this developer (and our architect, who is a bit disconnected from our team) primarily for consistency - he doesn't like the idea of having business logic in two places.

How can I convince this developer (and the rest of my team, who are fairly passive about these kinds of issues) that this is a bad idea? How do I combat the inevitable argument that writing business logic in Node would be inconsistent with the patterns we've already established?

Or, is it possible that I'm being arrogant in assuming that this developer's approach is wrong? He has more years of development experience under his belt and has a lot of institutional knowledge.

Some background: I have about 5 years of experience, 2 of which have been with this company. I lean more towards front-end development if I have the choice. The translation layer mentioned above has been developed on for about 2 years, but has not yet been deployed to production. See the comments for some additional technical details about this translation layer.

| improve this question | |
  • 86
    I'm confused as to it's description as a programming language. This sounds more like a framework. – Glen Pierce Apr 18 '18 at 14:55
  • 26
    This is classic Inner platform: en.wikipedia.org/wiki/Inner-platform_effect. If the architect can't / won't stop this, then it's settled. Live with it or move on. Fighting it is pointless. – Wesley Long Apr 18 '18 at 15:18
  • 37
    It is not a programming language unless it has a compiler or interpreter. It sounds like a program that is maybe doing more scope than it should. – paparazzo Apr 18 '18 at 17:59
  • 6
    The term I would expect to apply here would be "domain-specific language", and they're a pretty well understood way of approaching complex domains (which it sounds like this might be): en.wikipedia.org/wiki/Domain-specific_language They also have fairly well-understood disadvantages (which the wikipedia link discusses). Often, systems are more complex than they might at first appear, and it's quite possible that this DSL is a great tool for managing that. Which might not outweigh the problems, but I would be hesitant to go with the assertion that it's necessarily a bad thing. – autophage Apr 18 '18 at 19:21
  • 3
    To add some detail regarding the translation layer: the layer is written in RAML. The layer implements a series of "plugins" that are accessed through RAML Annotations. In a RAML file, each endpoint can define a number of "steps" that define how the endpoint behaves (i.e. validate the object, make a database query, transform the result, etc.). The developer has written a custom parser that consumes these RAML files for execution. TL;DR: All business logic is implemented exclusively in RAML files. – ThrowAway Apr 19 '18 at 13:19
80

Let's give your question some context.

Your company has:

  • A 25-year old product
  • A senior back-end developer with considerable institutional knowledge
  • Assumption: Several mid/junior developers
  • A desire to modernize the environment
  • A translation layer

Non-comprehensive list of translation layer requirements:

  • Message transformation
  • Message validation
  • Database connectivity
  • Network integration with 3rd party systems
  • Data handling
  • Complex conditional execution

So this translation layer can be called instead integration middleware. It handles business logic unrelated to the underlying 25-year old product, with (I expect) different flows and paradigms.

There are several aspects that I would call Bad Ideas™, but the existence of an integration middleware isn't one of them. The worst, IMHO, is lack of documentation: It may originate in the fact that you have only one developer with enough experience to visualize the whole process. He should document the process so if he gets hit by a bus the development doesn't come to a screeching halt. It'll also lessen the need from the devteam for API support tasks.

About the Node server - I'm forced to agree, at least initially, with your architect. Splitting business logic in different platforms/languages without a strong architectural reason can be called a Bad Idea™. Two codebases to maintain, two sets of requirements for new job positions. KISS principles may have been applied here to reduce costs.

I'm not saying that this decision shouldn't be made, but all costs (re-training, process mapping, flow validation, etc.) should be taken into consideration, and your architect is probably aware of them. The language used to code the legacy product may be discontinued, for example, forcing a full migration to a different platform, and a full NodeJS implementation could be chosen as Plan A.

Still, I think that the desire to improve the current situation is admirable and valuable, and instead of fighting an uphill battle I'd suggest a different plan.

The fact that the senior developer is leading the middleware effort suggests that he isn't adverse to change, and a capable professional. Share your point of view with him and listen to his feedback. He may appreciate the fact, offer his own point of view (which will help you understand the rationale for the current status) and consider your desire to modernize the solution while taking into consideration the whole institutional knowledge - and come to the conclusion that the idea isn't without merit. The company relies on his capabilities, and a modernization suggestion from him may carry considerable weight.

| improve this answer | |
  • 18
    +1. The best solution to this seems to be to document the shit out of what exists. – ESR Apr 19 '18 at 0:07
  • 2
    +1 for documentation. Regardless of the solution you use to meet your requirements you will still have some sort of API or framework for your developers to learn. – rooby Apr 19 '18 at 0:53
  • 3
    Since lack of documentation is the big culprit here, and the OP is reasonably experienced, perhaps they can approach the senior developer and ask to write the documentation for the middleware with the senior developer as mentor. Even documentation written only for the parts you use as you use them, has merit. – Logan Pickup Apr 19 '18 at 3:17
  • 3
    @LoganPickup I agree. The documentation need not be written by the original developer. Heck, most developers I know (myself included) aren't really good at writing it anyway. It always helps to have somebody else help out - at the very least, they can offer insight into what's important to document. – VLAZ Apr 19 '18 at 3:27
  • 2
    +1 Another vote for documentation. To add to this, documenting processes like this force the developer to learn how to communicate the process to someone else not at his level. This usually reveals a lot about maintainability inherently in the implementation, how intuitive the processes and implementations are etc etc. To me, if you present parts and people are making the wrong assumptions about how it works, then your design is likely flawed. This would be great for reviewing the architecture and how to improve it. – Shiv Apr 19 '18 at 4:56
58

Full disclosure: I have been the Senior guy you are complaining about, in the past. I have been, not by choice but by necessity, the primary architect and developer of a framework strikingly similar to what you are describing.

Let me tell you that I was aware of all technical and organizational problems during the whole time. It was an incremental process; there never was that one single point in time where one could have made one single decision to stop this. Almost every single step seemed to be a good idea at the time.

There is a mode of thinking, and certain other aspects (e.g., time pressure, perceived pressure, team culture etc.) which leads to solutions like this. It does not tell you anything about the intelligence, character or any hidden agenda of your developer.

I ended the mess by these measures, which literally took years:

  • Pick a whole new programming language and web framework plus module ecosystem to replace the antique languages and our custom framework we had up to that point.
  • Evaluate and learn all about it in my free time.
  • Implement a small fully paid customer project as proof-of-concept with that tech, making sure to use only approaches which are really standard, general.ly accepted ways in that ecosystem, to make sure that new people (having previous knowledge of the language, but not our domain) have a running start.
  • Find ways to run the new solution aside the old, without falling into the middleware trap again (i.e., loosely coupled through microservices or indirectly through the client browser.
  • Slowly and laboriously convince the very conservative team that it is a GoodIdea[tm] to not do new projects/features with the old framework, but take the plunge into the new. This was very emotionally draining.
  • Reorienting my own role in the company, eventually leaving that team and building a new one with a different angle, to cut all ties to the (technical) past (part of the problem was that I was a go-to problem solver in that old team, and had to force them to stand on their own feet by leaving).

It was a fun ride, and, in hindsight, very successful. The team is now well on its way to leave that era behind. But the approach was quite non-trivial as well. Maybe it gives you some ideas, and I daresay you will either need the wholehearted support of your Seniour dev, or remove him from that position (in whatever way open to you). Sit down, and have a long talk with him, if you're in any position to do so, the "where do we want to be in 5 years" type of talk...

One note, which cannot be stressed enough: The existing system can be great (actually it probably was, to be able to survive so long), but there will be organizational problems surrounding the approach. There's a huge problem with documentation, general know-how distribution, truck factor, and having a very long learning phase for new colleagues. Plus, many, especially younger developers, have not much interest to invest a lot of learning into a very specific application context which will not help them at all with further projects. Reasons like this were (in my case) what lead to the described change, and speaking openly about them takes the personal sting out of the issue.

Further reading: Per the comments, I seem not to be the one to have invented this process. :-) See the StranglerApplication pattern for some more thoughts on the subject.

| improve this answer | |
  • Precisely. There's a lot of institutional inertia in situations like that, and it takes time and effort to change direction. +1. – OnoSendai Apr 18 '18 at 19:59
  • 4
    This assumes that the existing system is actually bad and should be replaced. Based on the information in the question I wouldn't immediately assume that is the case. If that is the case though then this is a good solution. – rooby Apr 19 '18 at 0:50
  • The process you described sounds a lot like the "StranglerApplication" pattern. – tavnab Apr 19 '18 at 2:14
  • Great find, @tavnab! An alternative route is to gradually create a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled. Doing this sounds hard, but increasingly I think it's one of those things that isn't tried enough. => perfect. – AnoE Apr 19 '18 at 6:50
  • 2
    @rooby, on the contrary. The existing system can be great (actually it probably was to be able to survive so long), but there may be organizational problems surrounding the approach. There's a huge problem with documentation, general know-how distribution, truck factor, and having a very long learning phase for new colleagues. Plus, many, especially younger developers, have not much interest to invest a lot of learning into a very specific application context which will not help them at all with further projects. Reasons like this were (in my case) what lead to the described change. – AnoE Apr 19 '18 at 7:28
10

Looks like that dev cornered a market. Nice bargaining tool for year-end review salary negotiations.

IMHO, business logic does not belong in the translator, You can suggest a division of that translator to layers, separating a business logic from simple translation methods.

Be prepared for that dev to be against it ;)

Also, management can side with him, stating the extra effort / cost it would take.

It can be lengthy battle, and you could be the one doing the separation in the end ;)

| improve this answer | |
4

How can I convince this developer (and the rest of my team, who are fairly passive about these kinds of issues) that this is a bad idea?

It's not your task to "convince" the developer.

Basically, the main question here is about your position:

  • if you're just a colleague on the same level or below, you can flag the risks to the management. While doing that focus on the problems that can arise from the current system. It's then their decision and not your problem;
  • if you're the boss, align with other managers on the need to change the programming system and then communicate the decision to the programmers. The decision should be justified and communicated correctly (google "change management) and a transition should be organised, but then you should expect your subordinates to adjust to it. You may offer some buy-ins to the senior programmer in order for him to stay with the company, e.g. a salary increase, more days off and some catering to his ego.

When working for a company it's important to develop the ability also to detach oneself and not care. In a positive sense! If you're trying to change something nobody else wants to change, you will end up hated by the colleagues and the management.

So focus on what your task is. Flag and inform - but let people who are in fact responsible for taking decisions take them. It doesn't matter if you are right - I know plenty of people, myself included, who ended up fired despite being right - or because of being right.

| improve this answer | |
  • 2
    The "if you're the boss" paragraph is really bad advice. A manager who makes technical decisions and demands the team accept them is an incompetent manager who's not worth working for -- even if the decisions are sound from a technical standpoint. – ruakh Apr 18 '18 at 20:21
  • There is some good advice in here. The detach oneself was difficult for me but @Toss is very correct. Managers and co-workers become detached from the employee who keeps pushing their agenda. +1 sir. – JonH Apr 19 '18 at 14:01
2

Edit: Don’t quit your job; I had a misconception

Wow, when originally reading the post the technical solution sounded like a translation layer which needlessly performed unrelated tasks. Sounded very contrary to the “single responsibility principle” / UNIX philosophy, not very cohesive or well structured.

I think the key detail I missed was that this is an archaic back end system running the core business logic, and I think it’s implied that the business is stuck with it for some understandable reason. Thanks to the comments and down votes for pushing me to recognize that.

Since it’s ancient, the company probably doesn’t want to touch it. Or, it’s possible that the business logic implemented in the translation layer is completely unrelated to the purpose of this core back end.

Other comments are right; documentation is in order

The decision to write a translation layer sounds reasonable now that I feel I have a more accurate picture of reality in my head.

What you’re describing doesn’t sound so much like a new programming language as perhaps a new framework, specifically written for this use case. Sounds like pretty standard engineering in what I now understand to be the scenario.

You need to get familiar with the code, and you and your peers need training and documentation to understand the system. It would be efficient to write documentation about things as you learn them.

| improve this answer | |
  • 11
    This seems like an overreaction to a situation which is basically pretty normal in largeish organizations -- mostly comes down to "nobody wants to write documentation" from what I can see. – jkf Apr 18 '18 at 19:34
  • 3
    The "translation layer" is not necessarily bad. We have no data on that, only the OP's opinion. It's entirely possible that it is a good, solid piece of engineering (which needs better documentation), and the much junior OP is just showing a bit of "but that's not what they taught us in school" attitude. – jamesqf Apr 18 '18 at 22:47
  • Sincerely: thank you for the down votes and comments. I misunderstood the full picture. Edited my answer in suit. – Paul Statezny Apr 18 '18 at 23:59

Not the answer you're looking for? Browse other questions tagged .