The company I currently work for has a terrible amount of processes that require manual work, thus they are very limited in the amount of projects they can take, as their processes do not scale up, they have reached a point where they have to reject projects. These processes largely depend on manual setups, manual execution and writing Excel and Word documents and the like, an example of this, one of these processes involves filling more than 800 cells in a spreadsheet manually, another one exporting all those cells manually to another internal tool and so on.
I was hired by this company around six months ago with a technical lead role, with the intention of transitioning the current department I am in from manual work to automated work with Python as a programming language. My profile is purely technical, I am a software developer and software architect with a decade of experience.
During these six months, I have performed two different roles, building the Python infrastructure to automatically perform the processes and also use this same infrastructure to deliver results in a very important project they have, this is all working fine and I have been praised by the high quality of the deliverables.
The infrastructure is as generic and flexible as it can be, and it can be customized via Python to satisfy the different needs of the projects. The logic is the same as for example project in React or in Django. Everything complicated is abstracted out in the infrastructure and available via a function call in Python, it is all designed so that the user only needs to program project specifics and combine blocks together using Python.
The problem is, because other people in the department have been busy with other projects and the manual processes take so long, no one other than me has been working in this infrastructure or the projects derived from it, and no one else in the department is a programmer nor knows anything about Python, the little programming they have done is VBA.
I showed my colleagues how everything worked and they recognize the value it delivers and find it amazing, but completely dislike Python, implementing the project specifics or maintaining the infrastructure. They are not programmers and the role they are supposed to transition to requires programming to a certain degree, implementing the project specifics requires knowledge of simple Python so I was hoping they would start from there in each project and witness the results themselves, but I am starting to believe this is never going to happen with the excuse they are busy doing the manual work in other projects.
I have recently learned that an external company was hired in the past for the same thing I am doing but didn't get far.
The other problem is that neither my manager nor other higher ups know anything about software, there are no hours allocated for the infrastructure so I put all the hours I spent doing the infrastructure, which is not project specific, to a specific project, as I was told, and now they believe that automated processes take too much time because that project took longer than expected, despite me explaining multiple times that most hours are infrastructure related and very little in actual project functionality.
So I am looking for advice as I feel alone and I am not sure if I am hitting a wall, right now I have limited the growth of the infrastructure before it becomes too much for one person to handle, and focusing on gathering interest and getting proper statistics of what is possible to do and how much time it takes.
I will provide more context based on the answers provided.
A process itself can consist in measuring a physical magnitude 800 times and reporting the results between certain thresholds or some criteria in an Excel spreadsheet 800 hundred times, right now this is done manually by a well paid engineer, who presses the button of a tool 800 hundred times every five seconds or so (as fast as he can) and does the filtering in his head and writes the data to a spreadsheet manually.
The communication with the tool can't be done with VBA for several reasons including technical limitations of the language, nor this would scale or be maintainable to all the tools that measure that magnitude because they use different protocols and belong to different manufacturers.
Note this is just a process in a huge framework of processes and projects that vary a lot and can change, and note that this is also required to be in real time despite it not being done in real time due to the existing technical limitations of it being done manually, customers have complained but it is not addressed. This is the reason why they praised the results of my work, because this kind of concerns were also addressed.
So I need a proper language, and I chose Python because I know it well and it is the easiest to learn and start with, with Python I can handle all this well enough in real time and abstract away all the technicalities of tools, drivers and so on and just I expect from the user some Python code to establish the filtering criteria for the results and where the tool is located by its IP. I personally do not find this complicated.
I could abstract this to a graphical user interface to avoid users using Python, but note that if I do this for every process, it would mean I would be implementing everything piece of functionality myself in every project while my colleagues just press a button. And not all processes adapt well to a GUI, they require some logic to be implemented via Python, I would not be able to handle this volume of work myself nor it is the intention, the intention is that people start automating things like I am doing myself. The intention is that I handle the infrastructure and complex problems myself, while they automate the needs for other projects with this infrastructure using Python.
I spoke with everyone involved and shared the vision for the infrastructure before starting and explicitly mentioned that Python would be used unless and the multiple reasons for this, specifically the growing complexity of the projects and the fact that the current way of working implied that a lot of customer requierements were not being addressed. I now realize that perhaps they said OK without thinking too much.
The adoption of this tool is being encouraged by giving me time to show other members how it works, and two people have been selected to start working on it with me, and I have to teach them as much as I can throughout this year, but I am not sure if they are happy about it, and if you do not like programming life can be hell.
By generic and flexible as it can be I do not mean it is complicated or there is too many pieces of functionality, it means that they have a single entry point to a functionality regardless of the setup, and the infrastructure handles all the underlying details of communication, protocols and algorithms behind the scenes, they can use whatever tool is available with the infrastructure and reproduce the same results, by generic and flexible I mean that it is not dependant on specific tools or hardware, you simply call a function in Python to get a value from a combination of tools and it handles all the details and intercommunication for you and gives you a value.
In my oppinion, I find that learning a bit of Python is required for what the company intends to do with the department. The company is explicitly getting projects that require a high level of custom automation of processes to get results in very short timeframes.
I am supposed to be the technical lead for the transition, but the title is the same as everyone else in the department if not for the senior tag. By non tech I did not mean that I am working with people from other areas, I meant that I do not consider the manual work that is being done in the department as technical as anyone can press buttons and fill spreadsheets. By other companies standards, we are way behind in terms of processes and timeframes.