Restricted from installing my own python libraries [closed]

Question

first: a bit of background - I run a team of DB devs, I have a degree in software engineering, I can use python very effectively and have done in several organisations (of similar size and larger). I've just hired a data scientist do some analytical work in my team, they primarily use Python as their scripting language, they start in 4 weeks.

I got IT to install Python (& Anaconda) through visual studio (our IDE for other bits) onto my machine, I tried to pip install a library to warm myself back up and make sure the environment would be suitable for the new hire. Company have blocked the SSL. After asking to unblock the SSL they come back with (effectively);

Python libraries are open source and could have anything in so we need to check it. Once approved we will download it for you.

Things I'm concerned about;

• I've never had to ask to install libraries before.
• it's literally going to add weeks to developing any little POC.
• They are really slow at turning things around.
• The new hire will be put off and leave.

question how do I make the business case to turn their opinions of it around and make them see how the rest of the world do it?

Answer 1

1. Ask for sandboxing of the development team if possible

   I assume that you are in a highly regulated environment of some sort as I have never encountered this level of bureaucracy and I worked in both banking and government where we handled financial information. However, what they did in banking to allow us to do this was to sandbox our team away from infrastructure. We had a different wireless network, a different set of credentials, and a different set of computers for development. Everything was the same except that we didn't have direct access to customer data from the development machines or any other kind of infrastructure.

2. Ask that developers can get admin access.

   IT tried to implement this pre-approval requirement for libraries in my government job. Obviously, that did not fly and the security people in IT couldn't code anyway. What ended up happening was that admin privileges were given to the developers specifically so they could install whatever they needed and override the whitelist. They just more heavily locked down everyone else.

3. Do not screw it up.

   If you do not bother IT, it is less likely that they will bother you.

Answer 2

This is not as odd as you think. Companies have been burned when employees download software with no controls, so I've personally seen IT restrictions grow.

I can see at least two issues that you need to address specifically. One, using open-source code can cause your work products to become open source. It all depends on their respective licenses. Do your research and be able to prove to legal that what you are downloading does not do this. Don't assume legal and IT know, or will drop everything and figure out for you ASAP. Two, software often has separate paid license categories for corporate users. If you circumvent that by downloading something and just ticking the "personal use" box, the company can get served a notice or be sued. Be prepared to prove that this is not the case for your software.

After you are prepared, escalate this through the normal chain: your boss, division head, etc. Such alignment will go further in changing your company's procedures or granting you elevated permissions.

Answer 3

They are really slow at turning things around.

I would focus on this bit. Asking for the freedom to pip install any library you want with no oversight will likely be a non-starter but you should be able to make a case for streamlining the approval process. Explain why it's important that the new hire has access to these tools and how not having them is costing the company money. Checking the license and screening for malware shouldn't take more than a few minutes so it should just be a matter of putting pressure on the relevant parties to make it a top priority.

Answer 4

In my experience (in over a decade contracting in banks and government departments), the trick is to find other developers in your organization and find out as much as you can about their workflow and the level of access they've been granted.

If you can prove that whatever you want is functionally equivalent to what someone else already has then your request is much more likely to be granted, on the grounds that it already has been granted to someone (presumably) equally responsible.

If that fails then your best bet is to offer to use and enforce security precautions that are essentially paranoid. Think about what your workflow would be like if you were designing missile guidance systems and work from there. This is actually how the popular lightweight database system SQLite came about, when a contractor for the US Navy couldn't get permission to install a "proper" RDBMS. Depends on your circumstances, but maybe you can use VMs for everything, or a super-secure dev box, or even AWS Workspaces (if you don't need to work with anything confidential)

This "Plan B" will be uncomfortable and inconvenient for you, but the goal is to demonstrate a genuine business need for the tools you want to use. Once you've demonstrated the value of what you're able to create, it'll be relatively easy to show how you could do even better if you were able to work without both hands figuratively tied behind your back, and why specific restrictions are unnecessary.

And even if that fails, over time you will get better at finding creative workarounds for seemingly unnecessary restrictions, and you'll learn more about what you really need to get the job done. I have had so much practice configuring SSH port-forwarding and working around issues with non-standard installations of miscellaneous tools that junior devs think I'm basically a wizard now. Good luck!

Answer 5

The IT department that you are portraying as clueless is actually correct, and you are the one who is (excuse my harshness) clueless. You can't just download a runnable computer program from the web without thoroughly vetting it because it could contain malware that could destroy or greatly damage your company.

By default, you should be blocked from downloading programs. You need to explain to IT why this program is safe.

I recommend you read about dependency confusion, which is just one of many things that can go wrong when you carelessly download code from the web. https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610

Answer 6

There could be a basic misunderstanding in IT's answer. Here is an answer from a software engineering perspective.

IT's answer that there could be anything in open source libraries is basically correct and our cognitive bias prevents us from understanding the threats, which are two.

As correctly stated by @MichaelMcFarlane, uncontrolled use of open source dependencies can lead to licensing issues where you company could be sued to reveal the source code to their expensive projects.

As for security, consider the Docker cryptominer scandal as a proper example. Actually, the issue is present and ongoing, but I decided to pull an article from 2018.

The strength of open source dependencies is the large community behind a piece of software. This is especially valid for software such as Anaconda itself, which is popular and maintained.

But smaller, mostly-unknown projects will just go under the radars, exactly like a pedestrian crossing the street on red light in a mostly-desert downtown intersection at off-peek time. I have witnessed many.

Maybe this should be discussed better on Security SE, but the concept is that you just don't pick any random johnthehacker/ repository (quoted from DockerCon 2021), because:

1. Anyone can pull-request anything evil to OS projects, and the author may accept it without intention to make harm, but by not fully understanding the impact (e.g. lack of experience/education)
2. You decided to pull what is revealed a malicious fork of a popular project
3. The project claims to do something innovative but its malware (e.g. cryptominer) hasn't been discovered yet
4. You don't read the license terms carefully finding that the dependencies sends a lot of data to the developer(s)

Bottom line

Never pull open source dependencies too easily "just because they are free". Request review from software architect or IT department, motivate your request to pull certain software. Make sure that the process does not become too bureaucratic, otherwise report constructively to IT.

Answer 7

You need to change your perspective.

Don't call that opinion, because it's not their opinion, it's their security policy. And you're not going to change that policy, because it was established on the higher level.

You need to learn to deal with it. You are not restricted from using the libraries only because you need to ask for the permission. The normal procedure is:

1. You ask for permission
2. They check the library
3. If everything is OK, it will be available on the internal server or in some other form

So they key is, to plan in advance. Do not wait until you start implementing things to find out that you need some dependencies. Try to find everything out before starting the project. And you'll learn to minimize the number of the libraries you use.

Or you need to find another jobs. Big corporations are not for everyone.

Answer 8

As others have noted, it is not unreasonable especially for large, conservative, and regulated companies to be cautious of security and/or licensing risks with third-party code and to want to have some process in place to manage that. What is unreasonable is to do it in a way that entirely destroys productivity by requiring a lengthy process where IT must download any package indvidually.

In comments, you noted that you "just lost two staff, one reason quoted was the restrictive IT." This gives you a business case for addressing the problem; the cost and lost productivity of replacing these employees is significant and will likely be repeated if things aren't changed.

It may help to assemble a list of specific examples, addressing both things like recruiting cost caused by departing employees and the delays caused by this process (e.g. "the VP of Finance requested we produce an XYZ analysis. We could have had it ready within 24 hours, but obtaining permission to download the library needed to parse Excel files took two weeks, which meant the VP didn't have the analysis for the board meeting"). Finding out, as Andy suggests, how other teams have managed to handle this will also be valuable information.

But many companies with these policies do not simply rely on a manual "IT will download all packages" policy, because they understand that's not a productive working environment. Large tech companies have custom in-house repositories of third party code (see, for example, Google's processes around the management of third-party Python packages). This means that most common packages are simply already available for corporate use and can be used by everyone immediately, and there's a clear process with ready-made tools for developers to import new ones from Github and PyPI and have those additions reviewed quickly.

There are tools like Sonatype's products that can manage this process so that the company can have visibility and control into its third-party dependencies while maintaining productivity and compatibility with package managers, CI pipelines, and other development tools. I would see if the company is willing to invest in a company-wide effort to manage third-party code to actually address this problem.

If your company is imposing these restrictions yet is unwilling to implement a real package management solution, they're shooting themselves in the foot, not only because of the loss of productivity, but because "IT will manually download approved packages" means there's no process in place to keep track of dependencies (IT is surely not manually reviewing and tracking updates for the hundreds of dependencies that may come with a Node project) and ensure vulnerable packages are patched.

Answer 9

You hint at the answer in your question:

it's literally going to add weeks to developing any little POC.

What you do is give realistic estimates to your boss.

"This would usually take me a week to do, but given X, Y, and Z hoops we have to jump through here, it's going to take 4 to 6 months"

This way you put the ball in your boss's court. Do they want to accept that you'll have to rewrite common libraries, or do they want to go to IT and fight your case?

I was in a similar situation once, about ten years ago, and I applied this technique. The concerns of the IT department were overridden by the concerns of the business.

Answer 10

There are already some great answers about how to handle this situation assuming it's a security restriction at your company. However, I've been in the exact same situation at a highly regulated company and the resolution was very different from the answers above.

The IT help team assumed since my package downloads were blocked, that I was trying to download insecure content and they gave me the exact same message. They were wrong. We had no such cyber security policies, and the IT help team made incorrect assumptions about what I was trying to do. I'd like to share a few practical tips for you to try.

1. Is Pip really blocked?

You mention that you tried to pip install some-package and it failed with an SSL error. This can often be caused by a proxy. Try downloading a python package from your browser. Here's an example to download Flask: https://pypi.org/project/Flask/#files

If you can access those files, then pip downloads are not blocked. You simply have the wrong CA cert settings on pip. Try disabling SSL verification (insecure but easy to do) or linking to your company's root CA certificate (secure but harder to do).

2. If pypi is blocked, do you have a pypi mirror?

My company blocked pypi installations on some machines that need extra security. To install python packages, we had to use a pypi mirror. Our company hosted an internal website that acted like pypi and only hosted approved python packages. You should ask others in IT if such a site exists. Some common platforms for this are Artifcatory, devpi, nexus, etc.

3. If pypi is blocked, go old school

If your company blocks pypi.org, and does not have a pypi mirror, then you can go old school. You can manually download wheels from github and install locally. For packages without wheels, you can download the code and install directly with pip setup.py install

Answer 11

If you want to install python libraries to support your own learning, that can be done on a separate computer, not connected to the company network. You could get a cheap computer and monitor, install things at home, get an internet connection through mobile data using your own phone. Completely separate from your company network. IT should be able to lock it down so you cannot access the company network at all. Maybe that’s agreeable.

If you want to use open source libraries to develop software use by your company, it depends. I have some software where my company wouldn’t care if other people had it because it is useless to anyone else. There’s other software that might save a competitor money but wouldn’t hurt my company. Obviously anything containing company secrets is a problem. For that you’d need a serious conversation with IT. And there’s a decision what is cheaper: Additional development cost because you can’t use some library, or cost for company lawyers to check things.