11

I work in a small software house. We are all developers and we do servers/network management required for our software (many servers per customer).

We are expanding with more customers; and the overhead of server management is increasing and now we plan to hire server(s) administrator.

Normally the servers/networks will monitor/manage dozen of VPSs, couple of vpn/domain servers, some database and email servers.

My main concern is trust. How can I trust someone with all these credentials just because he passed technical/hr interviews?

EDIT:

  • By trust I mean trust as in malice, not incompetence.
  • Impact of dishonest developer is lower. We have code reviews and testing. The worst thing to do is to take the code. But server admin can delete servers, delete backups, ...etc.
  • I read many times about server admin who took server control and asked for ransom (after conflict with management).
Improve this question
7
  • 5
    You won't trust the new hire to do what? We all hire based on a certain level of trust. Hell, your own management hired you on trust. Didn't they? So why is it that it's OK that they should trust you but not the other person? Are you high enough on the totem pole that it matters what you think anyway? Do you have an explicit rationale for your question i.e. something the new hire said or did, or are you having a knee jerk reaction to outsiders coming in? – Vietnhi Phuvan Nov 10 '14 at 7:44
  • In what country is this? I have never heart of professional sysadmins act like this in Europe. After all it is highly criminal to demand ransom ... – s1lv3r Nov 10 '14 at 9:49
  • 1
    www.xperthr.co.uk/blogs/employment-intelligence/2013/05/revenge-reprisal-retribution-w/ – ItsMe Nov 10 '14 at 17:19
  • 1
    @s1lv3r: Some people, especially those that feel wronged by a company, will do some seriously dumb stuff. – NotMe Nov 11 '14 at 15:38
  • 2
    But your current programmers have been doing server management, why would someone who exclusively do these tasks be less trustworthy? Code review is moot when your devs are able to function as admins. – user8365 Nov 11 '14 at 22:14
20

I'd take into account a few things. First and foremost, that you're going to have to spend some time getting your new employees used to the way your company does things, and that period is the perfect time to get to know them. I'd suggest pairing them up with experienced, trusted employees, and well, to put it bluntly, very politely spy on them to get to know them better while training them.

Secondly, good security fundamentals. A good system would log access accurately, ensure your employees don't log in as someone else, so if one of your new employees does turn out to be a bad apple, you can work out the damage done and mitigate it. Likewise have a good system for managing access to systems so you can minimise access to a system. In short, make sure they have the access they need and no more.

At the end of the day though, you can't watch everything your new hires are doing. If you feel something is off, act on it.

Improve this answer
6
  • System logs don't help if a stupid sys admin decides to get revenge by deleting the company servers. I believe there was a city in California that was essentially held hostage when a network admin changed all their router passwords and left. They continued functioning and he ended up in jail but there was a lot of money spent after the fact trying to clean up the situation. – NotMe Nov 11 '14 at 15:44
  • Hence minimising access by need. That should reduce the damage any one person can do – Journeyman Geek Nov 11 '14 at 23:11
  • How do you propose to minimize the access of someone whose very job it is to manage access for others? By definition this person is the super administrator or root or whatever. They hold the keys. – NotMe Nov 11 '14 at 23:13
  • Separation of responsibilities. You're not handing all the keys to everyone, and the whole idea is to know who has what keys so that if they're compromised, the damage they can do is limited. – Journeyman Geek Nov 11 '14 at 23:26
  • I know what that means. What I don't know is how you would implement it given the OPs situation. Are you going to have servers which their one and only sys admin doesn't have admin rights for? – NotMe Nov 11 '14 at 23:29
5

While Journeyman Geek's answer is probably ideal for a normal threat profile for a small company, you can look to elements of what larger organisations carry out for some guidance.

I have long worked with financial services companies, FTSE 100 and Fortune 50 companies, and key to hiring is carrying out criminal checks, residency checks and credit scoring in order to be able to assess the risk a new employee may bring.

For those in sensitive roles (think security, server admins etc) high levels of checks may be required, and in fact many banks require you to have a current account with the bank in order to monitor finances at a level which would indicate risk.

Malicious activity requires motivation and opportunity. So for server admins there is only so much you can do to reduce opportunity (admins use normal accounts for most purposes and get break glass access for additional permissions), so being able to assess motivation (mortgage in arrears, gambling debts etc) is a useful way to protect your organisation.

While a full package of assessment may be out of the reach of a small organisation, the basics can be quite cheap/easy, so worth having a look at.

Improve this answer
5
  • +1, do you have any references for reputable companies doing these kinds of checks (residency, credit...)? – ItsMe Nov 11 '14 at 14:39
  • krollbackgroundscreening.com is one obvious one. google.co.uk/search?q=criminal+checks gives you more – Rory Alsop Nov 11 '14 at 15:55
  • I was asking about companies requires checks nor doing the checks :) – ItsMe Nov 11 '14 at 16:41
  • 3
    Okay - in that case, every company I have worked for in the last 17 years, and almost every company I have consulted for (totalling about 80 FTSE 100's and about 15 Fortune 50's.) These include banks, oil companies, government departments, global manufacturers etc. Most even have established processes for staff in those countries where credit checks, or criminal checks may not be allowed (Switzerland, Germany etc) – Rory Alsop Nov 11 '14 at 16:43
  • 1
    @ItsMe: I'm in the US and don't think that I've seen a company in the past 20 odd years that didn't do background checks like that. Heck my own company is relatively small and we do them before making an offer and repeat them about once a year for everyone regardless of job function/title. – NotMe Nov 11 '14 at 23:31
5

Part of a sysadmins job is to maintain system backups in the event of total failure.

When they are hired in, make sure they understand this. Those backups should include ones that are offsite. For a small office, this can be as simple as having the sysadmin deliver a copy to the manager/owner on a weekly basis.

To make sure the backups are good, the manager/owner should have random "surprise" restore tests where those offsite copies are used.

This does two things. First, it ensures your backup system is working - which is pretty critical. Second, management has physical access to them and can easily keep them away from the sysadmin.

As off site backups are pretty common you don't have to let them know that fear or lack of trust is a driver here.

Improve this answer
2

I think it is important that employees trust the company. Some people are never satisfied, but if your devs didn't corrupt the system when acting as admins, there's not reason to believe someone whose main role is admin is any less trustworthy.

Since your devs have some admin experience, you can have some redundancy with your backups. The admin may set them up, but make other's responsible for the tape backups or other off-site storage. You may need more than one.

Make sure this person is truly given the responsibility of maintaining and protecting your system. The other programmers may be less productive/hindered if some of their privileges are removed, but you need to show the new admin your trust by giving them more control in some areas.

Improve this answer
0

In a midsize to large company, you aren't just trusting the individuals -- you're trusting that their management chain vetted them before hiring them, and is actively educating them to be good at their jobs. Which is the same reason you trusted the previous employees, so the fact that there are a few new kids shouldn't significantly change your level of trust (though you may want to more actively verify their work to make sure they did exactly what you asked for, until you're sure they have settled in).

In a small company... well, you still have to trust that their management is managing them. If you have actual reason for concern, you need to take that up with management so the company can act on it... usually YOUR management, so this goes through appropriate channels. If you don't have evidence they aren't trustworthy, then either trust them or consider changing jobs, because that's what the company is providing.

Improve this answer
0

The abuse of a system by a privileged user is an inherent risk that you can minimize through good management, but this risk can never be completely eliminated. @JourneyMan Geek has made great points regarding least privilege and segregation of duties. You can increase your trust but ensuring that incompatible duties such as system administration / security administration or system administration / change management are done by two or more different persons.

I will add on the following:

  • Establish policy that spell out what the consequences of misuse are and make sure your users acknowledge the policy. This is a directive control
  • Routinely review access and confirm all existing access rights to ensure they are still proper to the employee's role. This is a detective control.
  • Follow the principle of trust but verify. You should trust that privileged users (ex: domain admins) are competent with good intent. Nonetheless, it is imperative you verify that they are by maintaining adult logging of all activities by privileged users. Information to log include at a minimum:

Time of transaction

Unique identity of privileged users who made transaction - nonrepudiation

Description of transaction

Whether transaction was successful

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .