74

Our team -

I work in financial and operational reporting as one of a team of 3 -- I'm not an accountant or in the finance function as such, it's more like Management Information (or Business Intelligence). We pull out numbers for senior managers and do daily reporting and stuff.

To do the reporting we have "view" access to operational systems (ie. the actual customer relationship information, accounting ledgers, etc) and our role is just to pull information from these systems and collate 'Management Information' reporting through the systems we maintain.

For example if we were Amazon (we're not) we would have access to the "customer orders" database etc. We'd update a daily report including something like "Total dollar amount of orders, number of orders, number of unique customers, number of repeat customers, number of returned items... etc etc"

There's an automated process that pulls the information daily, and through which we write "one off" queries as well.

I found potential security problems...

About a year ago I discovered that due to the way the system is set up with usernames etc, an automated account (used for reporting) inadvertently has access to update those operational systems (where it has no business doing that) and someone on the reporting side, if they knew that that was the case and were so inclined, could log in and write a query to "delete all of John Smith's orders" (or something more destructive, they could even delete all the Orders if they wanted!)

I have raised this and so have my co-workers many times to our managers (who have come and gone), senior managers, legal/compliance people who didn't seem to understand the risk or didn't have enough knowledge to see that it was relevant, etc. Every time we have received responses like "don't bother your pretty little head with that!" -- but I know it is a huge hole in the system.

We've got auditors in

Now we have an imminent audit (not related to this issue, it's just a part of the business cycle) in which the auditors will be talking to people around the business including me or someone else in my team (among many others) about things like "ensuring integrity of financial information".

I know the usual rule when dealing with auditors is to answer honestly, but don't volunteer information outside of what they ask (and I understand the reasons for that).

In this situation could/should I "let slip" to the auditor that we follow xyz process to ensure the daily figures match up to the operational system etc etc.. but there's this account that actually has "UPDATE" access to the operational system which is used to do that and I see that as a vulnerability?

11 Answers 11

79

You've already mentioned this vulnerability to your managers and compliance people. Unless you personally are responsible for information security for this system, you have done your duty. And, it sounds like you have done your duty well.

If nobody has mentioned the vulnerability in some kind of writing (trouble ticket, email message, etc) then you, or a co-worker, should do so.

Then, it's not on you to decide whether to follow up. Let somebody else deal with the hassle. (Unless, again, you're responsible for infosec on this system.)

If you have an interview with auditors, truthfully answer the questions they ask, but don't volunteer any extra information. You, seriously, don't want to be the reason an auditor finds an exception in your organization's processes. Auditors can't fix problems. They can only give the company a hard time.

Don't be tempted to think of infosec in a legalistic way. Even if you follow all the rules you can imagine, you still aren't completely safe. And, even if you don't follow all the rules you can imagine, you may still be OK. Generals are fond of saying "battle plans never survive first contact with the enemy." That's also true of infosec.

  • 6
    And if they don't have it in writing already (maybe it was a verbal discussion with previous managers), now is a good excuse to formalise it in writing - using the audit as the reason for flagging it again. – delinear Nov 14 at 14:42
  • 12
    Auditors can't fix problems. They can only give the company a hard time. I have mixed feelings regarding this line. If by giving a hard time you mean detecting problems, then yes. But compliance is required for some sectors. Crappy auditors will make up molehills out of orbital electron excitement, so yes. Those give the company a hard time with no benefit. Disclosure: I'm not an auditor – Mindwin Nov 14 at 17:05
  • 9
    "battle plans never survive first contact with the enemy" I prefer the way Mike Tyson said it "Everybody has a plan until they get punched in the mouth" – Kevin Nov 14 at 18:33
  • 5
    Linking concerns to the audit as @delinear suggests is probably a good idea: "I'm concerned that the auditors may ask questions about <gaping security hole>. How should I respond if they do?" Would be an innocent way to broach the subject. The written reply will inevitably be "you must tell the truth", which effectively clears mentioning it, if you are so inclined. Merely raising the question may be enough to get the issue fixed though. – bain Nov 15 at 14:30
  • 12
    Auditors can't fix problems. They can only give the company a hard time. If non-compliance is a problem, then giving the company a hard time is the way to get it fixed. As a consumer whose personal data is floating all around the internet, I'd be livid to know that the convenience of sloppy security practices were justified by the desire to avoid "a hard time." – mac9416 Nov 15 at 18:36
19

I would recommend reporting this risk to your CISO or Risk Manager if you have one available.

For example:

IF a reporting analyst with access to the reporting database credentials performs an INSERT, UPDATE or DELETE query THEN the integrity of live customer data could be catastrophically impacted as a result.

Consider a likelihood and impact level and a suitable treatment.

e.g.

  • AVOID by reducing reporting database credentials to read only or
  • AVOID by changing reporting database to read only

To be honest reporting should be performed off of a separate (read-only) copy of the Production database, not the live production database itself (for reasons like you've mentioned). A CISO especially should absolutely understand that.

Risks such as this should end up on a risk register with the highest rated risks (likelihood x impact) on a heat map which senior management is generally accountable to address.

But some risks are not worthwhile to address because the likelihood x impact might not justify a response.

For example, if the reporting database was already a copy of the Production database then changing the reporting database could compromise the integrity of reported information without compromising the master source data itself.

I hope that's useful.

  • 5
    It sounds like they have already done that, and it didn't work. So I'm not sure this is a good answer – New Alexandria Nov 13 at 22:27
  • 2
    @NewAlexandria Thanks for the feedback. I tried to point out that a CISO or Risk Manager are the most appropriate roles to raise this sort of risk to (since they hadn't been mentioned), and also provide some context as to why the risk may / may not be regarded as urgent to resolve. – ChrisFNZ Nov 13 at 22:30
  • @ChrisFNZ OP already mentioned bringing this up with "legal/**compliance** people". The fact that OP didn't mentioned CISO and Risk Management is a pretty clear sign that the company is too small to have CISO or Risk Management separate from the CIO/CTO office. – RonJohn Nov 16 at 5:01
14

There are many kinds of risk, and your post has not differentiated between them. The kind of risk involved has a huge impact on what you might do, and what might happen from you doing it.

  1. risk to the consumer
  2. risk to clients' personnel' information
  3. risk to employees' information
  4. risk to the clients' profit
  5. risk to your employer's profit
  6. risk to your employer's operations

Some of these could be regulated. Consumer info usually is. Any personal info usually is. In industries like health, weapons, infrastructure, and some banking have regs for operational processes (e.g. NIST 800-53)

If the situation is a direct risk to one of those, you may consider the ethical responsibility to those that can be affected, and laws protecting them. This may also include whistleblower protections.

If it is not one of those, then you're just barking up the wrong tree. In that case, your employer does not care about the risk to their profit nor their reputation — no matter how large or small a risk. That's not your call. You brought it up, and if you brought it up to the right people then your job is done:

  • security or compliance officer
  • the board
  • an anonymous reporting 'dropbox'
  • related C-staff or CEO

..... in that order


source: I work as an information security officer.

  • 1
    I would also add that you should consider they know of the issue but there might be more pressing problems they need to address. Once you have the record you reported and provided the documentation you have, let them do their job. – raubvogel Nov 14 at 15:58
  • I think this is the correct answer with the correct reasoning explained. If it were customers data at risk (like in the Equifax data breach) it would be different, but now its just the company taking a (stupid) risk. – Ivana Nov 14 at 22:36
  • 1
    Some of the information in this system (which could get updated by that account) gets used in various kinds of 'external' reporting that we have to send to various regulatory-type organisations. In the case of 'Orders' data imagine that we have to submit a report each quarter of the number of orders per product in a particular category as it is considered "notifiable" in some way. – user111727 Nov 15 at 18:42
  • @user111727 it's still not clear if that process requires a control (for non-modifiability) under a given regulatory framework — industrial or state/federal. Without knowing that, the answer cannot be more clear. I.E. please update the question with context about the regulatory frameworks this falls within. – New Alexandria Nov 15 at 19:05
7

Tread very carefully

Ultimately, you're a human being who needs to eat. Your decision should not jeopardize that. Once you've raised these concerns, it's management's responsibility to take a course of action to protect their interests. Remember, that data is theirs and if they want to leave it exposed to deletion or inadvertent access, that's their prerogative. You can point out to them that this kind of risk, in the long run, is costly. You can point out that if someone found out about it, it would damage reputation and thus is costly.

You and others have raised these concerns already. They've been rejected. You put yourself at risk the more you continue to raise these concerns.

In my career I have seen:

  • A coworker threatened with legal action for "hacking" for reporting a security vulnerability he stumbled across in an application he actively maintained.
  • A coworker terminated without severance for overzealous persistence over similar risk-related defects in reporting software.
  • A coworker chastised by management for happening to stumble across a data-exposure bug while on a call with a client (the data that came up was from a different client). That employee's contract was not renewed at the end of the term.

And I've only been doing this for 15 years. I'm sure there are people on here who can give examples of even worse treatment for things that we, as boots on the ground, recognize as obvious "You really should fix this" kind of situations.

Personally, I would not recommend raising it to auditors. If they're worth what they're being paid, they'll find it anyway. You may want to file a bug report in whatever software you use, and close it as WONTFIX, citing the emails sent by management regarding this. (You DID remember to get their rejection in writing, right?)

What I wouldn't do is anything that might make me wonder where my next meal comes from. I have a wife and kids, so that might influence it more than if I didn't, maybe I'd take the more principled stand. But even without them in my life, I'm a big fan of having food every day.

  • 4
    +1 for citing emails in writing and adding it to the bug report. Really important that there is a paper trail to show you did everything in your power – Bee Nov 14 at 16:25
6

Be careful about conflating "poor business practice" with "security risk" .. and especially more careful about letting information "slip out" during an audit.

First, does your company not back up its data? If it does, this is less a security issue and more of a data integrity issue. If it doesn't perform data backups, that would seem to be the more pressing concern.

Second, a sound financial system would throw errors if financial data are deleted in one account without also deleting corresponding double-entries in others (i.e. an order results in value of inventory reductions; credit payments require adjustments to a receivables account, etc.). The ability to mess up the order data is unlikely to rise to the level of a security issue if proper accounting safeguards are in place. (The presence of auditors is a good indicator that financial safeguards are being evaluated and scrutinized.)

Even if you were so inclined to delete orders, a system audit trail should exist to indicate changes. As you pointed out that this was an 'automated account', that may make it difficult to track down the human culprit, but unlikely to be impossible in a small company. Still, the data could likely be recovered through financial forensics at worst (shipping information, inventory management systems, supplier re-orders, etc.).

Last, as someone that has managed a financial company which was frequently audited for a variety of reasons, I was often asked questions about security risks to our systems. Occasionally, an employee interviewed during an audit would offer information they felt was either inside the scope of the audit, or else "close enough" but worried about a "lack of action from management" (me). Usually, this was an annoyance to me as a manager since, in my experience, every security or operations risk lobbied by a "concerned employee" would ultimately end up as a non-issue for the auditors, in cases where I deemed the concern/risk acceptable.

I am very grateful to employees that would raise concerns so that they could be addressed. But when the responsibility was mine, yet they took the matter into their own hands in the presence of auditors, it was annoying, at best. At worst, I would lose trust in the employee since they did not regard my evaluation of their concerns to be valid. (That is to say, an astute manager will recognize your "slip up" as a vote of no-confidence in their authority. That is risky.)

You may have truly incompetent managers. This may be a very serious security risk to your company. Before you step outside the scope of your responsibilities, please consider that the management responses you are getting to "not worry about it" may be entirely valid.

Last, the issue you raise is obviously not desirable. Probably the most useful thing that you could do is try to solve the problem gracefully. Look to the vendor, a system administrator or other technical resource that can help fix the rights of the automated user. If that is not feasible, then if you like your job otherwise, continue to periodically remind management of the risk, while focusing on doing your job the best that you can. Or else consider a move to a company where you feel more secure (although there is no guarantee that you will continue to feel secure either way).

  • 1
    Even if they back up the data, you still have the possibility of losing everything since the last backup. A true story -- at one of my first jobs (1998 or so), I was asked to do some reporting using MS Access. Something went wrong mid-way through, so I asked if I was working off a local copy, or the master. I was told it was a copy, so I went and deleted the records I had already processed ... and it went really, really slowly ... because it was linked to the mySQL server. We had nightly backups, but had to re-process all of the payments made that day. – Joe Nov 15 at 3:33
  • 1
    That is a valid concern, but it operational not security related. Categorical, operational risks are vastly different from security risks. Still, a disruption to operations hurts a business and can be harmful - if there isn't a clear security risk, the OP should present the issue as an operational concern, which might give it real traction with management if they realize how it could hamper operations. – Jim Nov 15 at 4:38
  • 1
    The company does back up its data yes, but a 'bad actor' could use this backdoor to insidiously delete/alter data in a way that no one would necessarily realise was happening, so wouldn't know that they needed to restore from a backup. So we can't really guarantee the integrity of that data as we wouldn't know it had been updated.. – user111727 Nov 15 at 18:46
6

About a year ago I discovered that due to the way the system is set up with usernames etc

It sounds like the underlying issue here has nothing to do with management/operational concerns, but is actually a technical bug.

The solution is simple, and the same as it should be for any other technical issue:

  1. Create a ticket in your bug-tracking system.
  2. Give it a priority befitting its severity.
  3. Assign the ticket appropriately.
  • I like this answer. Someone is in charge of user accounts and permissions for the system (or if no specific person has been designated as in charge, someone nominally maintains the database system). You want to talk to that person about whether a particular account should be given more limited permissions (or a separate limited account created, if multiple processes are using the same one). So file a ticket that the appropriate person will see. If making account changes like this requires documentation or approval or some other process at your company, that person should know what's required. – Zach Lipton Nov 16 at 4:27
5

You should answer questions honestly to the auditors as instructed. Nobody here can answer whether or not this "vulnerability" raises to the level of whistle blowing. We simply don't have the context. If you have brought it up with the appropriate people in your organization, I'd say let it go. Good luck.

3

Your problem is an ethical dilemma because the situation gives you an inescapable choice in which each alternative can have undesirable consequences. Either:

  • You report the vulnerability to the auditors, which might endanger your job if you cannot trust the auditors.
  • You report it otherwise, which you have already tried and has been ineffective so far. Going beyond that might also endanger your job.
  • You don't report it, which puts the company at risk.

As outsiders, we (the answer-givers) can give advice, but the dilemma remains yours. The way to go into an ethical dilemmas is to analyze it. Can you get any help with this from a trusted party? I know that some hospitals have ethical committees that offer such help, but that "ethical infrastructure" may not be available in your organization. In that case, you might consult a trusted friend to help you analyze your dilemma. After that, the decision remains yours. I hope you will find an answer in which your integrity remains uncompromised.

2

To help frame the response from management, here is a back-of-the-napkin cost and risk assessment.

Unless there are some serious deficiencies in your organization, this is a low risk, low severity, low cost to mitigate vulnerability.

There are few people who have access to the account, namely software developers (who can do damage without access to that account), Systems Ops (who can do damage without access to that account), and your team.

Individual members of your team may be able to do more damage to the company through other malicious actions, without needing to use that account. (I honestly don't know if this is the case, but I'd be willing to bet long odds that it's true.)

That is, everyone who has access to that account is a trusted employee. Even if they were inclined to act maliciously, everyone has higher payoff and lower risk targets.

Risk: Low.

If anyone were to act on your impulse, to change or delete order information, it would be quickly noticed by other departments who depend on that information. At the very least, trained accountants will notice on daily or weekly reports. I would be surprised if the fulfilment department doesn't find it within an hour.

When it is found, your Systems Operations team will be pulled away and the database administrator will find the discrepancy. At this point, if there are any logs, it will point to the automated account, which points at your team. After completely locking out that account, a database restore will be done, using the full weekly backup, then the daily incremental backups, followed by replaying the Write-Ahead Log file up to the second before the malicious query.

The total cost will be a few hours in wages paid for each employee impacted due to work not being done. Senior management will be furious, but it will hardly impact the health of the business as a whole.

Severity: Low

If the issue is addressed, it will cost at least 5 hours of developer time, spread across 2 devs and a database admin. One dev will need to make sure that, if the permissions are changed, it won't impact any automated processes. Since you have auditors, it means a second dev (usually a manager) will need to sign off on the changes. And finally, the database administrator will need to actually make the change.

It's a low cost to mitigate it, but it's still a cost. (Developers aren't cheap.)

Perhaps, if the change can be implemented in the course of changing related systems, then it can be justified... but most management will be reluctant to incur any cost when they're certain that their employees are trustworthy -- or their employees are at least not foolish enough to do such an obvious and low impact act of sabotage when they are able to perform more subtle and higher impact acts.

Cost: Low (but not zero)

Unless the auditor is doing an account security audit, they don't care and can't do anything about it.

If you're adamant about getting it fixed, you have already followed nearly every step. The steps available are:

  • Tell your manager.
  • Tell the Systems Ops team through their official channel.
  • Mention in casual conversation to officers. (You've already made your official report before this point. You have done your due diligence; any other non-casual actions after this are meddling.)
  • Mention in casual conversation to a developer who might someday work on the system, and who might be able to add the change when working in related systems in a no-cost way.

If management is unwilling to incur a cost, and you can't get it done in a no-cost way, then you're done. Third parties, even auditors, shouldn't hear about things that you consider vulnerabilities, unless they're hired specifically to identify those vulnerabilities.

2

Although I have never been in your position, I am an auditor. The answer (from the auditor's perspective) will depend on your firm's relationship with the auditors.

External Compliance Auditors

The most distant and far removed auditors are external compliance auditors. They may be evaluating your firm's compliance with governmental regulations, industry standards, contractual obligations, or some other kind of requirement. This is fundamentally an audit that someone else imposes on you.

Don't lie to them. If they ask about you about an area of security, and you know of an issue, tell them. If they discover this issue even without your help, then there may some much harder questions later. Neither you nor your company want that.

If they don't ask, should you tell them? There's no easy answer here. Only if it's germane to their audit concerns. For example, you shouldn't tell a federal grant compliance auditor about an IT security concern. On the other hand, if it seems to be related to their audit work you should think hard about your position. Do you have a moral, professional, or legal obligation to tell the auditor? Is there a practical benefit to your company or the public? Is it on point for the audit?

Contracted External Auditors

Your company may also hire external auditors for various kinds of engagements. IT security is a common area to do this. Your firm has hired IT security auditors to determine whether some security criteria are met. Both the auditors and your company are interested in knowing about IT security risks and whether you are addressing them correctly.

In this case, it's much easier to tell the auditors what you know. Although the auditors here are much more friendly, the main risks to you will be from your management and coworkers. Will it be obvious to them that you disclosed this risk? How will they treat if they know?

Maybe they will see it as a helpful gesture intended to move the company forward. Maybe not.

Internal Audit

Internal audit is your firm's own auditors. They are regular staff members of your company. They probably report to a member of your executive board.

These auditors are your friends. You want to disclose problems to them. Why? Because if internal audit finds a problem and helps fix it, that means no one outside your company had to find out. And that's a solid win.

  • I once thought an internal auditor was a friend. He lied to me (or lost control of the information--either dishonest or incompetent) and raised concerns in a way that made the CEO (huge public company, utterly ridiculous for it to go up there) get involved and caused us a huge amount of headache. – msouth Nov 16 at 21:02
1

This might come as offensive to some, but it's necessary evil.

"You have to eat, tread very carefully!"

"What will you ever do if they fire you!?"

Please OP, don't listen to a dime of what people who are slaves to the system, content with going by in this society, have to offer you as actual advice.

What's the most important thing in your current prerogative? Doing the right thing.

You already know what that is, but are asking this question to see if other people would leap in your stead.

Don't bother. You're not them. Even if the whole world tells you it's wrong, remember that an invisible friend will always be rooting for you when you decide on taking action -- and for a good reason.

  • The question is what is the right thing. Not whether to do the right thing. Suppose you know about a problem that would make your company fail the audit. The auditors don't catch it. You bring it up of your own accord. The audit fails, everyone loses their jobs because the company loses a critical contract. What is the "right" thing here? I think it's "keep your mouth shut around the auditors and continue to work on it internally". This vulnerability may never be exploited, after all. We are not omniscient. – msouth Nov 16 at 20:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.