SOX is poorly implemented for IT operations

What’s worse:  Not knowing you have a problem, or being told you don’t have a problem when you really do?

What drives me crazy about SOX is the whole approach is flawed.  Look up anything you can find on the history of SOX and you’ll read it was put in place to help prevent fraud, specifically, another MCI WorldCom or Enron-type scandal.

When I think of fraud in general, I think of people trying to work around the rules or break them in a sneaky way to avoid detection.  People committing fraud (as opposed to more overt criminals like bank robbers with ski masks and getaway cars) don’t announce themselves by committing visible criminal acts; they quietly exploit holes in a vulnerable system or process.

From an IT systems perspective, SOX is disturbingly vague.  There are literally two small sections of the code relating to IT controls in SOX legislation, and since 2002, audit firms have been trying to standardize how to test an IT organization’s compliance.

Where we’ve arrived is an approach to fraud detection that seems more CYA than legitimate analysis: Users are properly authenticated; they’re added and removed via an audited process; systems are backed up and those backups are checked; unauthorized personnel don’t have access to sensitive data, and on and on and on.

See a problem?

A SOX audit is so concerned with making sure all the right rules are in place and enforced, that it completely misses the MO of the fraudulent.  No intelligent person hell-bent on committing fraud announces they’re breaking the rules by trying to hack a password on a production database when they know failed logins are reviewed; they’ll find a legitimate password via an exploit or test system or static backup file or some other way so they can log in legitimately and bypass all the tedious, arbitrary “controls.”

What good is a 7×24 surveillance camera on your triple-locked door when a creative thief can make himself invisible to the camera and pick the locks without leaving a fingerprint?

But wait… there’s another problem.

If you want to prevent systems fraud, you need to at least start by hiring people who are incredibly skilled and experienced on the technology they’re auditing.  How can you expect to effectively audit an Oracle database when the person asking the questions doesn’t have the same level of Oracle knowledge as the client’s DBAs?

Here are a few of my favorite SOX auditor questions, pronouncements, and requests.  No, I’m not making these up:

• Which computer runs the operating system?
• What’s sudo?
• The DBA should have no operating system access.  Not even an account.
• The authentication strategy isn’t effective unless you use Active Directory.
• (After receiving the log output of a successful Oracle RMAN restore) Would you summarize that in a Word document?  I don’t think anyone can read that.

The last is one of my favorites.  While original supporting documentation is fundamental to a financial audit, I’ve been asked I-don’t-know-how-many-times to discard technical detail because it’s too “tough to understand.”

Another good example is database server migrations.  Instead of being concerned with the disposition of the new server, I’ve had auditors all over me making sure “all” the data made it across.  Um… physical datafile copy, folks.  Seriously.  I’ve had to “prove” a physical clone contained the same records as the source db by selecting a few random records from tables on both sides.  And while we’re vigorously proving 2+2 still equals 4 (phew!) no one is asking what happened to the original copy on the old server.

All while the client is billed both by me, and twice as much by the auditing firm.

IT professionals spend careers learning the ins and outs of Oracle, SQL Server, Windows, Solaris, SAP, JD Edwards, PeopleSoft, Cisco IOS, and innumerable storage platforms, not to mention myriad physical components of a data center.  But assign the best Cisco network engineer you can find on a SQL Server audit and they’ll be useless.  What chance does an inexperienced generalist have?

I know I sound like an arrogant SOB, but I’m not knocking an auditor’s intelligence or motivation.  I’m saying the model is flawed and sets good people up to fail without any idea they’re failing.

And then there’s the really nasty problem.

Often, the burden for information gathering and ultimately implementing change in an IT organization falls to IT operations themselves.  Even in large organizations, these are the same people whose primary job is to keep 7×24 systems running.  They don’t receive extra staff or budget to support SOX audits that may turn their worlds upside down, or worse, get them in trouble.

Can you guess what happens next?

I’ve truly never seen anyone outright lie, but I’ve seen auditors ask how many people have the root password without asking who has sudo access and no one stepping up to fill them in.  Like a dentist appointment or watching a C. Thomas Howell film, IT operations staff just wants the audit to end.

So… what?

Well, if the organization fails the audit there will be an indication on the financial statements indicating as such.

But what if the organization passes the audit when people are be running around with wide open sudo because the auditor didn’t know to ask?  Or when the audit wasn’t sophisticated enough to verify known OS/database vulnerabilities had been properly patched and monitored?

Yes, we can probably be sure that user privileges are documented, there’s a verified process for adding and removing accounts, and the database backups can be restored (well, one of them can… according to a Word document), but what about all the stuff that falls outside the rules and skill sets of the auditor?  Should we explicitly tell investors everything is A-OK?

Don’t get me wrong, I’m not saying we shouldn’t audit and I’m not saying the task is impossible; I’m saying SOX, in it’s current implementation, almost certainly provides a false sense of security where IT is concerned.  Yes, we should definitely do many if not all of the things a SOX audit is cares about, but to stop at controls out of context is not a good fraud detection or prevention tool.

I think the five points below would be huge improvements to a SOX audit:

1. Understand exactly what data we’re trying to protect.  General control policies should be established and audited, of course, but SOX is concerned with financial data.  What and where is the financial data?  How does it flow through various systems?  Are copies made (e.g. test environments, backups, reports, table exports, data warehouses, etc.) and how can they be used to access the data?
2. Audit the disposition of the platform.  What are the known vulnerabilities of the various components surrounding the critical data flow?  Have they been patched?  Can they be compromised with a knowledgeable user and a simple Google search?
3. Audit the human processes related to documenting and auditing the data flow. (Yes, I just advocated a recursive audit.)  Are there any?  If a new downstream environment is added, or the database is ported from Solaris to Linux, what happens?  What happens when the flow changes??
4. Audit the qualifications of the technical staff.  I know this sounds jerk-ish, but you don’t hire a babysitter with a history of leaving kids at the grocery store, so shouldn’t we ask if the people charged with protecting critical data are qualified to do so?  No, “certifications” don’t count.  Not everyone need be a database ninja, but beyond a drug and background check, an audit should include a skills assessment from a qualified technical auditor.
5. Include some hacking.  Seriously.  If we really want to know about data security, an audit should include some level of hacking, where we get away from all the checklists and spreadsheets and a skilled technical person sits down at a keyboard and makes an attempt to penetrate the system.  Seems like too much?  Fine.  But most SOX IT audit questions can be answered truthfully without being complete.  Such inaccuracy may be wilful (which is bad) or unwitting (which might be worse) but leads to the same place, and a good hacking attempt would expose many such holes, paving the way for better questions.

Again, I don’t think this is easy, but it would be a good start.  I know _I_ would feel a lot better about a financial statement that had been through the 5-point rigor above than what we have now.  Unfortunately, it will probably take another scandal where pensioners and shareholders are screwed before anyone puts a microscope on the SOX IT audit process.

We can only guess what the outcome will be.