January 4, 2011 at 9:21 pm
Comments posted to this topic are about the item In Case of Breach
January 5, 2011 at 1:08 am
This happened to one of our customer facing products. The database wasn't exactly hacked, but it was effectively the same.
A subscriber figured out how to work trick the application (web) and sequentially worked through/downloaded all the documents in the product.
They were dumb about it, logging in with their account in parallel with the leech which was running from the same IP and they they performed over 400,000 document requests in about 3 hours.
I was the one that spotted it due to abnormal activity on the database outside of any promotional periods, and I did the reseach on the application and web server logs to identify which account it was.
Customer services said they'd handle it. And they "did". They warned the customer not to do it again or their subscription would be terminated early (with a pro-rata refund of course)
ARGH! The customer downloaded the entire database for that productline. We normally sell that for $60k+.
According to the ISV who built the app, they should have a fix in place for the platform as part of the end of Q2 update.
January 5, 2011 at 6:47 am
Interesting topic. Never had the misfortune of going through anything like this so am looking forward to some insight.
"Database administrators often disable transaction logs for fear of affecting database performance, especially on lower-end systems. If that's the case, you will need to refer to other logs, such as those that track activity on systems or applications. " <----What?!
January 5, 2011 at 7:13 am
Tony Savoie (1/5/2011)
Interesting topic. Never had the misfortune of going through anything like this so am looking forward to some insight."Database administrators often disable transaction logs for fear of affecting database performance, especially on lower-end systems. If that's the case, you will need to refer to other logs, such as those that track activity on systems or applications. " <----What?!
I think they are talking about logs in the sense of the error log. You cannot disable the "transaction log" in SQL Server, but you might be able to stop events from being logged as in an audit trail in some systems.
January 5, 2011 at 7:38 am
I suspect that too many organizations have nothing written addressing a breach (what to do, who's to do it, contacts) and if they have something written, have never done a table-top exercise to validate and/or spot holes in the written document(s).
I know of a place where the CIO sent out an email about a "national cell phone do-not-call list" on the basis of a verified urban legend, yet when apprised that it was National Cybersecurity Month did nothing to promote that fact and attempt to raise employee awareness.
And sometimes you can tell them but you can tell them much.
January 5, 2011 at 7:40 am
s/b "...but you can't tell them much..."
January 5, 2011 at 7:43 am
I'm not sure what useful information could be gleamed from database "transaction logs" in the event of a data breach, unless the hacker was modifying data.
In addition to the event logs, another very useful source of information would be the Default Trace and sys.db_exec_requests from which you could query what T-SQL statements were executed by the hacker's login.
http://www.sqlservercentral.com/articles/DMV/64425/
You'd probably want to play it safe and lockdown the server until you determine what exactly is going on. However, again remember that restarting SQL Server would purge critial forensic information held in the default trace and cache. It'd be best to immediately send an email notification to everyone ont he DBA team reminding them not to do this until the initial investigation has been completed.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 5, 2011 at 9:20 am
Steve, I really like your idea of having a "run book" that describes what to do in case of a data breach. Has anyone put together such a book that could be shared with the SQL Server community, as I have never seen one before.
Brad M. McGehee
DBA
January 5, 2011 at 12:01 pm
Brad, there is a book on this topic: "SQL Server Forensic Analysis" by Kevvie Fowler.
January 5, 2011 at 12:11 pm
In addition to responding to data breaches, you also need some type of monitoring process in place to detect a breach. Any thoughts on the best methods or services to do that?
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 5, 2011 at 12:31 pm
Eric Russell 13013 (1/5/2011)
In addition to responding to data breaches, you also need some type of monitoring process in place to detect a breach. Any thoughts on the best methods or services to do that?
Over the last five years I had several clients for whom I did this. Unfortunately, what I did is confidential in perpetuity. So far asw I know, confidentiality is common for this type of contracts.
January 5, 2011 at 1:44 pm
Revenant (1/5/2011)Over the last five years I had several clients for whom I did this. Unfortunately, what I did is confidential in perpetuity. So far asw I know, confidentiality is common for this type of contracts.
Monitoring breaches from the outside, or even internal users, is easier than detecting a breach from an authorized web application account that been hacked using a technique like SQL injection. In a previous job where I was on rotation as a production support DBA, we would run a profiler trace on specific events like "database not accessible" or "invalid object name" and have it send us an email notification. Generally speaking, ad-hoc querying is normal on a development or QA server, but on a production server it might indicate a hacker poking around. Definately don't grant an application account view schema or view server state permission (which by default is not granted in SQL Server 2005/2008) and only grant limited select or exec permission using role based security. That limits exposure and also increases the number of failed attempts that a hacker creates in the process.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 5, 2011 at 1:46 pm
Donna Hawley-340938 (1/5/2011)
Brad, there is a book on this topic: "SQL Server Forensic Analysis" by Kevvie Fowler.
I'll check it out.
Brad M. McGehee
DBA
January 5, 2011 at 7:40 pm
Now here's a question: is there insurance available to cover loss & damages due to data breaches? You can take have product liability insurance if your product can kill people so wouldn't it make sense to have "data insurance"? I'm not so sure the scale of a data breach is so different to - say - a product recall (e.g. Toyota). Maybe the insurance should just be re-labeled as "bad business practice" insurance.
This also makes me wonder about cloud apps and where the liability lies in cases of data breach. For example, if Azure has some fundamental flaw that facilitates a breach, shouldn't Microsoft be liable for this? I've not read the Azure T's & C's, so maybe MS already does assume liability. What's that...flying pigs??
James Stover, McDBA
January 6, 2011 at 12:16 pm
I too will have to check out that book.
My first response is to find where the attack may have come from and then to find what may have been copied.
Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events
Viewing 15 posts - 1 through 15 (of 20 total)
You must be logged in to reply to this topic. Login to reply