Have you ever tried to read a transaction log? I mean used a query against fn_dblog() to read data and try to reconstruct what happened with a transaction or a series of transactions? It's a cumbersome process and takes a lot of knowledge, practice, and most importantly, patience. It's not something I'd want to wish on anyone. There are a few products to help, but no one really does this that often, and it's almost easier to just change some data by applying your own manual fixes.

If you're in the UK, you might have heard about the TSB bank meltdown. If you're unlucky, you've been affected by the outage, which has been going on over a week as a system cutover failed. You can read some reporting about the plans, the rollout of some services, the initial problems , and the warning signs. If you go to the end of the third link, you'll find this awesome tweet. Beans and bombs, he he.

There are a lot of potential issues that we could discuss here. I've been a part of a failed rollout and I have sympathy for the IT staff dealing with this. The thing that I wonder about is the data. With the magnitude of customers (millions), the seemingly long list of places where things failed (notifications, scheduled payments, inquiries, etc.), and the rate at which people can bang on a system from their phones and various applications, how much data has been mangled and altered? 

I'd guess a lot, in which case, we aren't just talking about updating rows on the basis of someone's authority. Whoever is tracking through data needs to essentially read transaction logs, unwind the actions where data was converted incorrectly and then (potentially) subsequently changed. Then they need to work out the reversing entries. The database needs help from DBAs, developers, and probably financial staff to understand why things are in a state. Why are closed accounts are open, why payments are scheduled years in the future, where balances are, and more. With the possible cross contamination of data between accounts, this is an area where TSB needs to be thorough and careful.

Data is important in today's complex, interconnected world. There are certain areas where data problems are highly disruptive and can have lasting repercussions if mistakes are made by the data processors. The financial and medical areas certainly fit in these categories, and it's sad that people are going to go through pain and problems that may affect them for years. Hopefully TSB will get things working soon and data issues corrected. If there's one thing I learned from this is that for certain issues, I need to ensure I have my own paperwork to prove my side of the story.