I've always thought that you couldn't have too much data. Keeping more data around, more metrics, more data points was a good thing. It allowed you to make better and better predictions. But perhaps you can go too far.
I saw this article on Wall Street technology as it relates to the recent financial crisis. Apparently as regulators had wanted more precise analysis of the risk of loans, banks used technology to better measure risk, taking many variables and factors into account. And as a result they lowered the liquidity requirements.
They knew the models weren't perfect, and they actually were expected to prove the models worked, by comparing reality to the prediction. Unfortunately for us, the models were disproved by the market failing.
Now supposedly they were more accurately reporting and calculating their own risk, but as we can now see, they aren't taking into account the entire market and other competitors' portfolios. So when you start to have more than one company with issues, the risk escalates dramatically, almost snowballing across the economy. Granted we might be in a "perfect storm" situation in the economy now, but I'm not sure this couldn't happen in other places.
To me this is a problem that can afflict people that think they're too smart. It's not that the developers are manipulating numbers or that they aren't doing a good job examining their data to determine risk, but more that they aren't considering enough information. In my mind, this is the old economic prediction problem, which was solved in the Foundation series by Isaac Asimov. If we had enough information, we could predict what would happen; the problem is we can never get enough information, or calculate it quick enough.
Data mining to determine how to run your business, whether it's ordering widgets, assessing risk from loan portfolios, or deciding which marketing campaigns to run, makes sense. It can work, but it has to be done with some tolerance for error. Betting your business on the results being very accurate is a good way to go out of business, or in this case, crash a large section of the economy.
I'm not thrilled with the current situation, and I don't think this is completely a technological problem or even a problem with our banks. There's plenty of blame to go around, but I think one lesson for us data professionals is that it's not just garbage in/garbage out we have to worry about. We need to allow for error in our calculations and conclusions as we move to fuzzy methods of analysis.
Steve Jones
The Voice of the DBA Podcasts
The podcast feeds are now available at sqlservercentral.mevio.com to get better bandwidth and maybe a little more exposure :). Comments are definitely appreciated and wanted, and you can get feeds from there.
or now on iTunes!
- Windows Media Podcast - 28.8MB WMV
- iPod Video Podcast - 25.4MB MP4
- MP3 Audio Podcast - 5.2MB
Today's podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.
I really appreciate and value feedback on the podcasts. Let us know what you like, don't like, or even send in ideas for the show. If you'd like to comment, post something here. The boss will be sure to read it.