The more data you have, the better you should be able to predict something. Or at least that's one of the things that I learned while studying economics. If we could actually gather enough data about someone or some system, we could determine what the most likely outputs of the system will be.

In the real world, the problem is gathering enough data. We've never been able to, and probably still aren't, able to actually get enough data to make an accurate prediction. We get close with things like weather prediction, but as most of you realize, it's still a lot of guesswork.

We gather more and more data all the time, as witnessed by our databases that are growing and growing, at accelerating rates that were unbelievable even 10 years ago. And there's quite a bit of research going on in many fields to mine that data, discover new patterns that might not have been visible in the past, and make (hopefully) more accurate predictions.

The first decade of the 21st century has to a large extent in the US and Europe, focused on dealing with security issues, terrorism in particular. There was an interesting press release about a system being developed at the University of Buffalo that seeks to mine data in real-time, developing the ability to determine the liklihood that any particular person is a terrorist. It's a research project with computer and behavioral scientists.

It's a real-time BI project of sorts and I'm sure it's probably fascinating to be involved with the details of how they tune the system.

This system is supposed to be used for interrogations and should "learn" as the interview progresses. Like many projects, I'm sure those building it have the best intentions and they're trying their best to solve a problem that is hard for humans to handle.

Like any such system, I'm concerned that any level of success with this system might end up with it being abused. It might end up making a decision that someone should continue to be investigated. My worry is that it would generate some preconceived notion of guilt rather than just a trigger for other investigation.

Just remember that it's still just a machine, doing what we tell it. And like anything built by humans, it's as fallable as we are.

Steve Jones

Voice of the DBA Podcasts

Today we've got just an audio podcast as I didn't get to upgrade the lighting and by the time I'd worked through chores and recovered from the World Series, it was too late to shoot. We'll be back tomorrow with a new video segment.

• Mining or Profiling (audio) - MP3, 4MB

We'd like to expand the podcasts to include some other video segments that might help you with SQL Server, so let us know if you have any show ideas or any stupid DBA tricks that you might like to see Steve perform.

Music for today's podcast from Josh Woodward. This one is called: Methylchloroisothiazolinone. Check out more at joshwoodward.com.