… and by government data, I mean the mountain of data recently made available by the G-Men on Data.gov. This site contains what must be terabytes of data on every topic from environmental measurements to crime statistics, from geographical data to labor statistics. The Obama administration has committed to greater transparency, and the availability of this data is a significant step toward that goal. The trendy geek magazine Wired.com recently did a feature on Data.gov that is worth reading.
It’s obvious that Data.gov is an immature portal. Delivery types are inconsistent – some files are available only as flat files, others as only Excel, and a few claim to offer XML feeds. The formatting can vary wildly from one set of data to the next, and often includes headers and footers which muddy up otherwise clean raw data files.
So why should you, as a database professional, care about this information? If you’re trying to improve your skills in database technologies (and especially in this economy, who isn’t trying to improve him/herself?), this data store is a great place to start. Because of the sheer size and sometimes unusual layouts, this information is an excellent test bed for honing one’s skills at Integration Services, Analysis Services, or for creating VLDBs (very large databases) on which to practice. And if you’re truly ambitious, there’s a contest to come up with the best application of this data, with a $10,000 bounty to the winner.
As for me, I’m currently pulling down some FBI crime data with the intention of using it in an upcoming SSIS class I’m presenting. Perhaps I’ll think up an app that could win the $10K as well….
