One of the things that I often do when analyzing data is examine visual representations. I don't ignore data, but often a graph or picture of who the data is distributed or organized gives me a starting place for more details examination of the actual numbers. It has worked well for me and going back and forth from numbers to graphs allows me to better understand the ways in which patterns and tendencies are embedded.  Humans are visual, and we are expanding that capability more and more to machines.

Years ago I worked for a company that imported wood and sold it in the US. In one of our plants we sorted and organized wood pieces using a combination of human and machine efforts. Conveyor systems moved wood along and humans examined the individual piece as they traveled by. Using chalk marks, they were able to mark items as better or worse quality than others. A computer system scanned for chalk marks and was able to separate the wood more effectively and efficiently than in the past.

That type of human and machine analysis is being used more in many industries. However instead of using humans to do analysis, computer systems can now do that in some cases. I ran across a piece about software that examines physical parts from a manufacturing process, looking for defects. The computer system can do a better job, faster than most humans. There might be a verification step from humans, but for many parts, this means a higher level of both quality and productivity.

I could imagine this type of computer examination has a place in data systems as well. We could train an algoirthm to look for patterns in a visualizaiton, and perhaps highlight them for more examination by a human. We could even have some secondary systems that examine error outputs from something like an ETL process and direct developers to the potential issues in their logic, or in the source files, pointing out data or formatting problems. A little more checking, or real time testing, might help improve the overall quality of our processes, whether in the real world or embedded in software.

Steve Jones

The Voice of the DBA Podcasts

We publish three versions of the podcast each day for you to enjoy. Due to technical issues, we have no podcasts today. Actually, Steve's camera died and he is in the process of replacing it. Hopefully we'll be back tomorrow with the podcast.

The podcast feeds are available at sqlservercentral.mevio.com. Comments are definitely appreciated and wanted, and you can get feeds from there. Overall RSS Feed: or now on iTunes!

Today's podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

You can also follow Steve Jones on Twitter: