I drove up to Sanford after work on Thursday for the oPASS meeting to see Mark Kromer present Big Data and SQL Server in the Real World. It was interesting, even intriguing. Definitely worth looking at the deck. He had a couple of interesting tips too. One was that if you have an MSDN account use some of your free Azure hours to try out HDInsight (essentially Hadoop on Windows). The other was to download PowerQuery for Excel, let’s you pull data in from various big data sources fairly easily (I think Office 2013 only, worth taking a look at – has a nice AD connector too).

I think the meeting just reinforced again for me how hard it is as an IT professional to know where and when to invest time if you want to be on top or ahead of the curve. Do I invest time in MongoDB, etc, in case I’m asked? Is that part of a deliberate transformation of skill sets or a broadening of the one I have? Can I learn enough about it in a reasonable time – say 8 or 16  hours – to at least see the possibilities, or is it a deeper dive? HDInsight seems closer to the core I maintain, but will the average company have a use case for it? Is it because they don’t have it that they don’t use it?

It was interesting to hear Mark talk about the differences between key/value stores (we do this within SQL at times, easy enough) versus the JSON/whatever stores. It bothers me to talk of ‘unstructured data’, but maybe it is just a matter of degree. Logs for example, from whatever app, have a known structure. That’s different than parsing the body of an email to pull out something. The body is part of a structure, but within in, who knows.

I was also struck that while I could immediate see uses for predictive analytics from the last oPASS meeting with Jen Underwood, it is harder for me to see immediate uses for big data.

It was interesting and positive that we had a meeting that wasn’t even close to 100% Microsoft. I don’t think that indicates anything except an interest and willingness to consider new points of view.