Assuming you’ve been on earth lately, you probably heard the terms “Big Data”, “Hadoop”, “NoSQL” and so on.
Companies want to check what they can do with Hadoop, trying to understand what Big Data is and how it can generate more income. Big Data experts pop up on LinkedIn, newspapers and websites publish articles about the Big Data revolution, and some writers declare the relational database is dead.
Of course we can hear it at work more and more, and without a doubt it’s not always pleasant to hear it as a DBA. Is it the time for a career change?
I decided to go to Google Trends to see some numbers.
First, I wanted to see a graph for “SQL Server”. This is how it looks:
And now, here’s the graph for “Big Data”:
It doesn’t look good. SQL Server is declining while Big Data is significantly inclining.
But let’s put the two searches on the same graph (SQL Server in blue, Big Data in red):
We can see that despite the hype, the number of searches for Big Data is less than 10% of the number of searches for SQL Server.
The graphs against NoSQL, Hadoop and MongoDB look the same.
What does it tell us? It’s hard to reach a clear conclusion. We don’t know how deep are the pockets of the ones searching for Big Data, or how senior are the ones searching for SQL Server. That’s why I don’t want to start writing some “conclusions” that may or may not be true in a few years from now.
What we can say for sure is that everybody can calm down. even though there are very serious implementations of Big Data and NoSQL, these disciplines are only at their early stages, and the relational database is not going anywhere.
In addition, SQL Server (and the relational database in general) is a mature product, with known advantages and disadvantages and a big and active user community. Big Data and NoSQL are technologies that are not yet at the same area.
There’s still a-lot of mess on their domain, partially because there are a-lot of NoSQL database types and vendors (although some try to make things more clear – you decide if they succeed). Companies are still groping in the dark in order to understand what they can do with it, and in my opinion it’s a marketing trend no less than it’s a technical trend.
Having said that, these are important technologies that will probably grow and prosper, and the trick is knowing when to use what. Facebook is a great example – the Timeline sits on top of MySQL (relational), and the News Feed sits on top of NoSQL.
And that’s why it’s worth getting to know them – I’ve already read the excellent post “Love your enemies.. it pisses them off” by Maria Zakourdaev, about to attend a presentation about NoSQL databases at my local user group, and planning to start playing with HDInsight.
In addition, it’s important to understand when we really need a Big Data solution. A few years ago there was a very popular term in our world: VLDB – Very Large Database. The best definition I heard for it at the time was: “A database you need to maintain and work with in unconventional ways”.
The unconventional ways of these days are more conventional now, but this definition reminds me of what we talk about today:
A system that needs a Big Data solution is often referred to as a system that has to handle enormous amounts of data, which arrive at a stunning speed from a variety of source types, and can be both structures and unstructured (videos, mails, tweets, etc..). That’s why not anyone who claims he needs a Big Data solution really needs such a solution – most of the systems are quite standard.
I assume in a few years from now things will be more organized. It will be more clear which database type to use when, and in general SQL Server will work beside NoSQL databases, when each one doing what it’s good at.
Oh, by the way – for a mature look at the future trends, check out this post. To summarize in two lines:
- Big Data as a term is dead. Now let’s see what we can do with the technologies we have
- NewSQL is coming (go ahead and read)
