SQL Server, EarthQuakes and the End…Of DBAs?

What does the advent and integration of VMs, SSDs, and the Cloud mean for the traditional DBA?

These are very distinct and powerful emerging technologies that can and will have a direct impact on the future of the DBA.  Read on to find out why.

Well last week was big geophysical and meteorological news on the East Coast, with the unusual Earth Quake, followed by the  Hurricane Irene (Cat 3,2,1,Tropical St, Drizzle 🙂  With such massive weather events occurring so close to each other, many pondered if we are nearing the end.  Science puts them into rational perspective, and much of the hurricane coverage seemed, in retrospect, media hype and overblown.  Now where did I put my Home Depot receipts for the chainsaw, portable gas burner, and generator?  J

The point of the above is when our foremost weather experts tell us the end is near, not only do we listen to their predictions, we often panic and forget to rationalize things and put it into perspective.

Likewise, a possible earthquake in the SQL Server world, concerning the careers of us DBA folks, may be coming, according to the foremost SQL Server Industry experts.  Is this the END of the DBA as we know it?  Are we marked for extinction, like those old mainframe programmers? (Hey there’s a few of them still around!)   

Based on the recent writings of three separate SQL Gurus, I see a pattern emerging.  But let’s not panic!  All of these experts offer a unique perspective on trending technologies, how it will affect the traditional DBA role, and what you need to do to embrace the change.  Though very different takes on the subject, the consensus does appear to be that changes to the typical DBA career are inevitable.  Like when a feature in SQL Server is deprecated, you have 3 versions to prepare, and adapt your code.  As such, we must adapt our career strategies as well.

Let’s take a closer look at the writings of SQL Server Connoisseurs, Brian Knight, Brent Ozar, and Brian Moran.  We’ll refer to them as the three B’s of Budding TechnologiesJ.  Indeed, I’ll go one further, and say they’re among the Best and the Brightest in the SQL Server Space!   All of them have been around for many years, and all have their pulse on the SQL Community, including trends, technologies and careers.  We must listen to their sage advice, and heed their predictions for the future, as they foresee emerging and new technologies affecting the DBA. 

Back in June, Brian Moran, Chief Servant  Officer at Linchpin People, and longtime SQL Magazine contributor,  had a piece he wrote in SQL Server Magazine, entitled “Will SSDs Cause Performance Tuning Experts to Go the Way of the Milkman?”  If you’ve never heard of “the Milkman” growing up, then that’s the point!  For nostalgia sake, the Milkman used to go door-to-door in local neighborhoods, delivering milk and refilling those glass bottles of cow nectar.  If your only memory as a kid is having to leave the house every time the milk is delivered, well, I digress 😀

Anyway, Mr. Moran poses the question, will SSDs, (Solid State Disks), eventually eliminate the need for performance tuning experts, one of the roles taken on traditionally by the DBA.  As SSD’s exponentially improve the speed and performance of anything residing on them, in our case SQL Server, once they are cheap enough to become to norm (and they’re already headed in that direction), why spend thousands of dollars on Performance Tuning Consultants? (Hey, not trying to put anyone out of business, including myself! – I’m just talking about possible future scenarios, and reporting on Brian M’s article)  In fact, Brian relays a great story and real-world example, of SSD vs. Perf Tuning Expert.  He tells the tale where a company hired one of the best SSIS architects in the industry (one Mr. Andy Leonard), and how after weeks of optimizing and tuning some SQL code, they decided to test the old code on SSDs, and the result were stunning!  Click on the above-entitled link, to read this amazing article.  However, as Brian cautions and concludes his piece, “performance-tuning artists will never go away entirely, and this skill will always be among the most highly paid.”  Whew, {wiping the brow}!  This is just one perspective on the DBA’s future – let’s read on and see what Brian Knight has to say.

Brian Knight, SQL MVP, co-founder of SQLServerCentral.com, founder and Principal of Pragmatic Works, recently wrote a blog entry carried on the BI Development Network site, (BIDN.com) that asks another question taking direct aim at the DBA’s future, “Does the Cloud Mark the End of the Production DBA?

We’ve heard about the cloud, and we’re beginning to see the push, and the technologies deployed upon it.  SQL Azure, Microsoft’s new cloud database computing platform, comes significantly into view, with the upcoming and highly anticipated release of SQL Denali, the next version of Microsoft’s SQL Server. 

Brian K. further speculates with the “fundamental shift in the way we manage and store data, pushing the onus of managing the database, patches and operating systems to Microsoft. Reading between the lines, does that mean this will cost DBAs their jobs?”  Indeed, a fetching analysis from the field, as Brian has been on the road talking about the future of SQL Server in his Expedition Denali tour. 

Although he sees the future trending toward the cloud, he does not see significant adoption rates among the larger enterprise companies.  So far, the push is targeted at small to medium businesses, and the current technology prohibitive in terms of large database sizes.  As noted, SMB’s don’t often employ a dedicated DBA – outsourcing to a managed service provider, or just bundling up the responsibility to their IT director or systems admin.  One very interesting shift that Brian sees for the DBA is that the role will go in two directions, “one that manages multi-terabyte databases and another that manages hundreds of smaller databases across the company very efficiently.”  Enter the VLDBA (very large database administrator. J)  You may be already doing one of the above, or both.  Which group do you see yourself in over the next 5-10 years?

Despite all the trends and changes on the way, Brian assures us, “If you’re a DBA, you have nothing to fear but you should start making tweaks to your resume”

To read more on this, and see why you should pay attention to his recommendations, go to the above-entitled link to his blog entry.

Finally, last but not least, SQL MCM, MVP, SAN/VM/HW extraordinaire, and co-founder of the company that bears his name, Brent Ozar (PLF), announces on his blog “The Secret New SQL Server Denali Feature: Data Director”  Now, I don’t want to spoil the story, so to speak, and since I can’t explain it as well as Brent does himself, I urge you to read his blog, by clicking on the above-entitled link, and see for yourself what he’s talking about.  He discusses a combination of powerful technologies trending toward the elimination of the DBA (my words not his), but seems to imply with his no so subtle placement of pics from Jurassic Park, that we could be going the way of the dinosaur!

Brent believes that with the perfection of Virtualization, the development of Denali’s new contained database feature, while not fully realized but part of a greater trend, and the introduction of SQL Azure (ie: Cloud Computing), all bodes bye-bye for the DBA.  Ok, I’m exaggerating a bit, but the convergence of all these concepts is indeed materializing.  Discussing virtualization trends, in one paragraph entitled “VMWare Killed The DBA Star”, he talks about the earth-shaking development of the ability to deploy, manage, move around, and scale databases in a matter of seconds!  The possibilities of dynamically adding Memory and CPU resources to a throttling SQL Server without blinking, when adding virtualization to the mix is endless!  

Think of the implications of having the DBA strategize about provisioning SQL Servers, Performance and Disaster Recovery.  Who needs us?  Just virtualize the DB infrastructure, and manageability becomes hassle-free

I remember many shops, where I came in to take over the complex task of managing the database environment from the system admins and infrastructure folks, who had their hands full managing networks, hardware, SAN, OS, exchange, etc.  Thus began my career as a Production DBA.  Does it now seem that we’re trending backwards?  All these technologies giving the keys to the database kingdom, back to infrastructure!

Please read Brent’s unique blog on this, as referenced above, as I am taking some of his points out of context to make the bigger point on the Future of the DBA.

So, what does this all mean?  Is this the end? Or, just the end of the beginning?  Well, all these articles seem to indicate that it is indeed the end of the DBA {pause}, as we know it, {emphasis added}.

Us long-time experienced DBAs, used to laugh when SysAdmins and CIOs just wanted to “throw hardware” at the problem, as a means to solve a complex database issue.  We all knew that hardware could not compensate for poorly developed code and awfully designed SQL.  This was once true, but this notion now seems no longer the case. A very similar analysis by Messrs.’ Moran and Ozar, referring to different technologies, that “throwing hardware” at the problem may in fact be the right answer.  The cheaper and better the hardware gets, the more mature virtualization becomes, might make us DBAs obsolete.  Brian Knight and Brent both mention in their respective blogs, the cloud as part of that equation as well.

What if we take SQL Server databases, virtualize them on SSD’s and put it up on the cloud? 

Ok, enough rhetoric.  This could be the future, but let’s slow down a minute. The overarching theme here is that while everything may be changing, nothing is changing.  By that I mean the need to constantly adapt to changing and evolving technology has not, well, changed.  This has always been a constant, and no different in our own DBA career paths. 

The good news is that this scenario is some years away, and that gives us a chance to embrace and learn these emerging technologies, and even find a niche within the SQL Server space. Even if virtualization is close, the cloud is not quite there. As Brent states, if you’re not paying attention to these new tech trends, “you’re never going to see your career shift coming….and believe me, it’s coming…”

Does the shift indeed portend to be a SQL DBA Earthquake?

Brian Knight, also mentions in his article the double-digit growth of the BI space, and, although the “DBA job will change over the next decade, it will still remain strong”

Brian Moran, as I stated above, tells us referring to evolving hardware trends, “may reduce the number of performance tuning experts, but will never go away entirely”

Just like a doctor needs to keep up with new medicines and techniques, us DBA’s must do the same - keep fresh, constantly learn and adapt.

One great quote I saw, in response to my sharing Brian Knight’s article on LinkedIN that adequately sums it all up is, “As the world changes, everything will adapt or perish.  Those who succeed in adapting will live on.” Sounds just like the theory of evolution - indeed it applies to technology as well. 

Here are the referenced articles again for your edification:

Will SSDs Cause Performance Tuning Experts to Go the Way of the Milkman? – Brian Moran

Does the Cloud Mark the End of the Production DBA? – Brian Knight

Secret New SQL Server Denali Feature: Data Director – Brent Ozar

If you like my blog, and the many others that I’ve written, you can go to SQLMag.Com, and VOTE for me on question #13. Who are your favorite database bloggers? (Robert Pearl).  It’s a free-form text box, so you can list as many as you like.

You can also vote for SQLCentric which is up again for Best Database Performance and Monitoring product in the 2011 SQL Server Magazine's Community Choice Awards.

Item#4 "Pearl Knowledge SQLCentric" for Best Database Performance and Monitoring Product:

http://www.surveymonkey.com/s/sqlservermag-communitychoice2011-finalvoting

Hurry, voting ends soon, on September 7, 2011!  Thank you SQL Community for your support!