This week was a big one in terms of changes for the Microsoft Data Platform. Arguably it was a big change for Microsoft itself, but certainly there were lots of Data Platform items announced at either the 2019 Ignite conference in Orlando or the 2019 PASS Summit in Seattle. Both conferences were scheduled on the same week, which meant that many data professionals had to choose one or the other. I was in Seattle for a couple days for two pre-conference sessions, but then returned home. I also avoided a scheduled trip to Orlando for Wednesday, as I felt that might be a bit much for me.
The big announcement for SQL Server users is SQL Server 2019 has been released. This was announced on Monday at Ignite in the keynote, and reinforced at the opening PASS Keynote. I hope the PASS keynote becomes available to stream, as I only saw a portion of it on Wednesday. This is a big release, and to me, an inflection point for the data platform. With containers, the ability to scale out, and the enhancements in Polybase, this is a big release for SQL Server. Most of us might not see these features for years, but this will change the world. A couple nice things are no licensing cost increases, and the addition of the Secure Enclaves for better Always Encrypted implementation.
In addition to this, we get SQL Server Edge, which is a version of SQL Server that can run on ARM processors. This means we can do some processing closer to the data, in remote situations, without moving the data to a central place. The demos seem to involve AI/ML work, but this could be pre-processing of some data in a remote office and then reporting or making decisions and only sending some data back across a network. It is definitely built more for embedded or sensor work, but the ability to handle time-series data and not make a round trip to a central server is useful.
One of the more impressive demos at Ignite was the query of a trillion rows in 9 sec. This is obviously a shot at Google's BigQuery and Amazon's Redshift, and it's an impressive demo. It's amazing to me that we are talking about trillions of rows and PB of data now in relational systems. This is Azure Synapse, which combines the previous Azure SQL Data Warehouse and Azure Data Lake products into one.
Perhaps an announcement that might not seem like it's a big data related one is Azure Arc. This extends the Azure Resource Manager to on-premises systems, but also to other clouds (AWS, GCP, etc.). What wasn't quite emphasized, but is important, is that this can manage data services, which means we can start to more easily stand up and spin down database resources. With containers growing, and more desire to get consistent dev, test, and production systems, I think Azure Arc is something more and more of us will be using.
If you're looking for more coverage, we've got a series of blogs linked below that cover the announcements at the events from the perspective of attendees. Read through some of those in the Community section below.
