[read this post on Mr. Fox SQL blog]
Every now and then you come across a use-case where you need to do something with spatial data, and you need to do it in the cloud (Azure, of course)! Up until that very point you maybe didn’t know, or perhaps even care, much about the intricacies of spatial data assets, let alone how the heck you were going to store it, process it, and query it, without making a mess of your current data stack.
Well, if you’re that person, then I say welcome to this blog post!
So given the above, hopefully this breakdown of Azure Services that can manage spatial data assets will help you position which service is right fit for your scenario!
Got Spatial?
So what is spatial data anyway? (I hear you think)
Well electronic spatial data “processing” first really came into existence when the term GIS, or Geographic Information System, was coined by a guy named Roger Tomlinson back in 1963 (I know because I read it on Wikipedia). Although the actual use of spatial itself for a practical mapping purpose was about 130 years or so before this.
But the first time I’d heard about managing spatial data assets via GIS was around 1993 when I selected a subject named as such (ie GIS) back in my undergraduate Computer Science degree at Monash University. I was captivated by it, some switch flicked in my mind that just loved the concept of the physical represented digitally, that could also be reproduced on-demand. For some people today it may be analogous to the concept of first hearing about Digital Twins, which is also awesome.
And well, so naturally I got a High Distinction in the GIS exam
But GIS <> Spatial per se, its just a system by which spatial assets can be managed. Given the modern architectures and services provided in Azure your options beyond GIS are wide and varied. So without digressing too much (more), here’s some of the key points about spatial data;
- Spatial Data can take on many forms and formats, so there’s an international body which continues to set standards for the various data out there – its called the Open Geospatial Consortium (or OGC). The standards are pretty thorough, with a defined full list of standards. In respect to cloud vendors, Microsoft are also a Principal Member of OGC, and ok yes, yes, so is Amazon and Google.
- In general Spatial Data can be classified as either Vector Data or Raster Data.
- Vector Data consists of coordinates of information (ie points, lines, polygons, etc) normally taken from encoding systems / applications
- Raster Data consists layers of images (ie jpg, png, tif, etc) normally taken from cameras or satellites.
- There’s actually different types of Vector and Raster data — but lets not get into that today.
- Its difficult (and some say impossible) to fully represent all aspects of a Raster dataset as a Vector dataset without losing accuracy or fidelity of that data along the way. If you think about what a Raster image may look like then that makes sense. Thus Raster data is normally much larger, more cumbersome and more processing intensive than Vector data.
Azure Data Services for Spatial Data Storage and Processing
Okie Dokie, now we’ve done the backstory, lets get to the meat/eggplant of this post… where can we manage spatial data assets right within Azure!
Well up front (sorry to inform you) – there’s no single native Azure service that does nothing else except spatial – however spatial functionality is built into many of the core Azure services. Spatial functionality in each service can range from what I’d call “rudimentary” to “functionally complete”.
So depending on what type of data (ie vector/raster, static, streaming, structured, etc) you may want to look at one or several of these services for the building-blocks required for your use case…
Azure Service | Service Type | References |
---|---|---|
Azure SQL Database | Relational | SQL Server supports two spatial data types: the geometry data type and the geography data type for Vector data assets, along with a rich set of functions. See here |
Azure Data Explorer (Kusto) | Timeseries Streaming | ADX actually has several references to leveraging spatial, which are pretty new. See here, and here. |
Azure CosmosDB | NoSQL | Azure Cosmos DB supports indexing and querying of geospatial point data that’s represented using the GeoJSON specification. GeoJSON data structures are always valid JSON objects, so they can be stored and queried using Azure Cosmos DB without any specialized tools or libraries. See here |
Azure Stream Analytics | Streaming | Geospatial functions in Azure Stream Analytics enable real-time analytics on streaming geospatial data. These functions support all WKT types and GeoJSON Point, Polygon, and LineString. See here |
Azure Maps | Mapping | Azure Maps is a collection of geospatial services and SDKs that use fresh mapping data to provide geographic context to web and mobile applications. See here |
Power BI | Visualisation | Power BI has many places you can perform spatial, which is via inbuilt visualisations or 3rd party visuals too. These are the main ones here (PBI Maps), here (Azure Maps) and here (ArcGIS). There was also a recent announcement at the Microsoft Business Applications Summit (or MBAS for short) about leveraging Power BI for “Data in Space”. This is a really, REALLY, cool implementation, which also used Azure Spatial Anchors – see here |
Azure Spatial Anchors | Mixed Reality | This empowers developers with essential capabilities to build spatially aware mixed reality applications. These applications may support Microsoft HoloLens, iOS-based devices supporting ARKit, and Android-based devices supporting ARCore. Azure Spatial Anchors enables developers to work with mixed reality platforms to perceive spaces, designate precise points of interest, and to recall those points of interest from supported devices. See here for capabilities Point of note – the Minecraft Earth engine is based on Azure Spatial Anchors! Super Duper Cool! |
Azure Databricks | Spark | Databricks itself dosent have any native spatial capabilities, however there is great support for spatial in several 3rd party libraries. These 3rd party libraries can load, process and query spatial data types. This Ref here is an excellent summary of these. This is also a nice “how-to” reference here from one of our Microsoft colleagues. |
Azure PostgreSQL with PostGIS Extension | Relational | PostgreSQL DB is a PaaS database in Azure similar to Azure SQL DB, but for PostgreSQL databases. As such it also supports the full range of PG Extensions, as per here A key extension within PG is called PostGIS, which would be recognised as a best-of-breed feature to support many spatial types and functions. It can store both Vector and Raster. |
Go to the head-of-the-class if you’ve noted a flagship product absent above? As at writing Azure Synapse does not support spatial. I’ve blogged about this before here, with workaround options. If you want this feature (and hey who doesn’t!) then vote it up here!
Personally I’ve used spatial capabilities in SQL Server extensively and it’s quite powerful (it’s been in the product for over 12 years). I’ve also built large DW applications using SQL (VM) + spatial – and presented on topics like this at conferences (see presso deck here).
Conclusion
OK, so there you have it, a quick flash summary of the main Azure services where you can store and process spatial data assets. Remember as I said at the start, keep in mind that many of these services can be used for other purposes, and most likely WILL and CAN do the job, however may not be the “best fit” use case for their primary design.
By the way, if I’m missed something super obvious, or you think something needs to be changed, then ping me in the comments and I’ll check it out!
So as usual, and as I always say, give this a try yourself, as your own mileage may vary!
Disclaimer: all content on Mr. Fox SQL blog is subject to the disclaimer found here