Enterprise applications often rely on more than one database, but there is no accepted method today for versioning and managing complex data environments. Windocks is an independent port of Docker’s source to Windows, and recently introduced containers with built-in database cloning. The result is terrific for delivery of .NET and SQL Server Dev and Test environments. Writable Terabyte class databases are delivered in seconds, using less than 40 MB of storage. Teams increase productivity, with fewer VMs and post-release hot fixes.
The ability to image SQL Server containers with large and complex data sets also introduces a new capability for data management and governance. Images can include a collection of databases, are versioned and immutable, and can incorporate data masking and other data preparations during the build. Importantly, images are auditable artifacts, and should enhance data governance and compliance.
Data Imaging for Complex Environments
Data images are built using Full or Differential SQL Server backups, or snapshots, combined with SQL Server scripts for data masking. The image captures a full byte copy of the data in the form of Virtual Disks that can span multiple physical (or virtual) disks and large data sets. The Virtual Disk, in turn, supports creation of Windows “differencing disks” which are writable clones. Clones are delivered in seconds, and only require 40 MB or less of storage. Each clone is delivered with a SQL Server container, but can also be used by other SQL Server instances.
Images are built with a Dockerfile that specifies the location of backups, or snapshots, and SQL Server scripts for data masking. Windocks images support multiple databases, with source files located on the Windocks host or a network attached file. In the example below the Dockerfile specifies three databases, located on different network attached file shares, and SQL Server scripts for data masking.
The result of the build of this Dockerfile is an immutable image that is an auditable artifact, and supports delivery of multi-terabyte environments in seconds for Development and Test, and for reporting and BI.
Data Imaging enhances Governance and Compliance
Data governance and policy compliance should be built into the Continuous Integration processes being implemented with containers. The workflow shown above does this, by delivering complex and large data images in the container workflow. A container based workflow enhances data governance and policy compliance through:
Privacy/Security: security is improved with data delivered through a structured container process. Adhoc access to enterprise data can be curtailed, and approved and auditable images used to support dev and test, as well as reporting and BI needs. Privacy is enhanced as data masking is built into the image delivery. National boundaries are respected with image registries hosted in the appropriate country, as the Windocks solution runs wherever Windows servers are supported (on-premise, private, or public cloud).
Quality: the container workflow enhances data quality and consistency by supporting use of production databases as the authoritative source of data. The workflow outlined above will soon be enhanced with native Jenkins server support for Continuous Integration, making this approach unique as the first full stack Jenkins Continuous Integration solution that incorporates production database support.
Access and Use: Docker containers are emerging as the defacto standard for software development and test. Containers play a prominent role in Microsoft’s strategies for Windows Server 2016 and SQL Server 2017. The approach outlined here provides organizations with an on-ramp to Docker based workflows on Windows Server 2012 and Server 2016, with support of all editions of SQL Server 2008 onward. Not only does this workflow improve access to data (on-demand and in seconds), with the latest preferred dev and test tooling, but it also is uniquely useful for SQL Server reporting and BI purposes. This workflow also integrates with existing backup and DR system infrastructure, making it uniquely easy to add to existing systems and infrastructure.
Conclusions
Data governance should not be an after-thought for modern software development and delivery strategies. Windocks combination of SQL Server containers with database cloning delivers benefits for development and test, reporting and BI, and enhances data governance and policy compliance. The solution installs with existing systems and delivers Terabyte class data environments in seconds, while creating immutable, versioned and auditable images that address many data governance needs.
To learn more about Windocks, and our container innovation, visit us at www.windocks.com, and ask about access to our Jenkins server early access program with an email to info@windocks.com.
