Problems displaying this newsletter? View online.
Database Weekly
The Complete Weekly Roundup of SQL Server News
Hand-picked content to sharpen your professional edge
Editorial
 

A New World of Data

I started in the early days of SQL Server, when having a gigabyte of disk storage was unheard of, much less a gigabyte of RAM. My watch has more storage space than the mainframe we replaced with an early version of SQL Server years ago. The technical possibilities and amounts of data we are capable of storing have changed rapidly every year of my professional life. When I got started, we strictly focused on automating processes that were previously paper-based. Once organizations started to get paper-based processes automated, they realized they could start analyzing that data and find out interesting things a lot faster than with paper spreadsheets.

With basic processes automated, we started attempting to store not only store transactional data, but data about every movement that any customer makes on our websites, social media, along with recording and translating phone calls (with permission, obviously) to hopefully find details that can increase the income of companies without increasing spending. One of my first databases I designed as the lead data architect was to capture data from student web browsers working for a school system ISP. We captured a lot of data, but never did much with it because, as we realize now, relational databases are not great for capturing and analyzing un-, or even semi-structured data, especially when you have to adapt the structures over time for new needs and requirements.

New, fail-fast development methodologies coupled with the desire to capture more and more data is equal parts amazing and terrifying. With the advent of unstructured data storage, there is a feeling that we can just store the data, then go back and analyze it later when we know what we are looking for. There is nothing generally wrong with this concept, but there is an old saying that the devil mixes truth in with the lies to make them sound believable. The problem lies with dealing with this vast data that is being stored. If it is not designed and catalogued as well as possible initially, the burden of analyzing the data will be too much.

Let’s make the data of today into the answers to the questions of tomorrow. Just by spending a reasonable amount of time designing, ideally when data structures are new, we can make it that much easier on future data scientists to figure out what we were doing and come up with realistic understanding of the past that can help predict and shape the future.

Louis Davidson (@drsql)

Join the debate, and respond to the editorial on the forums

 
Redgate SQL Monitor
The Weekly News
All the headlines and interesting SQL Server information that we've collected over the past week, and sometimes even a few repeats if we think they fit.
Vendors/3rd Party Products

New release: Mask SQL Server data 3x faster with Data Masker 7

In the latest version of Data Masker for SQL Server, it’s now much easier to rapidly identify sensitive data that needs masking, set up appropriate rules, see which masking rules are applied to which column, and locate what sensitive data you've yet to mask. We’ve also improved the performance of commonly used masking rules, which lets you run masking operations 3 times faster.

New: 2020 State of Database DevOps report

Now in it’s 4th year, the State of Database DevOps report continues to deliver valuable insights. In the report, you will learn multiple tactics to improve the quality of database code deployment, whether it is by changing the type of environment used for development, enhancing your code review practices, or your change management/approval practices. Download your copy today.

Administration

Scripting and Deploying SQL Agent Jobs

Basically, we need to put all the SQL Agent Job .sql scripts into a git repo. Then we need a PowerShell script that executes each .sql script against the necessary target databases.

Bulk Update Database Mail Password

Database mail is a critical part of getting notifications from your SQL Server when things go wrong. The problem comes when you have a database mail SMTP account where...

Extended Events: Histogram Output

The single most important thing to remember about Extended Events is that this functionality is not simply a replacement for Profiler/Trace, but a whole new tool with new functionality....

Update Statistics

Is there any safe way to ‘update statistics’ without causing major slow downs? As far as a safe way to update statistics. It is best to rebuild statistics in...

Azure SQL Database

Azure SQL Database and Error Handling

For some reason I have friends / colleagues tellin...

Backup and Recovery

SQL Server Backup and Restore Strategy Step by Step

Introduction: Who is not aware of a SQL Database? ...

Big Data Cluster

Setting up to use External Tables in SQL Server 2019 BDC

SQL Server 2019 Big Data Clusters is one of the cool new features that’s available with the release of SQL Server 2019. The idea behind SQL Server 2019 Bid...

Conferences, Classes, Events, and Webinars

Level up your deployments for SQL Source Control

Free Webinar - Thursday 20 February, 4-5pm GMT / 10-11am CST - Want to combine the state-based development experience implemented in SQL Source Control and Source Control for Oracle with the customizable power of migrations-based deployments? Join this session to discover how Redgate's new flexible workflow allows exactly this.

Redgate SQL Source Control
DMO/SMO/Powershell

Starting and Stopping SQL Server with PowerShell

Have you ever had this issue? You are trying to ex...

Data Warehousing

Data Lake or Data Warehouse

Abandoning a data warehouse for a data lake may not be the answer. Over the last few decades, we’ve seen an explosion in the use of data warehousing within the IT industry.

Database Design, Theory and Development

Why you Should Always Specify Whether a Column Accepts Nulls

Phil Factor explains the factors that determine wh...

DevOps and Continuous Delivery (CI/CD)

But the Database Worked in Development! Checking for Duplicates

As developers should not have access to production data, it’s possible that duplicate values can sneak in during migrations. In this article of the series, Phil Factor demonstrates a...

ETL/SSIS/Azure Data Factory/Biml

For a Limited Time: Use SSIS Catalog Browser to Manage DtsConfig Files

For a limited time, I am making DtsConfig Management – a feature of SSIS Catalog Compare – available in the latest (free) version of SSIS Catalog Browser. Am I...

HA/DR/Always On/Clustering

SQL SERVER – Always On Listener Creation Failure – Enabling Object ProdListener Failed With Error 5

I have written a few blogs where I explained a few...

Notebooks

.NET and Powershell 7 Notebooks

Rob Sewell forwards on some exciting news: A noteb...

Performance Tuning SQL Server

Interleaved Execution in SQL Server

I wanted to write an article about the interleaved execution, but what you’ll see here is a good illustration of the improvements in SQL Server database engine since the version SQL Server 2012.

Get Your Scalar UDFs to Run Faster Without Code Changes

Microsoft has added a group of features called Intelligent Query Processing to SQL Server 2017 and 2019. In this article, Greg Larsen explains one of the features, Scalar UDF...

PowerPivot/PowerQuery/PowerBI

Creating Time-Intelligence Functions in DAX

DAX contains a host of time-intelligence functions with exotic names such as SAMEPERIODLASTYEAR and PARALLELPERIOD. In the last article in this series, Andy Brown explains how to write measures...

Five tips for Power BI Paginated Reports

For those who are new to Power BI Paginated Reports, this blog should help you hit the ground running and create reports with high productivity.

Power BI Slicers and Filters

Besides the built-in cross-filtering and cross-highlighting among visuals, Power BI supports two explicit filtering options: slicers and filters. Which one to use?

Power BI API – Usage Monitoring – A Power BI Report for Tenant Usage Data

Over the past few weeks, I’ve been putting together a series of posts on how to connect to the Power BI Rest API programmatically to extract inventory and usage data from the service.

Creating an Income Statement with DAX in Power BI – Part 2

This is Part 2 in developing an Income Statement in Power BI. Part 1 can be found here. The example files used for this post can be found here.

An M Function To Help You Explore Power Query Diagnostics Data

Last week’s post showed an M function that took Power Query diagnostics data and formatted in a way that made it suitable for visualisation in a Power BI Decomposition...

Lots of Power BI tips… (Roundup | February 10, 2020)

Thanks for watching this week's Power BI news roun...

Power BI Real-time Streaming

With Power BI real-time streaming, you can stream ...

Professional Development

I can do better; we can do better; let’s do better

In the past week or so, the Microsoft Data Platfor...

I Presented with Live Captioning and Sign Language Interpreters

I had the pleasure of presenting a full-day pre-co...

#TSQL2sday: My Life Hacks

Jess Pomfret (newly awarded MVP!) is hosting month’s T-SQL Tuesday, and asked us to share our favorite life hacks – something that makes our day easier. I have two that I consider indispensable.

Why Empathy is Not Enough in Tech

In 2020, my main personal goal is to reduce doing harm, and this has been mainly inspired by the work of Kim Crayton and the #causeascene Guiding Principles. Kim...

#TSQL2sday: My Life Hack is an Hourglass. Yes, an Hourglass.

For this month’s T-SQL Tuesday, Jess Pomfret asked for our favorite life hacks. My life hack is a half-hour hourglass. (Halfhourglass?) Seriously, an actual physical object: I keep it...

Be Kind

Simply put, we are not always going to agree. Please, take this as someone who was nicknamed “The Scary DBA” for reasons comic and tragic. I’ve screwed up a...

#TSQL2sday: My Life Hacks

  Jess Pomfret (newly awarded MVP!) is hosting month’s T-SQL Tuesday, and asked us to share our favorite life hacks – something that makes our day easier.  I have...

R Language

Three ways to calculate distances in R

Calculating a distance on a map sounds straightforward, but it can be confusing how many different ways there are to do this in R. This complexity arises because there are different ways of defining ‘distance’ on the Earth’s surface.

Replication

Transactional Replication Use Cases

This blog post is a part of a series on SQL Server...

Security News and Issues

Sharenting, BYOD and Kids Online: 10 Digital Tips for Modern Day Parents

Today is Safer Internet Day which marks the annual occurrence of parents thinking about their kids' online presence (before we go back to thinking very little about it tomorrow!)

Why is the healthcare industry still so bad at cybersecurity?

This demanded a cross-industry summit—so now med...

Equifax breach was the work of Chinese state hackers, DOJ says

Act was “a deliberate and sweeping intrusion,”...

T-SQL

T-SQL TUESDAY #123: LIFE HACKING BY OPENING WINDOWS

I have lots of little life hacks, but the one that I want to mention today is Aggregates of Aggregates using Windowing Functions.

String Split Function in SQL Server

Did you know that a native STRING_SPLIT function b...

NULL complexities – Part 3, Missing standard features and T-SQL alternatives

Itzik Ben-Gan discusses missing T-SQL features lik...

T-SQL 101: #56 Date components YEAR, MONTH, DAY in SQL Server T-SQL

Sometimes we have a date and we need to extract components of the date. For example, we might have a date and wonder what's the year or we have...

 
RSS FeedTwitter
This email has been sent to {email}. To be removed from this list, please click here. If you have any problems leaving the list, please contact the webmaster@sqlservercentral.com. This newsletter was sent to you because you signed up at SQLServerCentral.com. Note: This is not the SQLServerCentral.com daily newsletter list, and unsubscribing to this newsletter will not stop you receiving the SQL Server Central daily newsletters. If you want to be removed from that list, you can follow the instructions on the daily newsletter.
©2019 Redgate Software Ltd, Newnham House, Cambridge Business Park, Cambridge, CB4 0WZ, United Kingdom. All rights reserved.
webmaster@sqlservercentral.com

 

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -