The Social Impact of Data

Let the data drive your decisions.

This has been something of a mantra for many technical people, and even many business people, across the last twenty or so years. The allure of business intelligence is harnessing lots of data to make decisions that are rooted in some rational analysis of what has happened. Many companies use "data driven decisions" as a way of achieving success.

However, what about when the data is flawed? When deliberate or inadvertent actions give us data that isn't quite as pure as we expect. In the last week we have seen many protests and complaints about the ways that many people feel they have been unfairly treated by police. That brutality, particularly for African Americans in the US, has been a problem for decades. Some of that is due to human biases, beliefs, and more. However technology plays a part, and will for some time to come.

I have watched as algorithms have been used in sentencing, and I've questioned their use, as have others. There is this idea that computers will be more fair, looking at inputs and making a decision that isn't encumbered by human biases. The problem is that humans that program the systems might have some bias. Maybe more disconcerting is that the data used to train systems is likely biased as well.

There is also the concern that as technology advances, it can be put to new uses, perhaps in ways that the inventor regrets. Oppenheimer regretted the violent use of his work, and I wonder if technology inventors will feel the same way. Surveillance technology is controversial. It can help retails companies prevent theft, but it can also be used in ways that might enhance and reinforce bias in police work. This can be controversial, and no matter how you may feel about the technology, there are moral questions of privacy and prejudgment that are worth debating.

The last couple weeks have saddened, upset, and angered me at different times. I am also confused and concerned, unsure of how to discuss and debate these topics. I find my position moving slightly with different stories and different information, as I should. I learn more and my views grow and change, shaped by what touches me. I do worry about how we use data in the future, and how it can be abused. There are ways in which more data can help improve our world, but the potential for abuse is high, and I do believe we need governance, transparency, and an independent appeal process for those wrongly impacted.

Steve Jones - SSC Editor

Join the debate, and respond to the editorial on the forums

Help Redgate to build the next generation of Database DevOps solutions so that we can help you to deliver the next generation of your software by completing this short 2-page survey on current development practices.

From performance to recruitment, cloud adoption to estate growth, the findings from the 2020 State of Database Monitoring report offer us some key insights into the trends and challenges facing the industry this year. Redgate’s Jamie Wallis unpicks the findings in this latest blog.

This story might sound familiar for most of the DB...

Recently we had a new customer contact from someone with a corrupt database. They were getting corruption error #824, which usually indicates there is some type of corruption in...

I’m working on a project where it’s useful to automate environment setup and teardown for testing some devops deployment scenarios for databases using transactional replication. To make this easier,... The...

The more your SQL Server is out of date, the more things become vulnerable, broken and unfixable, limited in features, and expensive. The post What happens if your SQL Server... The...

Ever wonder what’s “normal” out there for SQL Server adoption rates, hardware sizes, or numbers of databases? Let’s find out in the spring 2020 version of our population report....

Databricks is fantastic, but there is a small issu...

Parquet is one of the most popular columnar file formats used in many tools including Apache Hive, Spark, Presto, Flink and many others.

So you have many databases in Azure, you accidentally deleted one which can happen and has happened to me because in my situation they had very similar naming conventions... The...

Scale your data in Power BI with Azure Synapse Ana...

After many months of waiting, Amazon today has finally made available their new compute-oriented C5a AWS cloud instances based on the new AMD EPYC 2nd generation Rome processors with...

Did you know that the system_health Extended Event session was running in your RDS instances? Well, it is. HOWEVER. This query, which works perfectly fine on my on premises...

In the last few posts, we were discussing about database deployment model and purchase model. Out of the three deployment models, managed instance works almost like on-premises SQL server... The...

I’m expanding my skill set into AWS in a big way. So, one of the things I do when I’m learning a technology is to write blog posts about... The...

Hey all, this is a historic moment in the SQL Community. PASS Summit will be virtual this year.  As a PASS Blogger I wanted you to know of the...

An online conference is coming on June 17, 2020 that is full of high level content. It’s the type of event that I think contains information relatively few people... The...

Distributed teams of all sizes need the ability to quickly develop and deploy quality code. Join Kendra Little to discover ways to increase your team’s rate of deployment to production databases and continuously improve code quality. Date and time: Wednesday June 10, 12-1pm CDT / 6-7pm BST

In our 2020 State of Database Monitoring report migrating your databases to the cloud, and monitoring them once there, came out as the biggest challenge. Join us to discover the best practices for overcoming these challenges. Date and time: Wednesday June 24 10-11am CDT / 4-5pm BST. Can't join us live? register to receive the recording.

Distributed teams of all sizes need the ability to quickly develop and deploy quality code. Join Kendra Little to discover ways to increase your team’s rate of deployment to production databases and continuously improve code quality. Date and time: Wednesday June 10, 12-1pm CDT / 6-7pm BST

Join Microsoft's Bob Ward and Anna Hoffman to learn about the choices you make in Azure SQL and how they affect your performance and availability. Date and time: Wednesday June 17, 11am-12pm CDT / 5-6pm BST. Can't join us live, register to receive the recording.

We have seen how we can export and save the results to a folder and commit them to a GIT repository on my last blog post Backup your SQL... The...

Schools hope tracking beacons can identify where s...

I just published a new paper with Karen Levy of Cornell: "Privacy Threats in Intimate Relationships." Abstract: This article provides an overview of intimate threats: a class of privacy...

I recently posted a graph to twitter and asked people to explain it. Let’s look at the graph. The graph is from Fitbit. It shows the number of steps... The...

Since the release of the State of DevOps report by Puppet Labs ten years ago, various organizations have looked at how we could build better software. The generally agreed...

The terms Continuous Integration and Continuous Deployment tend to be combined into the acronym CI/CD, often without any distinction between the two. CI and CD are distinct processes, even if this combining of...

Every ETL pipeline is only ever as reliable as the data that the upstream system provides. It is inevitable that assumptions you make about the data you are provided...

Could you spare 3 minutes to help us out at SQL Server Central? Take this short survey, tell us about who you are and why you come to SQL Server Central, and you could win a $50 Amazon voucher.

Could you spare 3 minutes to help us out at SQL Server Central? Take this short survey, tell us about who you are and why you come to SQL Server Central, and you could win a $50 Amazon voucher.

I recently helped someone build an iSCSI target server cluster in Azure and realized that I never wrote a step-by-step guide for that particular configuration. So to remedy that,... The...

Western Digital's SMR disks won't work for ZFS, but they're okay for most NASes.

The PC industry has introduced some remarkably exciting designs over the last five years or so. Some of those designs, such as the thin-bezel laptop, have been adopted by...

In my free How to Think Like the Engine class, I explain that SQL Server builds execution plans based on statistics. The contents of your tables inform the decisions...

Beginning OF The Tran This week I’m giving away a sampling of the videos from my Locking and Blocking course. If you want to see the entire thing, it’s available this...

Scale your data in Power BI with Azure Synapse Ana...

With Deployment Pipelines, you can control your ap...

Today I want to share how I’m keeping a copy of instances’ configurations using dbatools. Chrissy LeMaire (B | T) wrote about it before on the Simplifying disaster recovery... The...

In my last post I demonstrated how the size of a table affects the performance of Power Query merge operations on non-foldable data sources in Power BI. Specifically, I showed...

Super quick tip, I’ve seen this done via CMD or DaxStudio, but not via PowerShell (probably because I haven’t searched so much since I’ve done it myself), but here we are:

Merging (or, in SQL terms, joining) tables in Power Query is a common cause of refresh performance problems. I’ve often wondered whether there’s anything you can do to optimise...

Phil Factor on the fill factor, pages splits and index fragmentation, and how SQL Monitor can help you decide if a custom fill factor for certain indexes might help...

How well do the SQL Server licenses you own match up with what you need for your current use? William Durkin explains how to use SQL Monitor to find...

One of the coolest and most useful features of Azure Data Studio (ADS) is Notebooks. If you haven’t seen them, notebooks allow you to combine text with markdown and...

If you’re saving the contents of sp_BlitzFirst, sp_BlitzCache, and sp_BlitzWho to tables every 15 minutes, you should definitely install this month’s updates. There are several key improvements for you...

Learning never stops. When SAS had to change this year’s SAS Global Forum (SASGF) to a virtual event, everyone was disappointed. I am, however, super excited about all of...

[MSSQLFUN & Decode ITeS becomes a family of 1500 individuals across the globe. Please join us on Facebook & YouTube.] We will be discussion SQL Server Authentication and below topics:- What... The...

This is the second in a series of posts about how to survive an audit. These posts will include some basic guidelines I have learned over the years as well as how to get the information I have been asked to provide.

New high-pressure tactic is designed to increase the chance of a hefty payout.

This study shows that most people don't change their passwords after a breach, and if they do they change it to a weaker password. Abstract: To protect against misuse...

In a recent Twitter thread, Adam Jacob (co-founder and former CTO of Chef) talked about Chef’s switch from an “open core” model to a a “Red Hat” model for...

If you’ve not come across regular expressions (RegEx) before then you’re missing out, they’re an incredibly powerful tool for matching, extracting or validating patterns of text. I had a...

On Twitter, Michael Dyrynda writes: ?? Never ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever... The...

I wrote about sequences in an editorial recently and decided to start using them. Creating one turned out to be surprisingly easy. I had a table I’d been logging... The...

In previous posts, I mentioned that you can easily extract the year, month, and day from dates and times, but you might want other components. That's what the DATEPART()...

Total: 6 Average: 1.7 This article considers how to perform basic product analysis using Full-Text Search. Additionally, the readers are going to learn some practical examples of product analysis...

Hamid Khan has been a community organizer in Los Angeles for over 35 years, with a consistent focus on police violence and human rights. He talked to us on...

Black Americans have seen technology used to target them again and again. Stopping it means looking at the problem differently.

“The House bill you referenced, I think, is unfo...

New study shows how to expand and manage a grid to limit fossil fuel use.

“It flew really well, very crisp. We couldn't be happier.”

Today, we’re going to go through the process of turning a database you’ve built into a Docker container. Before we get started, here are the expectations: I want a...