Put Your Data in a Box

A few years ago I was attending a keynote talk from one of the scientists that works CERN's Large HADRON Collider (LHC). The talk was about data and how they deal with a large volume of data from experiments run on the machine. If you're wondering what large is, the scientist talked about peak experimental data being over 1PB/s. I don't know how much data you capture, but that's a lot.

In fact, it's so much, that if they tried to analyze all that data, they'd never get around to running experiments. Instead, they depend on some pre-processing of data in sensor hardware as well as some early aggregation to get the data to a manageable level. It's a good idea, and they have spent a lot of time and effort learning how to do this and still capture meaningful data for their work.

The capability to remotely process data before sending it on is coming to all of us in a pre-packaged container. The Azure Data Box Edge was announced this week from Microsoft. This is a data processing device that is cloud managed and has FPGAs that you can program. It can run on batteries and is ruggedized for the field. There are more docs at Microsoft on the specifics.

I don't know how many companies want this, but I suspect that some who have remote or portable operations might think about it. Certainly if this can gather some data and then upload when a connection is available, it might be a good fit for places that don't have good network connections and need a device that can handle some adverse conditions. I know sourcing and putting together a system for field offices is a pain. Off the shelf systems don't always have reliability, and can be finicky to manage remotely.

Recently I hosted a webinar with Abel Wang, and he talked about AI and ML being technologies whose use will grow dramatically in the next ten years. Perhaps a prediction without merit, but it does seem more and more companies and vendors are putting efforts into finding ways to deploy and operate ML systems. The Data Box Edge has capabilities in that area, which could be useful if you want to locally, and quickly, process data.

This doesn't appear to run SQL Server, or it's not mentioned, but it does seem useful. If you doubt this, I know this is exactly what I would have wanted over a decade ago. I had to install systems in a warehouse to visual inspect some products. We used crude AI-ish systems that were set up to watch evaluation by humans. Eventually, the computer took over part of the job, but with humans randomly verifying its results. Keeping that system running was a pain. A Data Box Edge would have been a much better choice, and I'm sure we would have purchased one.

There are likely plenty of customers that might be able to use this. It will be interesting to see if Microsoft can find them and sell many of these devices.

Steve Jones - SSC Editor

Join the debate, and respond to the editorial on the forums

Your PowerShell process for database comparison and deployment is running smoothly, and your instinct is to 'leave well alone'. However, you continually need to delve into the script to tweak comparison settings or add a new command line switch. Phil Factor proposes a way to avoid this using SQL Compare projects.

This article explains the basic components of a tech stack, and the decisions and requirements that affect how and where monitoring tools generally, and a specialist SQL Server monitoring tool like SQL Monitor, in particular, fit into your stack.

Video games often contain animation of the objects and characters, but how is that accomplished? In this article, Lance Talbert explains how a Unity plugin called iTween can make creating animation easy.

While installing SQL Server 2012 in a client envir...

Introduction: Today we are going to learn how to p...

I walked into a client engagement where I really g...

Twitter is a strange place. I mean, aside from all...

Total: 0 Average: 0This article talks about automa...

Last week I was asked to tackle a requirement by a...

This post is a response to this month’s T-SQL Tu...

It’s that time again! Alex Yates (b/t) has asked...

This month Alex Yates (B|T) is hosting T-SQL Tuesd...

For this month’s T-SQL Tuesday, we are to write ...

Alex Yates (b | t) is hosting this month’s T-S...

This post is for the most recent #tsql2sday, a mon...

Earlier this year, I published the first crack at ...

Let’s create a virtual machine in Azure that has...

Training Daze I’ve delighted to announce that I?...

Wednesday October 23 16.00-17.00 BST/ 10.00-11.00 Central - Discover different tactics enabled by Redgate’s latest new product - SQL Data Catalog – to build a clear picture of your SQL Server estate, enabling you to ensure that the correct data is protected in the correct way.

Wednesday October 16 16.00-17.00 BST/ 10.00-11.00 Central - Microsoft MVP, Kendra Little will show you how Redgate tools enable you to speed up software delivery with reliable, repeatable processes that remove the database bottleneck.

Redgate is hosting an upcoming series of exclusive Summits, in cities worldwide, to help senior data professionals deliver value quicker, while keeping their data safe. Find out more here and register for your nearest event

(Be sure to checkout the FREE SQLpassion Performan...

My great organization, Redgate, is doing things a ...

I’m taking a Powershell class this week and I ha...

Another post for me that is simple and hopefully s...

Loops are one of those crucial logic components th...

Instead of writing cumbersome case when statements...

Click to learn more about author Josh Bartels. Eco...

Machine learning is a skill that many data professionals are learning as they plan their careers over the next five to ten years. In this article, Supriya Pande gives an overview of machine learning and walks through a practical example.

Download slides and demos of this session using th...

DevOps might sound intimidating if your organisati...

Total: 0 Average: 0In this article, I am going to ...

eWEEK PRODUCT REVIEW/ANALYSIS: ThinkPd X1 is for t...

Intel has announced End-of-Life plan for most of i...

Samsung on Monday said that it had developed the i...

The XPS 13 many have been waiting for, but it may ...

G.Skill has now rolled out its 32 GB unbuffered DD...

We recently saw the launch of the consumer Cascade...

In another blog article, I explained how easy is t...

We started out the How to Think Like the Engine se...

Lost In The Woods When you find unused indexes, wh...

In the first post in my How to Think Like the Engi...

You’re a developer or a DBA, and you’re comfor...

Report development is iterative. Every good report...

Chris Webb joins Patrick to look at what happens w...

Patrick shows how you can easily use parameters, f...

Patrick has a conversation with the Power BI Queen...

Case sensitivity is one of the more confusing aspe...

Every DBA should have basic PowerShell skills. In this article, Greg Moore explains how to write a PowerShell script that takes parameters.

There is an old joke that upgrading to the latest ...

Phil Factor starts with the basics how to rebuild ...

Last year we launched EC2 High Memory Instances wi...

Microsoft is readying a ruggedized version of its ...

In March of this year, we announced a preview of W...

On October 8, 2019, Microsoft released SQL Server ...

In recent years, the meteoric rise of deep learnin...

Most SSRS reports are organized into grouping levels where it’s possible to add totals for each group. In this article, Kathi Kellenberger continues her SSRS series by walking through how to create both a Matrix and Table report.

Lately, I've been needing to write queries against...

Kaspersky has a detailed blog post about a new pie...

Billions who only connect with cheap Android phone...

Officials have obtained the decryption key and are...

Contractor looked up relatives; data was used to v...

Date and time values are not entirely intuitive to...

Itzik Ben-Gan discusses lesser-known T-SQL feature...

SCHEMABINDING keywords prevent tables used in the ...

Today, I'm continuing the discussion on the variet...

I received a very interesting question from the us...

Buggery If you seriously want to debug, you’ve g...

I’ve been teaching a lot more about SQL Injectio...

Interesting story about someone who is almost cert...

Nicklas Bågvinge sent me a very interesting SSMS ...

I have received many curious emails and questions ...

Let’s get you started on your Kubernetes journey...