• Stairway to SSAS Tabular Level 1: Why use Analysis Services
• Half Of You Don’t Understand Variables and Transactions.
• Free eBook: Performance Tuning with SQL Server Dynamic Management Views
• From the SQL Server Central Blogs - Failing SQL Agent Jobs – Part 1
• From the SQL Server Central Blogs - Podcast recording – Mixed Extents 01 – Cloud vs. On-Prem

• The pyodbc Connection from Python

Daily Coping Tip

Forgive yourself when things go wrong. We all make mistakes

I also have a thread at SQLServerCentral dealing with coping mechanisms and resources. Feel free to participate.

For many of you out there working in a new way, I'm including a thought of the day on how to cope in this challenging time from The Action for Happiness Coping Calendar. My items will be on my blog, feel free to share yours.

Do You Have Big Data?

Data sizes are always growing. Stats on world data are astounding, as are the stats many of us experience in our lives. Plenty of us have moved from MB management to GBs, and I see plenty of people dealing with TB storage at home. Most of that data is likely from images and video, but I wouldn't be surprised to find some people capturing lots of IoT and other random data about their lives.

As data professionals, likely we don't quite have the explosion of data inside or organizations, but we certainly do have increasing database sizes. I see this with customers all the time, often with continued or expanded collection of data and no archival plans. The number of customers with > 1TB databases increases every year.

Overall data volumes are growing at a faster rate than our growth in processing capabilities. That is nothing new, as we've always collected more data than we can easily process. Even if you avoid image, video, and audio data, likely you have more CSVs, extracts, and export files than you know what to do with. Maintaining this data is hard, and all the associated systems is a challenge.

I wonder today how many people think you have big data issues. No matter whether you deal with GBs, TBs, or something larger, do you feel that managing all your data is a big part of someone's job? Can you analyze all your data, at least at the rate your customers would like? Can you keep up with queries and extracts?

This article talks about some of the challenges, and how the problems with big data are going to to continue to plague us for some time. Does the article resonate with you? Or perhaps you feel that you have everything under control. I'd be curious if that is the case. Leave me a comment and let me know.

Steve Jones - SSC Editor

Join the debate, and respond to today's editorial on the forums