A Technology Collapse

This past week the Arecibo radio telescope collapsed, with cables and instruments falling into the massive dish. You can see images of the devastation, which saddens me. I doubt this will be rebuilt, though one can keep hope alive. there were reports of failures a month or so ago, with the intent to shut down the telescope, remove instruments, and institute a controlled demolition.

The facility was featured in a few movies, notably GoldenEye and Contact. The latter is one that first showed me the telescope, which I first thought was a movie magic trick. When I found out this was a real facility, I was amazed that humans could conceive and build such a structure. Even more amazing is that it was built in the 1960s.

That's quite a lifetime for a piece of technology. While I'm sure lots of wiring, electronics, computer systems, and more were added or upgraded, the core of this telescope remained the same. The ongoing engineering effort, fitting new capabilities around legacy systems and structures, was likely a study that many would find interesting.

I don't know many database systems that have survived that long, but certainly there are some long lived legacy applications. The Sabre reservation system for airlines is one that stands out to me. The history of this one is fascinating, especially as it's a system I've depended on in my travels.

Many of us that work with databases likely feel that everything we do is a legacy system. The dependencies our objects have on various applications, scripts, reports, and more can limit how much repair, improvement, or replacement we can make to schema.

There are good techniques for modifying our objects, and helping to ensure we don't break systems, but we often do need some cooperation and collaboration with application developers to implement those changes. Much of DevOps avoids talking about the database, but we shouldn't. Instead, we ought to embrace database refactoring patterns, both at the database and application levels, ensuring that our systems can survive for as long as we need them while adapting to the changing requirements of our clients.

Steve Jones - SSC Editor

Join the debate, and respond to the editorial on the forums

As more organizations are managing at least some part of their data in the cloud, estate monitoring can become more complex with a mix of on-prem and cloud-based instances. Grant Fritchey discusses why businesses might move to the cloud, why they might stay on-prem, and when a hybrid approach might be best.

Learn and share tool tips with your peers. Redgate has launched the Tool Tips Swap to help everyone find new ways to use their Redgate products. To get involved, share your top tip for increasing productivity with Redgate tools. All tips will be collated into a blog post and shared with the community. Plus, you’ll get the chance to win a 3-month subscription to Pluralsight.

Become the captain of your SQL Prompt ship by embarking on a voyage of discovery through the hidden treasures of the tool. You'll uncover features of the unknown that will enable you to write, format, analyze and refactor your SQL effortlessly.

It's your last chance to take part in the 2021 State of Database DevOps survey! If you or your organization uses databases, we want to hear from you! Complete the survey for advance access to the final research report, and to be entered into a prize draw for a $500 Amazon voucher. Plus, for each submission we receive, we’ll donate $1 to the World Health Organization’s COVID-19 Solidarity Response Fund.

Phil Factor takes a strategic look at common SQL data masking techniques, and the challenges inherent in masking certain types of sensitive and personal data, while ensuring that it still looks like the real data, and retains its referential integrity, and distribution characteristics.

Have you ever had to demonstrate a database-driven application, and longed to have the real data to do so? To do what, precisely? Well, so you can then scroll through the customers who have used the system, demonstrate the accounting and audit, browse through the products, maybe even demo the customer tracking system with comments, complaints and so on. All this is possible, using realistic, fake data.

Phil Factor explores a lesser-known capability of SQL Compare, which is to help you maintain a 'traditional', well-documented, single file build script, for creating the current version of the database during development.

We’re inviting you to take part in the 2021 State of Database DevOps survey! Now in its 5th year, the leading research on Database DevOps continues to explore how database development practices are evolving. Complete the survey to get exclusive access to the final report and the chance to win a $500 Amazon gift card. Take the survey.

High performing teams must implement a common approach to database migrations so that the database can be included in Continuous Delivery. This ‘kernel’ of Database DevOps leads to better alignment between developers and Ops teams, and increases the throughput of high quality releases. The key for many is Flyway, and this whitepaper explores how this highly configurable migrations tool enables a bespoke migrations-based deployment strategy that is easy to adopt and scale across teams. Learn more.

f you want to gain an even deeper understanding of the fascinating connection between those two popular machine learning techniques read on!

Deep Learning theories have been around for many decades, but solutions have not always been practical due to hardware constraints. In this article, Shree Das explains how GPU Acceleration...

Google's AI specialists tackle biology's toughest computational problem.

Are you interested in guest posting? Publish at DataScience+ via your RStudio editor. Category Basic Statistics Tags Decision TreesDecision Tree falls under supervised machine learning, as the name suggests it is a...

Do you need SQL Server Enterprise Edition? Maybe. Maybe not. Lately, we’ve been finding more and more folks who can answer that probably not. I’ve lost track of the...

We recently had customer ask why SQL Monitor creates an Extended Events session to capture deadlock graphs, when SQL Server has a built-in system_health Extended Events trace which also...

Series of Azure Databricks posts: Dec 01: What is Azure Databricks In previous blogpost we have looked into what Azure Databricks and its’ main features, who is the targeted...

Series of Azure Databricks posts: Dec 01: What is Azure Databricks Dec 02: How to get started with Azure Databricks We have learned what Azure Databricks is and looked how...

Last week I gave a presentation at Data South West on dbachecks and dbatools. One of the questions I got was whether you could run dbachecks against Azure SQL Databases, to which I had no idea. I always try to be prepared for potential questions that might come up, but I had only been thinking about on-premises environments and hadn’t even considered the cloud. The benefit is this gives me a great topic for a blog post.

Quick Video showing you have to use a BACPAC to “import” a database into Azure (Via Storage container), The post [Video] Azure SQL Database – Import a Database appeared first...

Without a good backup and recovery strategy in place, a DBA should make sure their resume is always up to date. In this article, Greg Larsen discusses what should be considered when developing a backup plan.

For the past couple weeks I’ve been trying to capture a lot of ideas about how and what and why we do things in Orlando and put them into... The...

I first starting presenting on mental health last December in Charlotte, NC on mental health.  I got some backlash at a couple of events for a few people on... The...

Amazon goes after Microsoft’s SQL Server with Babelfish for Aurora PostgreSQL. What is interesting is that a direct port of SQL Server to PostgreSQL or Aurora PostgreSQL can be...

When we launched S3 back in 2006, I discussed its virtually unlimited capacity (“…easily store any number of blocks…”), the fact that it was designed to provide 99.99% availability,...

Last updated: 8:30 am (PST), December 1 Join AWS O...

Embark on a voyage of discovery through the hidden treasures of SQL Prompt. Join Microsoft MVP Grant Fritchey and Redgate's Owen Standage to uncover features of the unknown that will enable you to write, format, analyze and refactor your SQL effortlessly.

In the first part of this series, we discussed the joint statement from AIR, EDUCAUSE, and NACUBO and its impact for supporting students:

Deep Learning theories have been around for many decades, but solutions have not always been practical due to hardware constraints. In this article, Shree Das explains how GPU Acceleration can help organisations take advantage of Deep Learning to speed up training of neural networks.

To stay competitive and bring value to customers, organisations are adopting DevOps, but finding people experienced in DevOps can be challenging. Robert Sheldon has ten tips to help managers attract and retain DevOps talent.

If your Data Factory contains a self-hosted Integration runtime, you will need to do some planning work before everything will work nicely with CI/CD pipelines. Unlike all other resources in your Data Factory, runtimes won’t deploy cleanly between environments, primarily as you connect the installed runtime directly to a single Data Factory.

I’m excited to announce Building Custom Tasks for SQL Server Integration Services: The Power of .NET for ETL for SQL Server 2019 and Beyond 2nd Edition is available for pre-order at...

As a follow up to my blog about Data Factory resource limitations here. I decided to dig deeper and expose some of these limitations with a view to understanding...

Oracle sometimes generates smaller archived redo logs than you might expect based on the size of the online redo logs. In this article, Jonathan Lewis explains how Oracle uses online redo logs and why there might be a discrepancy in the archived log size.

I Know You You have too many indexes on too many tables already, and the thought of adding more fills you with a dread that has a first, middle, last,...

You may be able to write a DAX expression, but wha...

Introduction In my last blog post, I have addressed an issue with DAX Median function consuming a lot of memory. To refresh the result, below is the performance summary...

Recently I had a client that wanted to know how they could use SQL Compare to catch actual changes in their code, but not have comments show up as... The...

Happening now! Join us for our liveblog of the AWS Partner Keynote, Thursday, Dec. 3, from 7:45-9:30 a.m     Analytics | Compute | Containers | Customer Engagement | Database...

We launched the compute-intensive z1d instances in mid-2018 for customers who asked us for extremely high per-core performance and a high memory-to-core ratio to power their front-end Electronic Design...

With AWS Lambda, you upload your code and run it without thinking about servers. Many customers enjoy the way this works, but if you’ve invested in container tooling for...

I love 538’s Riddler column, and I’ve enjoyed solving the November 20th puzzle.

Data professionals get requests to import, export data into various formats. These formats can be such as Comma-separated data(.CSV), Excel, HTML, JSON, YAML, Tab-separated data(.TSV). Usually, we use SQL Server integration service ETL packages for data transformations, import or export data.

Every so often I have been asked “how it is possible to log and review the logs when Developers change objects but do not have access to the SQL Audit?”. Now, even a security person like me should try to understand the real motivation for this before jumping to the conclusion that Auditing is the only right solution.

This is one of them little options that I see whic...

Total: 1 Average: 5 The previous article explained the creation of a virtual machine using Oracle VM Virtual machine. Now, we will learn how to install Ubuntu 18.04 and...

This is a scarily impressive vulnerability: Earlier this year, Apple patched one of the most breathtaking iPhone vulnerabilities ever: a memory corruption bug in the iOS kernel that gave attackers...

The government’s new reading of the CFAA surprised legal scholars.

Total: 4 Average: 5 Friend or foe? SQL Server views have been a subject of heated debates when I was in my first year using SQL Server. They said...

You may already know that common table expressions, like views, don’t behave like regular tables. They’re a way to make your query more readable by allowing you to write a complex SQL expression just once, rather than repeating it all over your statement or view. This makes reading, understanding and future refactoring of your code a little less painful.

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. Recently I saw Jason Horner do a presentation... The...

You may already know that common table expressions, like views, don’t behave like regular tables. They’re a way to make your query more readable by allowing you to write...

Sing Along If you have a workload that uses #temp tables to stage intermediate results, and you probably do because you’re smart, it might be worth taking advantage of being...

Entire support system fails, all cables and instruments crash into the dish.

It’s 2020: Do your museum heists, archeological digs, and alchemy tests from home.

Using multiple languages is a huge advantages when people choose notebooks over standard code files. And with notebooks in Azure Data Studio without switching the kernels, you can stay...

A project I’ve been meaning to work on for a while has been to build my own Kubernetes cluster running on Raspberry Pis. I’ve been playing around with Kubernetes... The...