The Pervasive Nature of Open Source

This week there was a headline that said "Open Source Software Powers 96% of Modern Applications, New Study Finds" and if you stopped reading there, you might think, hey, it's not in the apps I work on. Or you might think that because you use OSS software, most of the world also does. Microsoft, Oracle, etc. are headed for disaster.

If you read a little further, there's this gem: Open source components are present in 96% of codebases. That's a far cry from OSS powering most modern applications. I think a better headline might be that OSS helps build most modern software. However, this isn't an editorial on bad journalism.

I do think OSS stuff is amazing. Many of us in the Microsoft Data Platform space use sp_whoisactive or the First Responder Kit or Diagnostic Queries or some other OSS in our work. Lots of commercial products are built with OSS libraries or components, or there are free versions. Flyway (from Redgate) has an OSS version. OSS helps us build better software, though commercial packages also help.

One of the interesting things about OSS is that many very popular projects have just a few people maintaining them. If those 1 or 2 people disappear, then the project might stagnate. Or worse, if there are security issues, no one addresses them. One of the main attractions of OSS is that anyone can provide a fix or enhancement, but the reality is that most people don't. Most people just use whatever is out there.

And most people rarely upgrade their OSS. They get something that works and don't want to change. I get that, as I feel the same way often, especially in the real world. I have shoes, gadgets, tires, etc. that work and I don't want to change. In the software world, this creates vulnerabilities and security issues, as the report shows with many people still using Python 2. I both understand and don't understand why this is the case, but I do worry about security.

The other concern is that few people review changes to OSS packages, which has led to previous supply chain attacks with backdoors or vulnerabilities introduced in packages that many other software developers use. Again, OSS is supposed to be better than closed software at preventing this, but the reality is that most (the vast majority) of us are just too busy to look for issues. Even when vulnerabilities are published, far too few developers see the information. Automated scans in CI/CD systems are great, but again, too few people add these to CI/CD pipelines consistently.

Software is hard. In some sense, I'm glad databases don't have external compilers or use anything other than raw code, but plenty of people still write SQL Injection vulnerabilities in their functions and stored procedures, and many don't have good visibility into the code that is submitted to their databases, often because the code is assembled at runtime. I wish more people just used stored procedures and included more testing and vulnerability scanning, but that's a dream. For now, I suggest most of you developer patterns your staff can use and stick with them.

And use version control. At least then we can find all the old, bad code and fix it with some search and replace.

Steve Jones - SSC Editor

Join the debate, and respond to the editorial on the forums

This article demonstrates using PowerShell-based tokenization to compare two SQL migration files. It ignores non-functional changes like comments or formatting and pinpoints the first meaningful change in SQL logic, providing detailed feedback on its location and nature.

The FBI now recommends choosing a secret password to thwart AI voice clones from tricking people.

Microsoft Azure offers multiple services that enab...

In this Microsoft Azure AI series: Azure AI Foundr...

In this Microsoft Azure AI series: When you are in Azure AI Foundry, on the left navigation bar, select “Model Catalog”. For this demo, I will be selecting multimodal...

In this Microsoft Azure AI series: In Azure AI Foundry yo u will be able to create project that will keep your solution together. Select the “+ Create Project”...

I have a new video: In this video, I show how to back up a database directly to AWS S3-compatible storage (in SQL Server 2022)…

Is your company hiring for a database position as of December 2024? Do you wanna work with the kinds of people who read this blog? Let’s set up some...

In this coffee chat episode of Simple Talks, Louis spends nearly an hour chatting with SQL Server community champion Mala Mahadevan. Learn how Mala got into computing despite her passion for horticulture, her favorite thing about working in technology, and a love for national parks. And stick around to the end where Mala and Louis share their frustrations about a SQL Server feature they both love but really want to see enhanced, so it will be a great feature worthy of use by everyone, everywhere.

PASS Data Community Summit will return to Seattle next year! Save the date for this incredible in-person event for global data professionals, which will take place at Summit, Seattle Convention Center, from November 17-21, 2025!

Announced during Redgate's Keynote at PASS Summit in Seattle, PASS Summit On Tour will see smaller scale events hosted in New York, Dallas and the Netherlands in 2025. To be the first to know when tickets and dates are released, sign up to our mailing list.

A remote session can be set up with the help of PSSession. The predefined remote session is used by default. However, we can also create our own session configurations...

Upload your photo and get a thorough, three-paragr...

A real-estate company optimized its model with DAX...

Implications of having blank values in date columns and best practices for managing them in DAX calculations and Power BI reports.

At #MSIgnite Microsoft announced a new feature in Fabric that allows people from one organization to share data with people from another organization. You might ask yourself why is this even news, and rightly so. Up until last week, professionals have had to use tools like (S)FTP clients like FileZilla, Azure Storage Explorer, WeTransfer or similar products in order to share data. Some of these tools are in fact hard to use and/or understand for a great number of business users – they are familiar with Windows and the Office suite and not much more. This is all to be expected, as business users in general should focus on business stuff rather than IT stuff.

Polars provides a happy medium between pandas and spark

We are becoming used to being a bit lazy when granting permissions to Data Warehouses and lakehouses in Fabric. We only go to the workspace level and add the...

Oracle Database, often referred to simply as Oracl...

B-Tree indexes have multiple types: they can be covering, composite, descending, FULLTEXT, UNIQUE, hash-based, or have something to do with the PRIMARY KEY. B-Tree indexes can also have a clustered form: and that form is what this blog is all about.

Simulating WAITFOR In Scalar UDFs In SQL Server Thanks for watching! Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love my...

Finding Bad Density Vector Estimates In SQL Server Thanks for watching! Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love my...

This post comes off the back of my last, where I looked at issues caused by explicitly declaring a large number of values in an IN clause. The query... The...

Want to make your Import or DirectQuery Power BI S...

Video by: Reid HavensLearn about the basic benefits of utilizing Deployment Pipelines in Microsoft Fabric / Power BI, saving you time, effort, and reducing complexity for report/model management. Tune...

This article demonstrates using PowerShell-based tokenization to compare two SQL migration files. It ignores non-functional changes like comments or formatting and pinpoints the first meaningful change in SQL logic,...

Many of my customers are using Flyway Enterprise to create migration scripts that will then be used to deploy database changes. They’ve been using Flyway Desktop, but some of...

Discover, test, and use over 100 emerging, and specialized foundation models with the tooling, security, and governance provided by Amazon Bedrock.

Introduction These are my SQL Server Diagnostic Information Queries for December 2024, aka my DMV Diagnostic Queries. They allow you to get a very comprehensive view of the configuration...

Find out how to easily identify columns in your R data frame that contain only missing (NA) values using base R functions. Streamline your data cleaning process with these simple techniques.

Unlock insights from your data by learning how to interpolate missing values in R. Explore practical examples using the zoo library and na.approx() function. Become a master of handling missing data with this step-by-step guide.

This past week I stumbled across an ODBC Scalar Function for the first time. What was this which lay before me? Is that SQL with curly braces?! It returned...

I had mentioned some new T-SQL functions for SQL Server 2022 and a commenter asked about the difference between Min() and First_value. This post looks at a few cases.... The...

This article explores importing form data from a PDF file into a SQL Server database using a Visual Basic Windows Forms App.

In contrast to Biden, President-elect Trump is expected to adopt a hands-off policy toward AI development. The post Trump Set To Loosen AI Regulations appeared first on eWEEK.

Cloud provider moved most of its 20,000 VMs off VM...

A startup called Exa is pitching a new spin on generative search. It uses the tech behind large language models to return lists of results that it claims are...

The market surrounding data management tools and technologies is quite mature. After all, the typical business has been making extensive use of data to help streamline its operations and...

Microsoft won't lower Windows 11's requirements to save older Windows 10 PCs.

Amazon commits $100M to empower education equity initiatives, enabling socially-minded organizations to create AI-powered digital learning solutions. This aims to reach underserved students globally through innovative platforms, apps, and...

The Linux Foundation's Census III report reveals critical dependencies and growing security concerns in open source software.

This story is from The Algorithm, our weekly newsletter on AI. To get it in your inbox first, sign up here. President Biden first witnessed the capabilities of ChatGPT in...

Many startups face hidden time costs when relying on “free” open source testing tools.

Not quite an electric M5, it's a good driver's car.

fardle-din – n. a long-overdue argument that shakes up a relationship, burning wildly through your issues like a forest fire, which clears out your dry and hollow grievances and... The...

This will all happen nearly 40,000 miles above the...

Most of the major EV makers and charging networks are on board.

Save space, organize, and automate your desk space with these four doodads.

Below shows a very simple Kubernetes Network Policy object. This simply opens up port 80 to the outside in a locked-down environment. The key tags to understand are run,...