Scripts

Technical Article

Real-Time SQL Server to BigQuery Streaming ETL using CDC

CDC Changes: The script queries the CDC tables in SQL Server to retrieve the changes (inserts, updates, deletes) since the last sync. Each change is processed with a mapped operation type (INSERT, UPDATE, DELETE).
Real-Time Streaming to BigQuery: The captured changes are streamed directly to BigQuery using its real-time insert_rows_json method, avoiding the need for batch uploads via Google Cloud Storage.
Tracking Last Sync Time: The script tracks the last synchronization time and updates it after every successful sync, ensuring no data is missed.
Low Latency: By continuously querying the CDC tables and streaming the changes, the script achieves near real-time data synchronization.

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

5 (1)

You rated this post out of 5. Change rating

2024-11-13 (first published: )

229 reads

Technical Article

Concatenating Multiple Row Values into a Single Comma-Separated List

In scenarios where you need to consolidate multiple rows into a single, comma-separated value, you can achieve this using FOR XML PATH. This script demonstrates how to retrieve volunteer data and display the days they have selected for participation.

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

You rated this post out of 5. Change rating

2024-11-11 (first published: )

467 reads

Technical Article

While Loop in T-SQL

Often, we encounter situations where we need to loop through a dataset to process or update records iteratively. In such cases, I use WHILE loop like below. The example below demonstrates how a WHILE loop can be used to iterate Sales records. This approach is particularly effective when working with a numerical or date column, […]

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

3 (2)

You rated this post out of 5. Change rating

2024-10-15

155 reads

Technical Article

Comprehensive SQL Performance Monitoring and Optimisation Script

This script monitors and reports the execution statistics of SQL statements over a specified period, capturing metrics such as execution counts, CPU time, I/O operations, and elapsed time. It provides a comprehensive view of query performance, aiding in the diagnosis of performance issues and identifying problematic SQL queries in the database.

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

5 (1)

You rated this post out of 5. Change rating

2024-07-29 (first published: )

1,271 reads

Technical Article

Creating a CSV Parser in Apex

Salesforce supports the conversion of only csv files utilizing Apex. However if you use their LWC - js platform that provides wide options to bring in libraries to parse all kinds of file types. Here in this script i would be providing basic idea of functions utilizing the parsing logic along with options to iterate […]

★ ★ ★ ★ ★ ★ ★ ★ ★ ★

5 (1)

You rated this post out of 5. Change rating

2024-05-21

8,129 reads

Blogs

How to Run Databases on Kubernetes: An 8-Step Guide

By

In this step-by-step tutorial, learn how to run MySQL, PostgreSQL, MongoDB, and other stateful...

Episode 11 of Simple Talks: Oracle

By

The 11th episode is now live, recorded a few weeks ago at the PASS...

A New Word: Mornden

By

mornden – n. the self-container pajama universe shared by two people on a long...

Read the latest Blogs

Forums

Giving Thanks

By Ryan Booz

Comments posted to this topic are about the item Giving Thanks

What is the best index strategy for a table that gets truncated?

By water490

Hi everyone My SSIS package does a bulk insert of csv files into a...

Blob Storage automated downloads

By Brandie Tarvin

Dipping my toes into the waters of Azure and of course before I get...

Visit the forum

Question of the Day

Running Steve's Code

Can you run this code in any of your SQL Server 2019 databases without error? 

CREATE OR ALTER PROCEDURE [dbo].[StevesAmazingProc]
AS
    
        SELECT Consumer_ID ,
               Trend_Category ,
               Bit_Trace
        FROM    NewWorldDB.dbo.MarketTrend;
    
GO

See possible answers