How Effective Is Your Data Load Monitoring in SQL and Python?
Learn how you can use monitor your data warehouse load process with Python code and a structured process.
2023-10-06 (first published: 2023-09-20)
2,014 reads
Extract, Transform, Load (ETL)
The Art of Data Ingestion - Part 1
Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]
2022-11-11
6,070 reads
2021-04-16
552 reads
Function to find non-printable characters
The function is used to find non-printable ASCII characters in an input string.
2009-05-21 (first published: 2009-04-24)
2,879 reads
Kimball ETL Part 1 – Data Profiling via SSIS Data Flow
A new series sponsored by Actuality Business Intelligence on data warehousing. In part 1, the data flow in SSIS packages are used to profile the source data and determine how it should be handled in the process.
2008-07-23
9,922 reads
ETL for Free-Form Data
This article demonstrates how to extract, transform and load free-form data into a structured form like a staging table by way of XML conversions.
2008-03-20
5,573 reads
2008-01-22
2,966 reads
2007-12-10
3,004 reads
Importing Text-based data: Workbench
Robyn and Phil return with some fresh ideas about how to import text files into SQL Server, without resorting to DTS or SSIS scripting. They go on to show how much can be done in TSQL
2007-11-19
4,235 reads
Parallel Processing of Large Volume ETL Jobs
ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.
2007-11-08
7,103 reads
Blogs
From Planning to Practice: Setting Up Your FinOps Framework
By HeyMo0sh
As someone who works in DevOps, I’m always focused on creating systems that are...
“We love to debate minutiae”
By Brian Kelley
I am guilty as charged. The quote was in reference to how people argue...
Advice I Like: Knots
By Steve Jones
Learn how to tie a bowline knot. Practice in the dark. With one hand....
Forums
Restoring On Top II
Comments posted to this topic are about the item Restoring On Top II
SQL Art 2: St Patrick’s Day in SSMS (Shamrock + Pint + Pixel Text)
Comments posted to this topic are about the item SQL Art 2: St Patrick’s...
Breaking Down Your Work
Comments posted to this topic are about the item Breaking Down Your Work
Question of the Day
Restoring On Top II
I have a database, DNRTest, that has a number of tables and other objects in it. The other day, I was trying to mock up a test and ran this code on the same server:
-- run yesterday CREATE DATABASE DNRTest2 GO USE DNRTest2 GO CREATE TABLE NewTable (id INT) GOToday, I realize that I need a copy of DNRTest for another mockup, and I run this:
-- run today USE Master BACKUP DATABASE DNRTest TO DISK = 'dnrtest.bak' GO RESTORE DATABASE DNRTest2 FROM DISK = 'dnrtest.bak' WITH REPLACEWhat happens? See possible answers