ETL Antipattern: Skipping The Documentation
Documentation is an asset that is both loathed and loved. Creating technical and business documentation is often looked upon as a tedious chore, something that really ought to be...
2020-12-20
Tim Mitchell
- Interests: SQL Server, Data Warehousing, ETL, Data Architecture, Python, Dbt
- Website: https://www.TimMitchell.net
- Bluesky: tmitch.net
ETL Antipattern: Load Processes that Don’t Scale
One of the most significant design considerations in ETL process development is the volume of data to be processed. Most ETL processes have time constraints that require them to...
2020-12-19
252 reads
ETL Antipattern: Load Processes that Don’t Scale
One of the most significant design considerations in ETL process development is the volume of data to be processed. Most ETL processes have time constraints that require them to...
2020-12-19
1 reads
ETL Antipattern: Failure to Test and Validate
“If it compiles, it works.” – An unemployed developer Building ETL processes is quite easy. Building ETL processes that deliver accurate results as quickly as possible is substantially more...
2020-12-18
8 reads
ETL Antipattern: Failure to Test and Validate
“If it compiles, it works.” – An unemployed developer Building ETL processes is quite easy. Building ETL processes that deliver accurate results as quickly as possible is substantially more...
2020-12-18
3 reads
ETL Antipattern: Failing to Treat ETL Logic as Source Code
In most data projects, building the extract-transform-load (ETL) logic takes a significant amount of time. Enterprise ETL processes must do several things well: retrieve enough data to satisfy the...
2020-12-17
9 reads
ETL Antipattern: Failing to Treat ETL Logic as Source Code
In most data projects, building the extract-transform-load (ETL) logic takes a significant amount of time. Enterprise ETL processes must do several things well: retrieve enough data to satisfy the...
2020-12-17
3 reads
ETL Antipattern: No Error Handling Logic
I usually avoid talking about technology in absolutes, but here’s one that I can share without reservation: On a long enough timeline, every single ETL process will eventually fail....
2020-12-16
6 reads
ETL Antipattern: Lazy Metadata
If data is a train, then metadata is the track on which it travels. A good metadata definition in ETL processes will help to ensure that the flow of...
2020-12-15
9 reads
ETL Antipattern: Performing Full Loads Instead of Incremental Loads
In my last post in the ETL Antipatterns series, I wrote about the common antipattern of ingesting or loading more data than necessary. This brief post covers one specific...
2020-12-14
33 reads
Blogs
Learning Database Design and Power BI Performance
I spend my days working on and managing SQL Server instances—working days at least,...
Free SQL Training: Dive into Databases This December!
Want to learn SQL and get some real practice this December? Check out the...
Monday Monitor Tips: CIS Compliant
By Steve Jones
A new feature added to Redgate Monitor Enterprise automatically. CIS compliance is something many...
Forums
Testing whether TRY_CONVERT is losing precision
Hello All, I am using TRY_CONVERT to parse character data into a decimal column....
need help with assigning a value to a variable
Hi I am trying to assign the value of the variable using the output...
Merry Christmas 2025
Comments posted to this topic are about the item Merry Christmas 2025
Question of the Day
The Reindeer Count
How many reindeer does Santa have pulling his sleigh? (according to the famous song) Hint, the answers are not base 10.
See possible answers