Blog Post

ETL Testing

,

There is only one thing guaranteed when writing ETL pipelines and that is that upstream changes WILL occur and WILL break your ETL pipeline.

As Data Engineers we MUST make sure that we test well so that we can identify breaking changes, rapidly, AND fix and deploy just as fast. To do this we need to use automated testing.

To get started here are links some links to a four part blog series I wrote on testing ETL pipelines, the theory is the same whether your pipelines are SSIS packages, Informatica, Python / Airflow or anything you can dream of:

Original post (opens in new tab)

Rate

You rated this post out of 5. Change rating

Share

Share

Rate

You rated this post out of 5. Change rating