Ed Elliott's Sql Developer Blog

Blog Post

How to test ETL Processes in production

This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...

2019-10-02

119 reads

Blog Post

How to test ETL Processes in production

This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...

2019-10-02

Blog Post

How do test the upstream data is good in an etl process (etl testing part 3)?

In the first two parts of this series (https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ and https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/), I talked about how to unit test your business logic and integration test your ETL infrastructure code. Having...

2019-09-30

21 reads

Blog Post

How do test the upstream data is good in an etl process (etl testing part 3)?

In the first two parts of this series (https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ and https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/), I talked about how to unit test your business logic and integration test your ETL infrastructure code. Having...

2019-09-30

Blogs

Spark Connect Dotnet November 2024 Where are we?

By

All Spark Connect Posts Introduction There have been quite a few changes in the last...

A New Word: Ochisia

By

ochisia – n. the fear that the role you once occupied in someone’s life...

Create a Numbers Table in Power Query

By

This is a quick blog post, mainly so I have the code available if...

Read the latest Blogs

Forums

Writer methods in PowerQuery (LOL)

By pietlinden

I'm using PowerQuery in Excel on my desktop. (Yeah, I know you can do...

Need help with GROUP BY...

By iamjoe

Here's my data... I am trying to return total days in status for each...

Need help with GROUP BY...

By iamjoe

Here's my data... I am trying to return total days in status for each...

Visit the forum

Question of the Day

The Density Vector

When building statistics, there is the concept of density that refers to the duplicates in a table. How is this calculated?

See possible answers