using SSIS to perform data validation in ETL app

Question

using SSIS to perform data validation in ETL app

Viewing 5 posts - 16 through 19 (of 19 total)

You must be logged in to reply to this topic. Login to reply

Phil Parkin SSC Guru Points: 246959 More actions · Answer 1

jreece (6/22/2009)
John,
--
I am now going to add a few derived columns - I thought "Create_Date" and "Change_Date" might be very useful for tracking updates.

You might also consider doing these at the database level. Your CreateDate field just needs a DefaultValue of GetDate() and ChangeDate needs an UPDATE trigger to keep it maintained ...

The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin

John Rowan SSC Guru Points: 56440 More actions · Answer 2

Phil Parkin (6/22/2009)
jreece (6/22/2009)
John,
--
I am now going to add a few derived columns - I thought "Create_Date" and "Change_Date" might be very useful for tracking updates.
You might also consider doing these at the database level. Your CreateDate field just needs a DefaultValue of GetDate() and ChangeDate needs an UPDATE trigger to keep it maintained ...

I think that depends on what you are after. I routinely use column defaults on CreateDate columns in OLTP systems. For OLAP or data warehouse DBs, I typically use a LoadDate column when loading data and I assign the LoadDate value from a variable in the SSIS package that gets its value at SSIS startup time so that everything that was loaded as part of the SSIS package run has the same LoadDate.

John Rowan

======================================================
======================================================
Forum Etiquette: How to post data/code on a forum to get the best help[/url] - by Jeff Moden

jreece Ten Centuries Points: 1138 More actions · Answer 3

That's exactly what I want to do, tag all the records loaded with the same date stamp so I can identify them by batch. Thanks again - I can see it will take me a long time to get up to speed with this tool. I don't even know what I don't know. (sorry to get all Yogi Berra on you)

Deepmala Newbie Points: 1 More actions · Answer 4

?Well I have always used Daton's SQL connector for my Warehousing. You can add any Data Source here in a single place

Phil Parkin SSC Guru Points: 246959 More actions · Answer 5

Deepmala wrote:

?Well I have always used Daton's SQL connector for my Warehousing. You can add any Data Source here in a single place

Unless you are going to justify this comment by describing exactly how this product performs data validation, I am going to have to mark your comment as spam.

The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin