SSIS 101: Isolating Data in a Data Flow

Question

SSIS 101: Isolating Data in a Data Flow

Andy Leonard

SSChampion

Points: 10144
More actions
February 11, 2009 at 12:16 am

#201064

Comments posted to this topic are about the item SSIS 101: Isolating Data in a Data Flow
Andy Leonard, Chief Data Engineer, Enterprise Data & Analytics

Viewing 10 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic. Login to reply

Manie Verster SSCertifiable Points: 7152 More actions · Answer 1

Hi Andy,

Nice article but afraid I lost you along the way. You know, where you started with the Union All's and the rest, I still have the red error mark on my ContactStage but don't see where you handled that part. Excuse me if I am silly or stupid but can you help me out here please. I am busy training myself in SSIS and your aticle came on the right time.:hehe::hehe:

:-PManie Verster
Developer
Johannesburg
South Africa

I can do all things through Christ who strengthens me. - Holy Bible
I am a man of fixed and unbending principles, the first of which is to be flexible at all times. - Everett Mckinley Dirkson (Well, I am trying. - Manie Verster)

Jack Corbett SSC Guru Points: 184394 More actions · Answer 2

Nice article Andy. I've never used the multicast task and didn't have an idea why I'd use it, but this is a good way use for it in my situation. I've always edited my source query with a where clause to check out problematic or specific rows.

This is another tool in the toolbox. Thanks.

Jack Corbett
Consultant - Straight Path Solutions
Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question

KEVIN.L.STEVENS SSC Rookie Points: 44 More actions · Answer 3

Here is where Andy tells how to correct the target information:

This scenario occurs with enough frequency to justify the Restore Invalid Column References Editor, shown above. The ContactID column with Lineage ID 32 is shown as invalid, but the References Editor located (and suggests) a column with the same name and data type: the ContactID column with Lineage ID 333. In fact, it’s located new columns for all the invalid references (although the error did not report these additional columns).

Click OK to replace the invalid references with the suggested columns

You have several options in the Restore Invalid Column References Editor. One of them is to let the editor try to restore the invalid links but I usually highlight all of the rows listed and click on the delete invalid rows option and hit "Apply" (there is no OK button). This will close the editor and remove the invalid references. When you reopen the target again everything should fix itself and you are on your way.

DCPeterson SSCoach Points: 15266 More actions · Answer 4

Nice article Andy, I never really understood what the data viewers were. I sure wish I'd had this information a year ago...

/*****************

If most people are not willing to see the difficulty, this is mainly because, consciously or unconsciously, they assume that it will be they who will settle these questions for the others, and because they are convinced of their own capacity to do this. -Friedrich August von Hayek

*****************/

StarNamer SSCrazy Eights Points: 8746 More actions · Answer 5

Great article. I too am having to do more and more work with SSIS and, naturally, don't have time/budget to go on training 🙁 hence learning about the tools described here is very helpful.

Thanks.

Derek

bc_ Hall of Fame Points: 3168 More actions · Answer 6

I like the Data Probe idea. That will come in handy for sure.

Thanks,

Burke

[font="Arial Narrow"]bc[/font]

mcloney SSC Eights! Points: 885 More actions · Answer 7

Hi Andy,

Great intro article on SSIS! I've been working with it for over two years, but had never tried anything similar to your "data probe" technique. It makes perfect sense.

One thing I saw in this article that's a little different is the Multicast as sort of a dummy endpoint. I've often seen a Rowcount as a dummy endpoint but never a multicast. Does the endpoint actually do anything in either case in this type of scenario?

Cheers,

-Matthew

Jack Corbett SSC Guru Points: 184394 More actions · Answer 8

mcloney (2/11/2009)
Hi Andy,
Great intro article on SSIS! I've been working with it for over two years, but had never tried anything similar to your "data probe" technique. It makes perfect sense.
One thing I saw in this article that's a little different is the Multicast as sort of a dummy endpoint. I've often seen a Rowcount as a dummy endpoint but never a multicast. Does the endpoint actually do anything in either case in this type of scenario?
Cheers,
-Matthew

I actually use the konesans trash destination when I need a dummy end point.

Jack Corbett
Consultant - Straight Path Solutions
Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question

Andy Leonard SSChampion Points: 10144 More actions · Answer 9

Thanks for all the comments!

Matthew - Like Jack, I usually use the Konesans Trash Destination for an endpoint as well. I didn't mention it here. I also use the Script Component and the Derived Column transformations as dummy endpoints for data flow paths - basically anything that will succeed without configuration will do.

:{> Andy

Andy Leonard, Chief Data Engineer, Enterprise Data & Analytics