Getting the Duplicate rows?

  • Hi Friends,

    i am doing a project in SSIS 2008, SQL 2008R2...

    In the data flow,

    my source(OLEDB) has some duplicate records where i am removing the duplicate rows using SORT component....

    i need to get those duplicate records for some other formation in the run time...How can i get those rows from Sort Component?

    Give me suggestions please .....

    Thanks,
    Charmer

  • Charmer (8/8/2012)


    Hi Friends,

    i am doing a project in SSIS 2008, SQL 2008R2...

    In the data flow,

    my source(OLEDB) has some duplicate records where i am removing the duplicate rows using SORT component....

    i need to get those duplicate records for some other formation in the run time...How can i get those rows from Sort Component?

    Give me suggestions please .....

    You can't, they are removed. (and I wouldn't use the sort component either, it's a blocking operation)

    Another option is to not remove them and just insert everything in the destination table, which hopefully has a primary key. The duplicate rows will get constraint violations, and you can pick them up with error handling.

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

  • Koen Verbeeck (8/9/2012)


    Charmer (8/8/2012)


    Hi Friends,

    i am doing a project in SSIS 2008, SQL 2008R2...

    In the data flow,

    my source(OLEDB) has some duplicate records where i am removing the duplicate rows using SORT component....

    i need to get those duplicate records for some other formation in the run time...How can i get those rows from Sort Component?

    Give me suggestions please .....

    You can't, they are removed. (and I wouldn't use the sort component either, it's a blocking operation)

    Another option is to not remove them and just insert everything in the destination table, which hopefully has a primary key. The duplicate rows will get constraint violations, and you can pick them up with error handling.

    Thank you Koen, I will redirect the error rows from the destination and do some transformation to insert it back....:-)

    Thanks,
    Charmer

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply