i want load duplicate records in seperate table

  • Hi SSIS gurus,

    i have duplicate records in my table i want elimate this records for this i use aggregate transformation but my client ask load duplicate records in other destination

    this mean i want load orginal records in one table

    and duplicate records in another table

    how it is tell me its urget

    thanks

    murali

  • Prior to your aggregate task that you use to remove the duplicate records you have a multicast that would output to your agg task that removes the dupe and a second agg task that would group and count the records you could then conditional split these records where count > 1 and output them to a dupelicate record table.

  • Hi paul,

    sorry, i cant understand what u said

    plz tell me detail

    thanks

    murali

  • Murali,

    could u explain me what exactly u are doing as i haven't understood what u are trying to do ...

    if at all u want to eliminates the dup u can have the sql command written for that ..

    send me the exact issue so that we can solve it together...because using conditional split we can make them move to other folder based upon count>1...so let me know the exact situation...

    Thanks,
    Chinna

    Its the Journey which gives you Happiness not the Destination-- Dan Millman

  • Hi Chinna,

    i have flatfile table it contain duplicate records.my requriement is elimante duplicate records at the same time load duplicate records in another table.so based on my requriement i have to load duplicate records one table and without duplicate records another table.

    thanks

    murali

  • Murali ...

    did u get it solved ...if not i believe this should work as paul said ...

    from the source send it to multicast

    Source-----multicast ....it sends the same o/p to two diff agg task ...

    then from agg task in one of them u will just remove the duplicates...

    in second agg u will have a unique key by which u can say that u have duplicates ...for example empid ..

    then group by Empid ....

    so now u have got the tables without duplicates but what all u need is now all the duplicates in another table....

    so connect that second agg task to conditional split in that u vl have condition

    in condition write having count(empid)>1 and connect it to one table else connect it to some other source file jus to double check ....by which u vl get all th edup in other table ...

    .........If at all this is not an automated process u can import all the source into a table and remove the duplicates and move it to other table manually.......

    let me know if this wrks out...

    Thanks,
    Chinna

    Its the Journey which gives you Happiness not the Destination-- Dan Millman

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply