i want load duplicate records in seperate table

Question

i want load duplicate records in seperate table

muralikrishna37

Ten Centuries

Points: 1233
More actions
September 12, 2008 at 6:39 am

#123559

Hi SSIS gurus,
i have duplicate records in my table i want elimate this records for this i use aggregate transformation but my client ask load duplicate records in other destination
this mean i want load orginal records in one table
and duplicate records in another table
how it is tell me its urget
thanks
murali

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply

Paul M. Corley Old Hand Points: 365 More actions · Answer 1

Prior to your aggregate task that you use to remove the duplicate records you have a multicast that would output to your agg task that removes the dupe and a second agg task that would group and count the records you could then conditional split these records where count > 1 and output them to a dupelicate record table.

muralikrishna37 Ten Centuries Points: 1233 More actions · Answer 2

Hi paul,

sorry, i cant understand what u said

plz tell me detail

thanks

murali

Chinna SSCertifiable Points: 5408 More actions · Answer 3

Murali,

could u explain me what exactly u are doing as i haven't understood what u are trying to do ...

if at all u want to eliminates the dup u can have the sql command written for that ..

send me the exact issue so that we can solve it together...because using conditional split we can make them move to other folder based upon count>1...so let me know the exact situation...

Thanks,
Chinna

Its the Journey which gives you Happiness not the Destination-- Dan Millman

muralikrishna37 Ten Centuries Points: 1233 More actions · Answer 4

Hi Chinna,

i have flatfile table it contain duplicate records.my requriement is elimante duplicate records at the same time load duplicate records in another table.so based on my requriement i have to load duplicate records one table and without duplicate records another table.

thanks

murali

Chinna SSCertifiable Points: 5408 More actions · Answer 5

Murali ...

did u get it solved ...if not i believe this should work as paul said ...

from the source send it to multicast

Source-----multicast ....it sends the same o/p to two diff agg task ...

then from agg task in one of them u will just remove the duplicates...

in second agg u will have a unique key by which u can say that u have duplicates ...for example empid ..

then group by Empid ....

so now u have got the tables without duplicates but what all u need is now all the duplicates in another table....

so connect that second agg task to conditional split in that u vl have condition

in condition write having count(empid)>1 and connect it to one table else connect it to some other source file jus to double check ....by which u vl get all th edup in other table ...

.........If at all this is not an automated process u can import all the source into a table and remove the duplicates and move it to other table manually.......

let me know if this wrks out...

Thanks,
Chinna

Its the Journey which gives you Happiness not the Destination-- Dan Millman