Remove duplicate row

  • select distinct lastname,fisrstname, titile,cellphoneno , officeno from tableone

    union

    select distinct lastname,fisrstname, titile,cellphoneno , officeno from tabletwo

    then i have a "sort transformation" that order by lastname,fisrstname, titile

    and remove duplicate. after sort transformation i have "lookup" and "drive transformation" and ofcource "OLE DB destination"

    that uses all the coulmn from the source (lastname,fisrstname, titile,cellphoneno , officeno)

    the "sort transformation" is taking forever and degradeding the perfomace of my packages . i was wondering if there is way i can

    remove sort transformation.

  • It has to do the sort to do the union and dedupe. That's part of how it does that.

    - Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
    Property of The Thread

    "Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon

  • hi........ thanks for the reply i have done this by using derived column....but i am unable to merge them and sent to the oledb destination......

    i am taking oledb source from database 1 where i am taking three columns and after that derived column and adding the three columns and sort them......

    created another oledb source for database 2 and taking 1 column from it and sort it ..........

    and i am trying to merge them and sent to the database 2.......

    but i am getting error that i cannot merge them,,,,,,,,,,

    please help me........

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply