June 19, 2017 at 8:53 am
| Hello, I have a requirement to merge two large tables with redundant data. Table A and Table B both have more than a billion records, most of the data including columns are redundant except 5 additional columns in Table B. I would like to merge them physically and delete table B by comparing the data between both table and load the data that doesn't exist in Table A but exist in Table B to Table A. Ultimate goal is to get rid of Table B and load all data and columns to Table A and then partition table A. Did anyone had experience doing this ? how can i compare the data between two large tables ? can we use informatica or any native sql tools ? is it a goof approach to merge the tables and partition Table A or first partition Table A and them merge the tables ? Please let me know if you need any details. Thanks in advance. |
June 19, 2017 at 2:16 pm
Robin35 - Monday, June 19, 2017 8:53 AM
Hello, I have a requirement to merge two large tables with redundant data.
Table A and Table B both have more than a billion records, most of the data including columns are redundant except 5 additional columns in Table B.
I would like to merge them physically and delete table B by comparing the data between both table and load the data that doesn't exist in Table A but exist in Table B to Table A.
Ultimate goal is to get rid of Table B and load all data and columns to Table A and then partition table A.
Did anyone had experience doing this ? how can i compare the data between two large tables ? can we use informatica or any native sql tools ? is it a goof approach to merge the tables and partition Table A or first partition Table A and them merge the tables ?
Please let me know if you need any details.
Thanks in advance.
Given the complexity of your needs I feel this one goes beyond the realm of a for-free forum request. Unless you already have indexing in play that will allow for fast elimination of records not germaine to the comparison you best have some very substantial hardware or this project could take quite a long time.
Should you wish to engage someone for professional assistance with this project there are a number of individuals here on SSC.com that could help out.
Best,
Kevin G. Boles
SQL Server Consultant
SQL MVP 2007-2012
TheSQLGuru on googles mail service
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply