April 23, 2010 at 4:16 am
HI,
I have a table in Sql this table needs to be "updated daily" from the source data,the source data is a text file ,i need to set up a package in ssis to perform this task and schedule it for the daily process because the source data keeps changing and the changes need to be reflected in the target table too which is in sql .
How best i can implement this and also it is having million of records in the source so i cant just use the update statment in Execute sql task.
Please let me know ...
Thanks in Advance
Jaya
April 23, 2010 at 4:30 am
Read the text file in via a Flat File source and write it to a staging table with an IDENTITY-column.
Use an Execute SQL Task with a T-SQL script that loops over your staging table, taking batches of something like 50.000 records and updating the destination table. That way you will use an update statement of only 50.000 records (or more or less, that's up to you), instead of the whole million.
Before each run, truncate the staging table.
p.s.: your caps-lock is broken
Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply