October 10, 2007 at 2:20 pm
hey all,
I've been using the FOR EACH LOOP to do file processing. I have a situation where we have no control over an ftp directory we pull files from. This FTP Directory has all of the files they have ever exported there for us. We can only copy them over, process them, and then archive them locally...but not move them off the FTP Server.
We want to only bring in those file we haven't already processed. Is there a way to compare the contents of the FTP Directory and our local Backup Directory and only bring in those files we haven't processed yet?
I'm stumped on this one. Thanks.
-Pat
October 11, 2007 at 9:43 pm
Consider storing the filenames in a database table. You can then check the file name against this table to see if there are any new files that have not yet been processed.
hth,
Tim
Tim Mitchell, Microsoft Data Platform MVP
Data Warehouse and ETL Consultant
TimMitchell.net | @Tim_Mitchell | Tyleris.com
ETL Best Practices
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply