April 13, 2009 at 6:10 am
I'm importing data from 4 raw files into their own tables in the database. The files are pretty big. So far one of them has imported 70 million records into the database. This has been running for several days. Each import is using the same database connection. If I added a connection for each import would this speed up the process?
April 13, 2009 at 8:42 am
You can keep the same connection. Try to run your four operations as parallel tasks (do not use workflows to govern their behaviour, or put them disconnected in a container to govern the overall result.)
Try to maximize your network bandwidth (optimize your packetsize property to 32767 in the connection) and use of memory.
There are many ways of optimizing your packages, search online. here are a few resource
http://www.simple-talk.com/sql/sql-server-2005/sql-server-2005-ssis-tuning-the-dataflow-task/
April 13, 2009 at 9:00 am
I've decided to use Bulk Inserts. How do I create the format file for the raw file? I tried setting it to select file then selected the raw file I was gonna use but that gives me an error.
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply