August 7, 2008 at 6:27 am
I have an import job that import 12 files from an AS400 system (db files, no text files). In total it is about 10 gig worth of data. One particular load of the 12 takes about an hour and a half to load pulling from the AS400. The table the data is loaded into contains 982 columns. There are very few indexes on this table and all data is stored regardless of what month it is (no archiving process and no plans to have one). The table currently has about 45 million records. This is a monthly process soon to be a daily process.
My question is how can I improve load time on this particular piece and others? Has anybody else had issues or possible solutions to loading AS400 data into SQL Server quickly and efficiently?
August 7, 2008 at 6:55 am
have you tried loading the data into a temp table first to just pull the records down from the AS/400 and then move the records from the temp table into your destination table?
One other thought regarding the data you are pulling, ensure that all of the columns you are pulling across are actually being loaded, meaning, deselect any columns you do not need.
August 7, 2008 at 6:58 am
The current process is set up to load into a temp table and then process from the SQL side. All columns are populated. This data is then fed into downstream reports.......6 in total.
Thank you for the quick reply.
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply