February 8, 2011 at 8:08 am
I have a text file that contains two types of rows. The first row is the Batch Header which needs to be placed in one table, while the rest of the records (containing a different number of fields to the first row) are detail records that need to be placed in another table.
What would be the most efficient way of dealing with this, given that on the first pass I only want to read the first record in the file?
Regards
Steve
February 8, 2011 at 2:05 pm
You could certainly deal with both of them in the same data-flow. I see only one particular issue, you could have a point where the detail exists without a header row. There are some creative ways to work around this.. I think it might be best to only make one pass on the file.
CEWII
February 9, 2011 at 2:38 am
Yes - I'm going to make one pass using a Script Component and send it to two destinations. The files aren't very large so there is no issue with processing times.
This was a good example which I am basing my logic on:
http://agilebi.com/jwelch/2007/07/12/processing-a-flat-file-with-header-and-detail-rows/
Thanks
Steve
February 9, 2011 at 11:04 am
If you can easily tell from the data a script should not be required, you can use the conditional split component, which would likely be faster.
CEWII
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply