January 25, 2006 at 4:15 pm
In my very limited experience with DTS and SSIS, it seems very easy to skip the first X lines in a file you're reading data from, but I haven't found a way to skip the last X lines (in this case 1). I have a data file that has a header and a footer. I want to ignore the footer, or even better, pull data out of it in a separate SSIS package, pull that data out into a different table.
Any thoughts are most appreciated.
Rick Todd
January 26, 2006 at 2:21 am
Rick,
On the surface it would seem that you can't do this because the Flat File Adapter expects files with known and consistent metadata. In this context your files could be considered unstructured due to their inconsistent metadata throughout the file (I know they're NOT unstructured but bear with me for a minute OK).
Donald Farmer's book (available from the usual places) has a method for building adapters for unstructured files which uses the script component.
Alternatively you could treat the file as having just one column of data and then, in the pipeline, treat it accordingly. I'm assuming you can identify the rows which are in the footer just by their very nature and in this case you can use the Conditional Split transform to hive them off into a seperate data-flow and then treat the rest of the rows accordingly.
If you cannot identify the rows by what they look like then you'll have to do what you say - strip off the last X rows. This will be possible through the use of ROWCOUNT component although you may have use a couple of data-flows in order to do it - which isn't a problem.
This is a data cleansing issue more than anything and that is something that SSIS does rather well.
HTH.
-Jamie
Jamie Thomson
http://sqlblog.com/blogs/jamie_thomson
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply