April 30, 2008 at 5:19 am
I'm looking for ideas on a problem. We have many files sent to use from our customers that are your garden variety mainframe-generated data files. Typically that means
first row is a "header" row usually with only a single value in it (like a timestamp of when the file was generated)
second-thru-n rows are "data" rows. However each row is not the same. Usually, the first value is some sort of row or data type value. The subsequent delimited fields in that row will vary depending on the type.
last row is a "trailer" row usually with a record count and/or other gibberish.
Our typical approach in the data flow task is to set up the flat file data source as a single column file, then perform a conditional split on it. So far so good.
The question I have is how to "join up" some of the data from the split. For example, we want to use that value we find in the header row in our destination source along with the data rows. But it does not appear that you can have multiple pipelines end up in the same destination. At least, that's what I'm fishing for... ideas, suggestions, etc.
Thanks.
April 30, 2008 at 6:14 am
You could re-join these with a MERGE JOIN component.
I think you are getting to the point in which you should be considering using the script component or creating a custom data source component.
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply