Dynamic Flat File Creation

  • Hi is it possible to create mutiple csv files from a dataset where the folder and filenames are derived from the data.

    eg

    Column1 - Column2 - Column3

    AB01 - 03 - ZXY

    AB01 - 04 - RST

    CD01 - 05 - MNO

    Would be sent to 3 files located at

    \\AB01\03\ZXY.csv

    \\AB01\04\RST.csv

    \\CD01\05\MNO.csv

  • Yes, though you may have to script it.

    What will the files contain?

    The absence of evidence is not evidence of absence
    - Martin Rees
    The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
    - Phil Parkin

  • The files will contain a text line for each row based on Column 3. I thought scripting it would be possible.

  • I did not write my question very clearly, my apologies.

    What I meant was, if record 1 in your dataset requires a file to be created called \\AB01\03\ZXY.csv, where does the data for that file come from? Is it part of the same dataset? Is it derived?

    Phil

    The absence of evidence is not evidence of absence
    - Martin Rees
    The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
    - Phil Parkin

  • No problem Phil. It is sourced from the dataset ie it gets the folder structure from columns 1 & 2 and the filename from column 3.

  • OK, we're getting there.

    So, sounds like we need to get your dataset into a data source and feed that data source into a script component.

    The script component will read the rows in the dataset and, for each row, create a text file with the appropriate name and path, with data from the same line of the dataset. If any manipulation of the data is required before it goes out to the file, the script component can do it.

    This method depends on the fact that the data for the text file is on the same line as the file-name info - which is why I was asking the "where's the data coming from" questions.

    Phil

    The absence of evidence is not evidence of absence
    - Martin Rees
    The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
    - Phil Parkin

  • I have the directory structure built by passing the data into a recordset then using a for each container to delete and build the folder as required.

    Within this same script task which is building the folders can I write dynamic SQL to extract the lines I require from the dataset and also write this data to a specific file?

  • Sounds like you want to stream out an OleDbDataReader to a text file. I'm no VB.NET guru, but that is certainly possible. You may have some formatting issues, but in a Script task, you have the power to fix them.

    The absence of evidence is not evidence of absence
    - Martin Rees
    The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
    - Phil Parkin

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply