For each loop container not working as expected (only reads the first file)

Question

Post reply

For each loop container not working as expected (only reads the first file)

anavill

Valued Member

Points: 52
More actions
August 24, 2010 at 11:33 am

#140329

Hello,
I Configured a For each loop container in order to read each of the files from a FOLDER and then process them using Data Flow/Script Component but is only reading ONE file from the folder.
I used File Enumerator, and defined a USER:FIlename variable that is passed to the Script Component, them processed and saved to an Destination file.
Please advise!!

Viewing 15 posts - 1 through 15 (of 33 total)

You must be logged in to reply to this topic. Login to reply

Rick.Cornell.SME SSCrazy Eights Points: 8597 More actions · Answer 1

Rick.Cornell.SME

SSCrazy Eights

Points: 8597

August 24, 2010 at 12:01 pm

#1211198

Any error messages?

jdgonzalez@jdandrachel.com Hall of Fame Points: 3161 More actions · Answer 2

Is it looping through all of the files and only processing one file or does it only loop through one file then exist? I always make the mistake of not setting my connection to the variable and keeping it on the test file I'm working on.

anavill Valued Member Points: 52 More actions · Answer 3

Thanks for your replys

No error messages.

It only loops through one file and for testing pursposes I have just two files in the folder.

What do you mean about not setting your connection to the variable?

Here is a brief description of my package.

1.For each loop (DataFlow inside).

2. Dataflow steps:

a)Script Component (script in VB where the variable is passed)

b)Aggregate

c) Derived Column

d) Flat file destination (set to OVERWRITE existing file, so at the very final I should have all data from the files posted in this file)

3. custom control flow item

4.custom control flow item.

jdgonzalez@jdandrachel.com Hall of Fame Points: 3161 More actions · Answer 4

When you check overwrite data, it will overwrite any data in the file. So you are correct in that you will have one file but it will only be for the last file processed.

Alvin Ramard SSC-Forever Points: 41190 More actions · Answer 5

First, double check that both files match the wildcard that you entered in the "Files:" box.

Then you might try to add a VB (or C#) Script Task a the first task to execute in the For Each container. Pass the file name to this Script Task and have the Script Task display the file name using a MsgBox. Doing so will help verify if it is looping thru each file. If the MsgBox displays each file then you know it's looping correctly so the error is likely that you are not passing the file name correctly to the other tasks.

Alvin Ramard
Memphis PASS Chapter[/url]

All my SSC forum answers come with a money back guarantee. If you didn't like the answer then I'll gladly refund what you paid for it.

For best practices on asking questions, please read the following article: Forum Etiquette: How to post data/code on a forum to get the best help[/url]

jdgonzalez@jdandrachel.com Hall of Fame Points: 3161 More actions · Answer 6

Alvin Ramard (8/24/2010)
Then you might try to add a VB (or C#) Script Task a the first task to execute in the For Each container. Pass the file name to this Script Task and have the Script Task display the file name using a MsgBox. Doing so will help verify if it is looping thru each file. If the MsgBox displays each file then you know it's looping correctly so the error is likely that you are not passing the file name correctly to the other tasks.

Good to know I'm not the only to use that technique;)

anavill Valued Member Points: 52 More actions · Answer 7

Sorry it was a typo , I wanted to say Overwrite option NOT checked", so it is APPENDÏNG data to the file.

I already tried adding in the VB code msgbox(Name), and then when excecuted it only shows one msgbox displaying the name of one of the files. Just once.

Ana

--------------------------------------------------------------------------------

Alvin Ramard SSC-Forever Points: 41190 More actions · Answer 8

J.D. Gonzalez (8/24/2010)
Alvin Ramard (8/24/2010)
Then you might try to add a VB (or C#) Script Task a the first task to execute in the For Each container. Pass the file name to this Script Task and have the Script Task display the file name using a MsgBox. Doing so will help verify if it is looping thru each file. If the MsgBox displays each file then you know it's looping correctly so the error is likely that you are not passing the file name correctly to the other tasks.
Good to know I'm not the only to use that technique;)

I wish I had a dollar for each time I've done that, or better yet, a dollar for each time a msgbox popped up.

It's a simple process that let's you take a peek at what's going on.

Alvin Ramard
Memphis PASS Chapter[/url]

All my SSC forum answers come with a money back guarantee. If you didn't like the answer then I'll gladly refund what you paid for it.

For best practices on asking questions, please read the following article: Forum Etiquette: How to post data/code on a forum to get the best help[/url]

Alvin Ramard SSC-Forever Points: 41190 More actions · Answer 9

anavill (8/24/2010)
Sorry it was a typo , I wanted to say Overwrite option NOT checked", so it is APPENDÏNG data to the file.
I already tried adding in the VB code msgbox(Name), and then when excecuted it only shows one msgbox displaying the name of one of the files. Just once.
Ana
--------------------------------------------------------------------------------

Check what's in the "Files:" box.

Alvin Ramard
Memphis PASS Chapter[/url]

All my SSC forum answers come with a money back guarantee. If you didn't like the answer then I'll gladly refund what you paid for it.

For best practices on asking questions, please read the following article: Forum Etiquette: How to post data/code on a forum to get the best help[/url]

anavill Valued Member Points: 52 More actions · Answer 10

IT shows the name of the file. and the window is pop up just one time, so it is only looping once!

What else do you recommend me to check?

I already got rid of the rest of the control flow boxes, Now I only have the fore each and the data flow toolbox inside.

If I run the package finish succesfully without errors and I get the data of only ONE file.

anavill Valued Member Points: 52 More actions · Answer 11

When executing the package , a window is pop up with the name of the file, but just once, then continues and finish.

I already got rid of the rest of the control tool boxes and now i am only testing with the FOR AEACH and the Dataflow inside.

If I run the package it finish sucessfully without error. It just adds the data from one of the files.

What do you guys recommend me to check? what could be wrong?

Alvin Ramard SSC-Forever Points: 41190 More actions · Answer 12

I think I see your problem. It looks like you do not have a proper "search string" for your ForEach container.

Open up the ForEach Editor and look at the Collection properties. In the box labelled Files, you need to enter the search string that will be use to select the files. If you want to look process every file you would enter *.*

For all .txt file, enter *.txt

For all .dat files that start with "Input", enter Input*.dat

.... and so on .....

Alvin Ramard
Memphis PASS Chapter[/url]

All my SSC forum answers come with a money back guarantee. If you didn't like the answer then I'll gladly refund what you paid for it.

For best practices on asking questions, please read the following article: Forum Etiquette: How to post data/code on a forum to get the best help[/url]

anavill Valued Member Points: 52 More actions · Answer 13

Hi

for that option I have : *.txt

I tried with *.* and same result.

Thanks

ana

Alvin Ramard SSC-Forever Points: 41190 More actions · Answer 14

I'm afraid that there's not much more I can do without seeing the what the package/loop looks like.

Alvin Ramard
Memphis PASS Chapter[/url]

All my SSC forum answers come with a money back guarantee. If you didn't like the answer then I'll gladly refund what you paid for it.

For best practices on asking questions, please read the following article: Forum Etiquette: How to post data/code on a forum to get the best help[/url]