Post reply

File Watcher Task - Multiple File Masks

  • January 22, 2009 at 10:40 am

    #217685

    Hello,

    I am given a task to convert the existing DTS packages to SSIS. I need some help in dealing with a situation which is a mandatory. This is the setup I have

    1. We have an FTP location where .txt files are dropped with different naming conventions like datetime_images.txt, datetime_site.txt, datetimeProd.txt, datetime_jl.txt, datetime_ab.txt etc.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    4. For the next package which looks for remaining files other than the above (point 3), I was not able to successfully implement.

    5. I'm using the File Watcher Task from Konesan's. In the editor I've mentioned

    Filter : *_*.txt

    Find Existing Files : True

    Ouput Variable Name : User::MyFile

    Path Input Type : Variable

    Path Variable : User::SourceDir

    The remaining options are not changed in the file watcher task editor.

    This issue I'm facing

    As the FTP location for all files is the same, this filewatcher task is picking up the other set of files as mentioned in point 3 and I know it will b'coz I've mentioned the filter as *_*.txt.

    My question to all experts

    1. Is there any way that I can make my logic to look only for the specific files?

    Thanks in advance.


    Lucky

  • January 22, 2009 at 1:25 pm

    #931428

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

  • January 22, 2009 at 1:36 pm

    #931440

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.


    Lucky

  • January 23, 2009 at 12:20 am

    #931658

    lucky (1/22/2009)

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.

    How about this: datetime_??.txt

    ---
    SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

  • January 23, 2009 at 7:19 am

    #931776

    CozyRoc (1/23/2009)

    lucky (1/22/2009)

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.

    How about this: datetime_??.txt

    Thanks Cozyroc..

    In fact I happened to visit your site through a google search.

    However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?


    Lucky

  • January 23, 2009 at 9:12 am

    #931862

    lucky (1/23/2009)

    CozyRoc (1/23/2009)

    lucky (1/22/2009)

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.

    How about this: datetime_??.txt

    Thanks Cozyroc..

    In fact I happened to visit your site through a google search.

    However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

    Lucky,

    The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

    ---
    SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

  • January 23, 2009 at 11:05 am

    #931961

    CozyRoc (1/23/2009)

    lucky (1/23/2009)

    CozyRoc (1/23/2009)

    lucky (1/22/2009)

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.

    How about this: datetime_??.txt

    Thanks Cozyroc..

    In fact I happened to visit your site through a google search.

    However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

    Lucky,

    The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

    Thanks CozyRoc.. that was an absolute answer for my problem.

    Thanks so much...


    Lucky

  • January 25, 2009 at 10:53 am

    #932406

    lucky (1/23/2009)

    CozyRoc (1/23/2009)

    lucky (1/23/2009)

    CozyRoc (1/23/2009)

    lucky (1/22/2009)

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.

    How about this: datetime_??.txt

    Thanks Cozyroc..

    In fact I happened to visit your site through a google search.

    However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

    Lucky,

    The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

    Thanks CozyRoc.. that was an absolute answer for my problem.

    Thanks so much...

    Hello CozyRoc,

    I need your expertize once again...

    I could resolve the issue above. But now when I drop all the different pattern files at the same time it is working for the following

    1. datetimeProd.txt

    2. datetime_jl.txt, datetime_av.txt

    but not working for the following

    3. datetime_images.txt

    4. datetime_site_images.txt

    I'm using the filter as *_images.txt but the package is picking up the other one (datetime_site_images.txt). I tried different patterns like *_??????.txt to only pick the one but it is not working.

    Thanks in advance.


    Lucky

  • January 25, 2009 at 11:23 am

    #932414

    lucky (1/23/2009)

    CozyRoc (1/23/2009)

    lucky (1/23/2009)

    CozyRoc (1/23/2009)

    lucky (1/22/2009)

    Ray M (1/22/2009)

    Thats fine that they are using the same location. But makes them unique?

    How do you know what site file belongs to?

    Thanks Ray for replying back.

    As mentioned in my post (point 2 & 3) makes them unique.

    2. Currently there are 4 DTS packages -- each for images (datetime_images.txt), site (datetime_site.txt), prod (datetimeProd.txt) and copy (datetime_jl.txt, datetime_ab.txt) which looks for files mentioned in brackets.

    3. For the first 3 it was very easy to filter explicitly for the specific .txt (*_images.txt, *_site.txt, *Prod.txt) in the filewatcher task and process them.

    So my requirement is not to consider these files (*_images.txt, *_site.txt, *Prod.txt) when I'm looking exclusively for these copy (datetime_jl.txt, datetime_ab.txt)

    Hope I'm cleare this time.

    Thanks in advance.

    How about this: datetime_??.txt

    Thanks Cozyroc..

    In fact I happened to visit your site through a google search.

    However the point here is that the datetime is not static as it keeps on changing whenever the file is FTPed. So in this case how should I use the filter condition in my filewatcher task to only pick other than (*_images.txt, *_site.txt, *Prod.txt)?

    Lucky,

    The question mark (?) instructs the file processor the specific letter is variable. Can you please give more examples because based on your current example it will successfully detect each and every file having this structure: datetime_[..].txt

    Thanks CozyRoc.. that was an absolute answer for my problem.

    Thanks so much...

    Lucky,

    *_images.txt is not going to work because it means "grab everything which ends with _images.txt" . And datetime_site_images.txt ends just like that. What you have to do is handle explicitly [datetime]_images.txt . So for example if your files are like this:

    01-03-05_images.txt

    you have to handle it like this: ??-??-??_images.txt

    This wildcard will ignore datetime_site_images.txt files.

    ---
    SSIS Tasks Components Scripts Services | http://www.cozyroc.com/

Viewing 9 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic. Login to reply