FTP Connection Manager - Chunk Size

  • I hope someone can explain this because I can't seem to find a good explaination.

    What exactly is Chunk Size in the FTP Connection Manager (used when using an FTP Task)?

    I thought it meant it would break up what ever file I had into packets of a certain size and FTP them. Apparently I'm wrong. It looks like it requires a setting larger than the size of the file being FTP'd.

    When I had it set to the default, my FTP Task was failing with a 'Unable to send files using "MyFTPConnection" ' error. When I increased it to 100kb, I could send one file and then when I increased it to 300kb I was able to send both files.

    This worries me as my files are going to keep growing and growing over time and I don't know how large they could potentially get, which means at some point my FTP task will simply fail because the Chunk Size is set wrong.

    Does anyone know a work around to this potential problem?

    Thanks,

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • Brandie,

    I ran into similar problem when using the FTP Task as well.I could not find max size that was allowed for the transfer .I ended up using a Script Task with help from the following site .

    http://www.windowsdevcenter.com/pub/a/windows/2006/12/12/building-ftp-services-using-net-20.html.

  • My boss just found something on the Max Chunk Size in TechNet.

    Reading or writing in small chunks causes a less optimized round trip to the server because a full packet is not received or sent. The default value is 1 KB. The maximum value is 1000 KB (1 MB).

    Which means I'd better script out all my FTP tasks because I'm pretty sure my files will get larger than that.

    Thanks for the link. I'll take a look at it.

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • Okay, I haven't tested this yet, but something about it just doesn't seem right. I've take the Upload code based on the website listed above and I have a Server IP & a Directory to put my files in. The problem is, the website refers to the loopback IP address and not to a specific folder (other than the author says what his folder is).

    Any thoughts on how to properly redirect my task to the FTP Site & folder? Script Task read only variables are FTPServerIP (127.0.0.1:21), FTPFolder (/MyFTPFolder), PremFileSource (\\Server\Path\Filename), and PremControlFileSource (\\Server\Path\Filename2).

    Try

    Dim ftpURI As String = Dts.Variables("FTPServerIP").Value.ToString

    & Dts.Variables("FTPFolder").Value.ToString

    Dim PremFileSourceName As String =

    Dts.Variables("PremFileSource").Value.ToString

    Dim PremControlFileSourceName As String =

    Dts.Variables("PremControlFileSource").Value.ToString

    Dim Premfiledest As String = ftpURI & "FileName"

    Dim PremControlfiledest As String = ftpURI & "CONTROLFileName"

    Dim client As New WebClient

    client.Credentials = New NetworkCredential("MyLogin", "MyPwd")

    client.UploadFile(Premfiledest, PremFileSourceName)

    client.UploadFile(PremControlfiledest, PremControlFileSourceName)

    Catch ex As Exception

    MsgBox(ex.ToString)

    End Try

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • This is the Upload Function that I am using to upload any Binary Files [.zip/.txt/ and etc...]

    Imports System

    Imports System.IO

    Imports System.Net

    Imports System.Text

    Public Shared Sub FTPUpload(ByVal filePath As String, ByVal fileName As String)

    Try

    'filepath = "C:\temp\1.txt"

    'filename = "1.txt"

    Dim client As New WebClient

    client.Credentials = New NetworkCredential("DomainName/userlogin", "Password")

    client.UploadFile("ftp://xxx-xxx-xx/Input/" & fileName, filePath)

    client.Dispose()

    Catch ex As Exception

    Console.WriteLine(ex.Message)

    Console.ReadLine()

    End Try

    End Sub

  • Are you using IP & Port # for your FTP or just IP address?

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • Im using the ftp address [no ip or port specified]

    Example ftp3-uk.sqlcentral.com.

    Not sure if it works with ip address.Give it a try and let me know.

  • I did, but my other office (different state from me) says they can't find the files.

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • I checked the site I sent you earlier,and the examples are using ip address As Const ftpURI As String = "ftp://127.0.0.1/"

    http://www.windowsdevcenter.com/pub/a/windows/2006/12/12/building-ftp-services-using-net-20.html

    Can you connet to the ftp site using your IE or a third party tool ?

    Does your username has the correct permissions?

  • Looking into this a little more, I finally opened up my job log and found the following error:

    Files NOT Uploaded. Reason: System.Net.WebException: The remote server returned an error: (501) Syntax error in parameters or arguments.

    at System.Net.WebClient.UploadFile(Uri address, String method, String fileName)

    at System.Net.WebClient.UploadFile(String address, String fileName)

    at ScriptTask_703fee79f9bf443eb4b23bf7114429c5.ScriptMain.Main() 12/12/2008 9:58:45 AM

    Aha! The plot thickens! The files never actually sent. The step said it succeeded because it wrote to the log, when it didn't really succeed. So now I have to put a DTSFailure into that step if it hits my Catch block.

    I've got one of my developers who knows more .Net than I do helping me out. More info as we find it...

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • wow, anybody doesnt try to put ZERO in Chunk Size???:hehe:

    :cool:limitless download/upload size

Viewing 11 posts - 1 through 10 (of 10 total)

You must be logged in to reply to this topic. Login to reply