FTP Files When the File Dates Do Not Match
-
I cannot find a way in SSIS to examine the file properties before I download them. I am looking to compare the files I have versus the files available for download and only download/overwrite when there are newer files. I was looking at the GetListing method of the FTPClientConnection, but that doesn't seem to provide file properties.
Would someone please point me in the right direction? I have been searching for hours with no good results.
-
Well, I found a way to use request/response in a vb script to check the lastmodified property of the response.
I am not sure if this is the best way to do it, but that's what I am going with until I hear better. I'll post the code when I get done.
-
Neil Kessler (4/7/2009)
Well, I found a way to use request/response in a vb script to check the lastmodified property of the response.I am not sure if this is the best way to do it, but that's what I am going with until I hear better. I'll post the code when I get done.
Check the third-party commercial SFTP Task. It does support regular FTP Servers and also have API for retrieving remote file information (including modified date/time). Let us know if you have questions.
---
SSIS Tasks Components Scripts Services | http://www.cozyroc.com/
-
Thank you for the advice. I went the home grown route, the jist of the code is below. A pulled together code from a couple different places, so this is not my creation:
Dim request As FtpWebRequest = CType(WebRequest.Create(SourceUri), FtpWebRequest)
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails
Dim response As FtpWebResponse = CType(request.GetResponse(), FtpWebResponse)
Dim size As Long = response.ContentLength
Using datastream As Stream = response.GetResponseStream
Using sr As New StreamReader(datastream)
sResult = sr.ReadToEnd()
sr.Close()
End Using
datastream.Close()
End Using
response.Close()
sResult = sResult.Replace(vbCrLf, vbCr).TrimEnd(Chr(13))
sFiles = sResult.Split(CChar(vbCr))
For Each sLine In sFiles
m = GetMatchingRegex(sLine) 'See code below
bCopyFile = True
If m Is Nothing Then
'do failed stuff
Else
Try
dLastModified = Date.Parse(m.Groups("timestamp").Value)
Catch
End Try
Try
dLocalFileDate = FileSystem.FileDateTime(sProcessedPath + "Processed_" + m.Groups("name").Value)
If (dLastModified #12:00:00 AM#) Then
bCopyFile = False
End If
Catch
End Try
Try
If bCopyFile Then
FTPFile
End If
Catch
End Try
End If
Next
' The stuff below is a blantant grab from http://www.codeproject.com
Private Shared _ParseFormats As String() = { _
"(?[\-d])(?([\-r][\-w][\-xs]){3})\s+\d+\s+\w+\s+\w+\s+(?\d+)\s+(?\w+\s+\d+\s+\d{4})\s+(?.+)", _
"(?[\-d])(?([\-r][\-w][\-xs]){3})\s+\d+\s+\d+\s+(?\d+)\s+(?\w+\s+\d+\s+\d{4})\s+(?.+)", _
"(?[\-d])(?([\-r][\-w][\-xs]){3})\s+\d+\s+\d+\s+(?\d+)\s+(?\w+\s+\d+\s+\d{1,2}:\d{2})\s+(?.+)", _
"(?[\-d])(?([\-r][\-w][\-xs]){3})\s+\d+\s+\w+\s+\w+\s+(?\d+)\s+(?\w+\s+\d+\s+\d{1,2}:\d{2})\s+(?.+)", _
"(?[\-d])(?([\-r][\-w][\-xs]){3})(\s+)(?(\d+))(\s+)(?(\w+\s\w+))(\s+)(?(\d+))\s+(?\w+\s+\d+\s+\d{2}:\d{2})\s+(?.+)", _
"(?\d{2}\-\d{2}\-\d{2}\s+\d{2}:\d{2}[Aa|Pp][mM])\s+(?\){0,1}(?\d+){0,1}\s+(?.+)"}
Private Function GetMatchingRegex(ByVal line As String) As Match
Dim rx As Regex, m As Match
For i As Integer = 0 To _ParseFormats.Length - 1
rx = New Regex(_ParseFormats(i))
m = rx.Match(line)
If m.Success Then Return m
Next
Return Nothing
End Function
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply