Downloading file by script task over http

Question

Post reply

Downloading file by script task over http

FrankMeng021

SSC Veteran

Points: 255
More actions
May 13, 2008 at 3:09 pm

#129070

Hi,
I need to download a page via http. I tried 2 ways but they were all not working because they didn't pass the authentication. The file downloaded is not what I need but the page used for entering username and password.
the 1st way:
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
'
' Create an HttpClientConnection and use it to download
' a file from the location the connection manager specifies
'
Dim httpConnection As Microsoft.SqlServer.Dts.Runtime.HttpClientConnection
Dim temp As Object
' Try to get the connection
Try
temp = Dts.Connections("FileToDownload").AcquireConnection(Nothing)
httpConnection = New HttpClientConnection(temp)
httpConnection.DownloadFile(Dts.Variables("DOWNLOADEDFILE").Value.ToString(), True)
Catch ex As Exception
Dts.Events.FireError(1, ex.TargetSite.ToString(), ex.Message, "", 0)
End Try
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
-----------------------------------------------------
I do enter the username and password in FileToDownload http connection, but it doesn't work.
the 2nd way:
Imports System
Imports System.Net
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
Dim MyWebClient As WebClient
Dim RemoteUri As String
Dim LocalFileName As String
Dim FireAgain As Boolean
Try
MyWebClient = New WebClient()
' get the context from variables
RemoteUri = "http://abc.com/DataUniverse.aspx?SecurityTypeId=ST00000001"
LocalFileName = "C:\Test.html"
' tell the user what we're downloading where
Dts.Events.FireInformation(0, String.Empty, String.Format("Downloading '{0}' from '{1}'", LocalFileName, RemoteUri), String.Empty, 0, FireAgain)
' do the actual download
MyWebClient.Credentials = New System.Net.NetworkCredential("UserName", "Password")
MyWebClient.DownloadFile(RemoteUri, LocalFileName)
Dts.TaskResult = Dts.Results.Success
Catch ex As Exception
' post the error message we got back.
Dts.Events.FireError(0, String.Empty, ex.Message, String.Empty, 0)
Dts.TaskResult = Dts.Results.Failure
End Try
End Sub
End Class
------------------------------------------------------
this way also doen't work.
Thanks,
mengmou

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply

Frank Bazan SSCertifiable Points: 5378 More actions · Answer 1

Do you use a proxy? You might have to add one to your code?

The following has worked for me...

Imports System

Imports System.Data

Imports System.Math

Imports Microsoft.SqlServer.Dts.Runtime

Imports System.Net

Public Class ScriptMain

Public Sub Main()

Dim WebConnection As New WebClient()

Dim proxyConnection As New WebProxy("http://proxy.server.com")

Dim creds As New NetworkCredential("UserName", "Password")

Try

With WebConnection

.Proxy = proxyConnection

.BaseAddress = "https://www.myDomain.com/"

.Credentials = creds

End With

Catch ex As Exception

Dts.Events.FireError(0, "Problem connecting to website: ", ex.Message, "", 0)

End Try

Try

With WebConnection

.DownloadFile("https://www.myDomain.com/folder/", "myFileName.zip")

End With

Catch ex As Exception

Dts.Events.FireError(0, "Problem downloading file: ", ex.Message, "", 0)

End Try

Dts.TaskResult = Dts.Results.Success

End Sub

End Class

Kindest Regards,

Frank Bazan

dmusson17 Ten Centuries Points: 1235 More actions · Answer 2

Hi Frank,

How would your code look without the use of a proxy server?

Thanks,

Dave

Frank Bazan SSCertifiable Points: 5378 More actions · Answer 3

Hi David,

I'd try just removing the proxy declaration and method...

Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports System.Net
Public Class ScriptMain
Public Sub Main()
Dim WebConnection As New WebClient()
Dim creds As New NetworkCredential("UserName", "Password")
Try
With WebConnection
.BaseAddress = "https://www.myDomain.com/"
.Credentials = creds
End With
Catch ex As Exception
Dts.Events.FireError(0, "Problem connecting to website: ", ex.Message, "", 0)
End Try
Try
With WebConnection
.DownloadFile("https://www.myDomain.com/folder/", "myFileName.zip")
End With
Catch ex As Exception
Dts.Events.FireError(0, "Problem downloading file: ", ex.Message, "", 0)
End Try
Dts.TaskResult = Dts.Results.Success
End Sub
End Class

Cheers

Kindest Regards,

Frank Bazan

dmusson17 Ten Centuries Points: 1235 More actions · Answer 4

dmusson17

Ten Centuries

Points: 1235

May 16, 2008 at 7:06 am

#816446

Worked like a charm. Thanks, Frank!

FrankMeng021 SSC Veteran Points: 255 More actions · Answer 5

Thanks for your reply. But my URL is different from yours like a path.

http://abc.com/DataUniverse.aspx?SecurityTypeId=ST00000001

I have created a C# program to download the file.

thanks ervery one!

Jenny Zhou-459989 SSCommitted Points: 1961 More actions · Answer 6

Hi I'm doing the same thing, download file from https. the webconnection part works but the downloadfile part doesn't.

With WebConnection

.DownloadFile("http://www.site.com", "file.zip")

is file.zip the file on http or the filename to save as on my machine? and where do i specify where to download to? create a variable?

also i need to compare all the files on https with the files on my local archive folder and then only download those files which are not in my local archive folder. how do i add that piece in the script task?

Thanks!