Post reply

Efficiently copying big backup files over the network

  • August 1, 2005 at 7:47 pm

    #165105

    Hello!
     
       We are using SQLSafe software to backup/compress our databases. We would like to copy backup files (could be as big as 400GB compressed) to the different computer within the same network. I know there are tools like robocopy,xcopy etc. that are suitable for this job. I was wondering if anybody can share his/her own experience in copying big backup files. Ideally,  we wouldn't like to saturate our network/cpu resources while copying these files.
     
    Thanks,
    Igor

  • August 2, 2005 at 4:27 am

    #578775

    I've sucessfully used robocopy for this type of thing in the past, but because I'm paranoid I also generate a MD5hash before and after I sent it, so long as they both match there was no problem in the transfer (I know, paranoid )

  • August 2, 2005 at 5:02 am

    #578782

    Same here, we moved our slightly smaller 200GB backups using Robocopy. Can't say exactly how much it affected performance, but as I remember it we did not have any CPU problems from it. We did move the backup over a separate NIC though.

    --
    Chris Hedgate http://www.hedgate.net/
    Contributor to the Best of SQL Server Central volumes
    Articles: http://www.sqlservercentral.com/columnists/chedgate/

  • August 2, 2005 at 5:15 am

    #578784

    A seperate NIC is also the way to go, I'd forgotten that bit

  • August 2, 2005 at 12:29 pm

    #578949

    Thanks for your input!

    Igor

  • August 2, 2005 at 12:37 pm

    #578952

    Thanks Mike! Could you please eleborate on you generated MD5hash?

    Thanks,

    Igor

  • August 3, 2005 at 2:22 am

    #579041

    Not a problem, I found a small utility called MD5File.exe which you can use to generate an MD5 hash for a file and then repeat after it's been copied, so long as both hashes are the same then the file is identical (depending upon how big the file is it could take a while to generate).

    The best bet is to send the results to a text file which makes it easier to compare later.

    I can't remember where I got it but I've just googled it and there's plenty of hits comming back.

     

  • August 3, 2005 at 4:13 pm

    #579382

    You MUST compress the file before moving it across the network. I'm not sure how big of files WinZip will do, but this is what I use. I get about 90% compression.

    This will save you a tremendous amount of time; even when considering the time to compress and decompress. Saves time if you're on a standard network, that is...

    cl

    Signature is NULL

  • August 3, 2005 at 4:19 pm

    #579383

    Calvin,

    Our backup files already compressed by SQLSafe.

    Thanks,

    Igor

     

     

  • August 3, 2005 at 4:20 pm

    #579384

    Thanks a lot,

    Igor

  • August 3, 2005 at 4:27 pm

    #579385

    Whoops...not familiar with that software.

    Well...good job, then.

    Signature is NULL

Viewing 11 posts - 1 through 10 (of 10 total)

You must be logged in to reply to this topic. Login to reply