2

I need to copy a directory from one of our servers several hundred miles away. Both sites where I am and where the files are have business cable. The cable is 5 Mbps(Down) / 768 Kbps(Up). This poses a small problem considering I need to copy about 85GB of data.

I used a file transfer calculator and it said it would take 11 days and some change.

I was wondering if there is any way to leverage something like DeDup to speed up the transfer.

The server I would be copying from is Windows 2003 R2, my box is Windows 7 Pro x64.

I was going to transfer the data to my iSCSI target which is running Ubuntu Linux, with ZFS and SCST. Would the dedup being performed by ZFS aid in the transfer so does the data need to be DeDup'd on the sending end?

AtomicPorkchop
  • 2,005

3 Answers3

19

Buy an external hard drive. Copy data at site A. Mail it to site B. Copy to machine at site B.

Don't underestimate the power of the postal service.

MDMarra
  • 101,323
5

Is there duplicate data?

If not, then a dedup will help not at all; if there is, it will only be deduplicated over the wire if you transfer it with zfs send -D - since your destination is Windows, then that's probably a no.

Don't forget good old fashioned compression.

Community
  • 1
Shane Madden
  • 116,404
  • 13
  • 187
  • 256
0

For Directory with lots of redundant data, you can try DZO Compressor. Deduplication makes sense. DZO performs Deduplication as well as Lossless Compression. The product is still in beta phase. As per initial results on Data with duplicates, it performs 2-3 times better than winzip, winrar and others. See their website http://essensolabs.com/ for details or download the beta from below link. http://essensolabs.com/invitation-trial.html

If the data doesn't contain duplicates, then it gives the same compression as 7zip as it uses lzma for lossless compression.

Above can only help in reducing the size and thus lesser time for transfer, but it may still be in days !!!

Neeraj
  • 11