need rsync alternative or solution.

yfni&len · 12-31-2008, 01:58 PM

I have two fileservers. Fileserver A = Source. Fileserver B = Destination/Backup.

They are connected to eachother through the internet but are both on the same vpn.

Since the amount of data transferred is massive ( 750gb+ sometimes more).
I use an external hdd as go between. (inet connection is adsl2 max)

So what rsync does is compare A to B and anything that is missing on B is copied to the external HDD. Which I then bring to location B and do a normal copy paste. This all works fine.

The process of comparing A to B and then copying from A to ext HDD takes too long (it's esata). Takes about a day for 300gb. I don't understand why.

So I need an alternative or a fix for rsync.
both run windows.

below you see my rsync code

Code:

SETLOCAL
SET CWRSYNCHOME=%PROGRAMFILES%\CWRSYNC
SET CYGWIN=nontsec
SET HOME=%HOMEDRIVE%%HOMEPATH%
SET CWOLDPATH=%PATH%
SET PATH=%CWRSYNCHOME%\BIN;%PATH%
rsync -ar --exclude "\[A.00\]\ Nieuw" --exclude "\[D.00\]\ Familie\ Video8" --exclude "\[D.01\]\ Familie\ VHS" --exclude "\[D.02\]\ Familie\ DVR" --exclude "\[C.02\]\ TV\ Series\ (2\)" --compare-dest=/cygdrive/y/ /cygdrive/c/MediaJunc/Films/ /cygdrive/r/

Innoc · 12-31-2008, 03:16 PM

Can you provide a little more information on your hardware? What brand and model servers or motherboards? What brand and model is the USB drive. You could be seeing an issue with the USB or the drive itself. I have seen HUGE differences in the effective data transfer rates between drives and USB controllers.

By way of example...When imaging a Dell Optiplex 330 running Windows 2003 Server (clean build) it takes about 1/2 hour. The identical image made from a Dell 1950 server takes about 4 minutes. In both instances the same external drive is used. The installed operating system has no impact on the imaging time as this is all done from a bootable Windows PE environment.

I suspect that you're probably dealing with a USB bottleneck. Lower tier hardware either in the USB controller or the external hard drive.

yfni&len · 12-31-2008, 06:00 PM

I've just found a tiny program that seems to be able to do what I want.

It's called ThirdDir found it in one of the remote corners of the web...It's not pretty but it's easy to use and seems to do what I want.

Although it has some silly bugs....It doesn't allow capitals in the exception/ignore box. Can work around it by just replace the capitals with *.
http://www.mtg.sk/rva/

thanks for reply!

12-31-2008, 01:58 PM	#1
yfni&len Join Date: Dec 2008 Gametype: Capture the Flag Posts Rated Helpful 0 Times	need rsync alternative or solution. I have two fileservers. Fileserver A = Source. Fileserver B = Destination/Backup. They are connected to eachother through the internet but are both on the same vpn. Since the amount of data transferred is massive ( 750gb+ sometimes more). I use an external hdd as go between. (inet connection is adsl2 max) So what rsync does is compare A to B and anything that is missing on B is copied to the external HDD. Which I then bring to location B and do a normal copy paste. This all works fine. The process of comparing A to B and then copying from A to ext HDD takes too long (it's esata). Takes about a day for 300gb. I don't understand why. So I need an alternative or a fix for rsync. both run windows. below you see my rsync code Code: SETLOCAL SET CWRSYNCHOME=%PROGRAMFILES%\CWRSYNC SET CYGWIN=nontsec SET HOME=%HOMEDRIVE%%HOMEPATH% SET CWOLDPATH=%PATH% SET PATH=%CWRSYNCHOME%\BIN;%PATH% rsync -ar --exclude "\[A.00\]\ Nieuw" --exclude "\[D.00\]\ Familie\ Video8" --exclude "\[D.01\]\ Familie\ VHS" --exclude "\[D.02\]\ Familie\ DVR" --exclude "\[C.02\]\ TV\ Series\ (2\)" --compare-dest=/cygdrive/y/ /cygdrive/c/MediaJunc/Films/ /cygdrive/r/ Last edited by yfni&len; 12-31-2008 at 02:13 PM.

12-31-2008, 03:16 PM	#2
Innoc Hitman 2 1 Actual Join Date: Mar 2007 Location: "Oscar Mike" Gametype: FPS or RTS (just say NO to MMO) Affiliations: Your Mom Posts Rated Helpful 8 Times	Can you provide a little more information on your hardware? What brand and model servers or motherboards? What brand and model is the USB drive. You could be seeing an issue with the USB or the drive itself. I have seen HUGE differences in the effective data transfer rates between drives and USB controllers. By way of example...When imaging a Dell Optiplex 330 running Windows 2003 Server (clean build) it takes about 1/2 hour. The identical image made from a Dell 1950 server takes about 4 minutes. In both instances the same external drive is used. The installed operating system has no impact on the imaging time as this is all done from a bootable Windows PE environment. I suspect that you're probably dealing with a USB bottleneck. Lower tier hardware either in the USB controller or the external hard drive. __________________ Mooga on Obama: He can cut taxes. Actually do something useful. Punch Nancy Pelosi in the face. Just to name a few. You eventually run out of other people's money to spend.

12-31-2008, 06:00 PM	#3
yfni&len Join Date: Dec 2008 Gametype: Capture the Flag Posts Rated Helpful 0 Times	I've just found a tiny program that seems to be able to do what I want. It's called ThirdDir found it in one of the remote corners of the web...It's not pretty but it's easy to use and seems to do what I want. Although it has some silly bugs....It doesn't allow capitals in the exception/ignore box. Can work around it by just replace the capitals with *. http://www.mtg.sk/rva/ thanks for reply!

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)