[BackupPC-users] BackupPC and Windows/ Cygwin clients via ssh/ rsync

Discussion:

David Christensen

2009-02-16 20:56:20 UTC

backuppc-users:

I am a BackupPC newbie who has installed BackupPC on a dedicated Debian
4 machine and is successfully backing up Linux and FreeBSD machines via
ssh/ rsync. Good stuff! :-)

However, when I attempt to backup Windows XP Professional SP3/ Cygwin
machines via ssh/rsync, BackupPC will process anywhere from dozens to
hundreds of files and then hang, usually on a large file (console
session follows).

RTFM and STFW, I found various items, but no resolution.

I tried invoking rsync from the server console to pull in files from a
Windows/ Cygwin machine, but the process also hanged.

I think the problem lies with Cygwin, but don't know how to troubleshoot
it.

Has anybody been able to get BackupPC working with Windows/ Cygwin
clients via ssh/ rsync?

TIA,

David

--

2009-02-16 11:55:07 ***@p3450 ~
$ cat /etc/backuppc/p43400e.pl
# $Id: p43400e.pl,v 1.1 2009/02/15 19:49:00 dpchrist Exp $
#
# BackupPC host configuration file for p43400e.

$Conf{XferMethod} = 'rsync';

$Conf{RsyncShareName} = [ qw(
/cygdrive/h/data
/cygdrive/h/outlook
/cygdrive/h/userdocs
) ];

$Conf{RsyncClientCmd} = '$sshPath -q -x -l backup $host $rsyncPath
$argList+';

2009-02-16 11:55:15 ***@p3450 ~
$ bin/BackupPC_dump -v -f p43400e
cmdSystemOrEval: about to system /bin/ping -c 1 p43400e
cmdSystemOrEval: finished: got output PING p43400e.holgerdanske.com (192
.168.0.15) 56(84) bytes of data.
64 bytes from p43400e.holgerdanske.com (192.168.0.15): icmp_seq=1 ttl=12
8 time=1.39 ms

--- p43400e.holgerdanske.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.391/1.391/1.391/0.000 ms

cmdSystemOrEval: about to system /bin/ping -c 1 p43400e
cmdSystemOrEval: finished: got output PING p43400e.holgerdanske.com (192
.168.0.15) 56(84) bytes of data.
64 bytes from p43400e.holgerdanske.com (192.168.0.15): icmp_seq=1 ttl=12
8 time=1.39 ms

--- p43400e.holgerdanske.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.394/1.394/1.394/0.000 ms

CheckHostAlive: returning 1.39
started full dump, share=/cygdrive/h/data
Running: /usr/bin/ssh -q -x -l backup p43400e /usr/bin/rsync --server --
sender --numeric-ids --perms --owner --group --devices --links --times -
-block-size=2048 --recursive -D --ignore-times . /cygdrive/h/data/
Xfer PIDs are now 2887
xferPids 2887
Got remote protocol 30
Negotiated protocol version 26
Xfer PIDs are now 2887,2888
xferPids 2887,2888
create d 700 500/4294967295 0 .
create d 700 500/4294967295 0 attachments
<snip>
same 700 500/4294967295 47835 benchmark/tyan-s1854-trinity-4
00-p3-1000-20061203-1624.html
same 700 500/4294967295 29455 benchmark/tyan-s1854-trinity-4
00-p3-533b-20041101-1957.html
create d 700 500/4294967295 0 bittorrent
create d 700 1004/513 0 bittorrent/chrysaor.info
finish: removing in-process file bittorrent/chrysaor.info/debian-500-i38
6-netinst.iso
Child is aborting
Done: 182 files, 26562470 bytes
Got fatal error during xfer (aborted by user (signal=INT))
Backup aborted by user signal
dump failed: aborted by user (signal=INT)
link p43400e

2009-02-16 12:03:48 ***@p3450 ~
$

Cody Dunne

2009-02-16 21:05:42 UTC

Permalink

Hi David,

Post by David Christensen
However, when I attempt to backup Windows XP Professional SP3/ Cygwin
machines via ssh/rsync, BackupPC will process anywhere from dozens to
hundreds of files and then hang, usually on a large file (console
session follows).

This is a known problem with ssh/rsync on Cygwin. Use rsyncd on the
windows client instead and if you want ssh, set up a tunnel before hand.
Here is a guide that will be on the wiki soon:
http://www.cs.umd.edu/~cdunne/projs/backuppc_guide.html

Cody

David Christensen

2009-02-16 21:57:11 UTC

Permalink

Post by Cody Dunne
http://www.cs.umd.edu/~cdunne/projs/backuppc_guide.html

Which states:

Regular rsync over SSH doesn't work between Linux and Windows/Cygwin
boxes because of a bug fixed in rsync 3.0

On Debian 4, I see:

2009-02-16 13:37:56 ***@p3450 ~
# rsync --version
rsync version 2.6.9 protocol version 29
<snip>

2009-02-16 13:41:37 ***@p3450 ~
# ssh -V
OpenSSH_4.3p2 Debian-9etch3, OpenSSL 0.9.8c 05 Sep 2006

2009-02-16 13:57:13 ***@p3450 ~
# cat /etc/debian_version
4.0

On Cygwin, I see:

2009-02-16 13:40:44 ***@p43400e ~
$ rsync --version
rsync version 3.0.4 protocol version 30
<snip>

2009-02-16 13:40:50 ***@p43400e ~
$ ssh -V
OpenSSH_5.1p1, OpenSSL 0.9.8j 07 Jan 2009

So, it looks like the rsync and ssh binaries in Debian 4.0 are older
versions than those on Cygwin. Debian 5 has ssh 5.1p1-5 and rsync
3.0.3-2:

http://packages.debian.org/lenny/openssh-client

http://packages.debian.org/lenny/rsync

If I rebuild the box using Debian 5, should BackupPC work for Windows/
Cygwin via ssh/ rsync?

David

Cody Dunne

2009-02-16 23:50:58 UTC

Permalink

Hi David,

Post by David Christensen
Regular rsync over SSH doesn't work between Linux and Windows/Cygwin
boxes because of a bug fixed in rsync 3.0

The following line says the problem still remains with BackupPC, though.
I'm not sure if plain rsync 3.0/ssh works over cygwin, but it definitely
doesn't with BackupPC, which uses the File::RsyncP perl module which
uses the older protocol 28 instead of 30. Maybe someone who knows more
about it could tell us if it was fixed, and I can update the guide.

Post by David Christensen
If I rebuild the box using Debian 5, should BackupPC work for Windows/
Cygwin via ssh/ rsync?

No, see above.

Cody

royden yates

2009-02-17 00:05:53 UTC

Permalink

Post by Cody Dunne
Hi David,

Post by David Christensen
Regular rsync over SSH doesn't work between Linux and Windows/Cygwin
boxes because of a bug fixed in rsync 3.0

Post by David Christensen
If I rebuild the box using Debian 5, should BackupPC work for Windows/
Cygwin via ssh/ rsync?

No, see above.
Cody

The traditional problem with Backuppc and cygwin of a rsync over ssh
freeze on any sizeable back up has disappeared with Cygwin and rsync
3.0.3 and greater, as one can get a completed run. I had machines that
always failed and that now complete. There are however seemingly big
file issues (eg. *.pst files of ca. 2 gigs or so (VSS copied to temp
directory so file locking is not the issue!)) that strike me not on the
initial full which completes, but on subsequent incremental runs when
the exchange of differential info hangs.

So I guess it is a case of YMMV.

Royden

royden yates

2009-02-16 23:56:11 UTC

Permalink

Post by David Christensen

Post by Cody Dunne
http://www.cs.umd.edu/~cdunne/projs/backuppc_guide.html

Regular rsync over SSH doesn't work between Linux and Windows/Cygwin
boxes because of a bug fixed in rsync 3.0
# rsync --version
rsync version 2.6.9 protocol version 29
<snip>
# ssh -V
OpenSSH_4.3p2 Debian-9etch3, OpenSSL 0.9.8c 05 Sep 2006
# cat /etc/debian_version
4.0
$ rsync --version
rsync version 3.0.4 protocol version 30
<snip>
$ ssh -V
OpenSSH_5.1p1, OpenSSL 0.9.8j 07 Jan 2009
So, it looks like the rsync and ssh binaries in Debian 4.0 are older
versions than those on Cygwin. Debian 5 has ssh 5.1p1-5 and rsync
http://packages.debian.org/lenny/openssh-client
http://packages.debian.org/lenny/rsync
If I rebuild the box using Debian 5, should BackupPC work for Windows/
Cygwin via ssh/ rsync?
David

Concerning Backuppc and Windows systems, the issues pertained to the
interaction of Cygwin rsync < 3.0.0 and AFAIK Backuppc's perl
implementation of rsync. If issues remain with a cygwin rsync >= 3.0.0
then upgrading the rsync of the Backuppc server is not the solution as
Backuppc provides its own rsync.

There was a thread here a while back that noted that cygwin rsync v
3.0.3 appeared to solve the problems that plagued users before with
rsync over ssh. My testing confirms that this is so, BUT that issues
remain. As others here have noted, large files mysteriously can still
cause hiccups and my experience shows that this can happen with both
cygwin rsync over ssh and with rsyncd over ssh tunnel.

I now want to transfer these troublesome directories to a *nix box to
see if I can replicate the problems in a "native" environment.

Royden

Jeffrey J. Kosowsky

2009-02-17 00:24:20 UTC

Permalink

Post by David Christensen

Post by Cody Dunne
http://www.cs.umd.edu/~cdunne/projs/backuppc_guide.html

No the problem is the perl library perl-File-RsyncP does not work with
protocol 30 (the rsync 3.0 protocol) so it reverts to the older
protocol which has issues with various Windows files.

Carl Wilhelm Soderstrom

2009-02-16 21:06:21 UTC

Permalink

Post by David Christensen
Has anybody been able to get BackupPC working with Windows/ Cygwin
clients via ssh/ rsync?

Last I knew, there was still a bug in cygwin ssh+rsync that prevented it
from working.

--
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

Jeffrey J. Kosowsky

2009-02-17 00:21:58 UTC

Permalink

Post by David Christensen
I am a BackupPC newbie who has installed BackupPC on a dedicated Debian
4 machine and is successfully backing up Linux and FreeBSD machines via
ssh/ rsync. Good stuff! :-)
However, when I attempt to backup Windows XP Professional SP3/ Cygwin
machines via ssh/rsync, BackupPC will process anywhere from dozens to
hundreds of files and then hang, usually on a large file (console
session follows).
RTFM and STFW, I found various items, but no resolution.

You might want to also STFA (A=archives) because this problem has been
discussed MANY times before ;)