Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Linux Punk wrote at about 16:53:08 -0600 on Sunday, October 26, 2008: On Sun, Oct 26, 2008 at 4:49 AM, Steen Eugen Poulsen [EMAIL PROTECTED] wrote: Jeffrey J. Kosowsky skrev: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. I don't think it has anything to do with cygwin or how you backup windows machines, it's just that we keep running into file lock issues that messes up backups. I would tend to disagree because we have dozens of machines that backup successfully using rsyncd server (no ssh) but we ran into exactly the issues described when tunneling rsync in ssh. If it was a problem with locked files affecting rsync, we would see those problems in both instances. We've also tried backing up dummy directories that have no open files in them, and had that fail with rsync/ssh. I just switched over to using rsyncd (without ssh) and it seems to be working (so far after several gigabytes and several thousand files). I also just as an experiment tried manually setting up a ssh tunnel (e.g. -L 1873:localhost:873) and tunneling rsyncd over it -- and it too seems to work. So far I have tried the following permutations: 1. From Linux: rsync -av -e ssh linux remote:src local dest WORKS 2. From Linux: rsync -av -e ssh local src linux remote:dest WORKS 3. From Windows: rsync -av -e ssh windows remote:src local dest WORKS 4. From Windows: rsync -av -e ssh local src windows remote:dest WORKS 5. From Linux: rsync -av -e ssh windows remote:src local dest FAILS after ~10 files 6. From Window: rsync -av -e ssh local src linux remote:dest FAILS (completes but sends 0 files - when done under -vv lots of send/receive traffic but nothing effectively sent) 7. From Linux: rsync -av local src local dest WORKS 8. From Windows: rsync -av local src local dest WORKS 9. From Linux: rsync -av windows remote::src local dest plus 'rsync --daemon' running on Windows machine WORKS 10. From Linux: rsync -av --port 1873 127.0.0.1::src local dest plus 'rsync --daemon --address 127.0.0.1 --port 1873 running on Windows and ssh tunnel from Linux to Windows with -L 1873:127.0.0.1:1873 WORKS NOTE: when I say 'WORKS', I mean that it copies over at least several thousand files starting from root (either / on Linux or /c/ on Windows) since I didn't have the time to copy over the 50GB of each partition for each method. So it seems of all the multiple combinations of using ssh/rsync/rsyncd between src/dir and linux/windows (and I haven't enumerated other obvious permutations that clearly work), the only combo that doesn't work is using rsync -e ssh to copy over Windows root directory to Linux (whether you initiate rsync from the linux or windows machine). Can anybody explain what is going on here and what is causing the problem? Is this a fundamental ssh/rsync problem between Linux and Windows? Could the problem be due to the fact that I have different rsync and ssh versions? Currently: Linux: openssh-4.7p1 rsync-2.6.9 Windows: openssh-5.1p1 rsync-3.0.4 protocol version 30 I really would like to get to the bottom of this... - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Linux Punk skrev: I would tend to disagree because we have dozens of machines that backup successfully using rsyncd server (no ssh) but we ran into Well, just to shoot that theory down, my problem is with no ssh rsyncd server. smime.p7s Description: S/MIME Cryptographic Signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
it is just a fancy interface on rsync but it is setup as a service and is quick and easy. Also, it doesnt conflict with cygwin in my experience. On Sun, Oct 26, 2008 at 10:20 PM, Jeffrey J. Kosowsky [EMAIL PROTECTED] wrote: dan wrote at about 21:45:25 -0600 on Sunday, October 26, 2008: I have posted this a few times but here it is again. Instead of installing cygwin completely, install Deltacopy instead. 1) It is rsync on windows via cygwin 2) it has a nice GUI 3) directorys available via rsync are settable via aliases. (Documents and Settings can be docs) 4) can use authentication to provide some basic security. 5) runs as a service 6) service can be run under a username that is capable of reading all files but not writing I use this extensively while backing up windows desktops. - Can it coexist with a full cygwin install (I have cygwin installed already) or does it have conflicts with the cygwin1.dll (or other things? - Is it doing something different from cygwin rsync/ssh or is it just a nicer/cleaner UI? If so, what is different that makes it work better? Thanks - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. For me, Backuppc repeatedly hangs after backing up the first dozen or so files on any of my Windows machines. On any given machine, the hang always occurs on the same file but the specific file that hangs differs between machines. The file that it hangs on though seems to be a normal file (i.e. not corrupted). When I run 'rsync -avuxH windows machine:/c/ backup directory', the rsync process seems to copy over most directories (~12000/15000) but only 13 files before it too hangs. (Interestingly, the files copied over are not the same ones that get copied with Backuppc; also Backuppc only copies over a couple of directories (not 12,000) before hanging). I am using BackupPC-3.1.0 with rsync-2.6.9 The cygwin remote versions are: OpenSSH_4.5p1, OpenSSL 0.9.8d 28 Sep 2006 rsync version 2.6.6 protocol version 29 Has anybody figured out what is wrong with rsync/ssh and how to fix it? Are there other (free) versions of rsync/ssh for Windows that would work better? (I did see one solution posted that involves tunneling rsyncd over a manually set up ssh tunnel but I would prefer to just get the normal rsync/ssh process working). - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Jeffrey J. Kosowsky schrieb: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. For me, Backuppc repeatedly hangs after backing up the first dozen or so files on any of my Windows machines. On any given machine, the hang always occurs on the same file but the specific file that hangs differs between machines. The file that it hangs on though seems to be a normal file (i.e. not corrupted). When I run 'rsync -avuxH windows machine:/c/ backup directory', the rsync process seems to copy over most directories (~12000/15000) but only 13 files before it too hangs. (Interestingly, the files copied over are not the same ones that get copied with Backuppc; also Backuppc only copies over a couple of directories (not 12,000) before hanging). I am using BackupPC-3.1.0 with rsync-2.6.9 The cygwin remote versions are: OpenSSH_4.5p1, OpenSSL 0.9.8d 28 Sep 2006 rsync version 2.6.6 protocol version 29 Has anybody figured out what is wrong with rsync/ssh and how to fix it? Are there other (free) versions of rsync/ssh for Windows that would work better? Your Windows version of rsync is very outdated. You may try the newest one from: www.itefix.no/cwrsync -- Tomasz Chmielewski http://wpkg.org - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Jeffrey J. Kosowsky skrev: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. I don't think it has anything to do with cygwin or how you backup windows machines, it's just that we keep running into file lock issues that messes up backups. To successfully backup Windows machine I need a huge exclude list, without it things always seem to be very troubled. smime.p7s Description: S/MIME Cryptographic Signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Steen Eugen Poulsen schrieb: Jeffrey J. Kosowsky skrev: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. I don't think it has anything to do with cygwin or how you backup windows machines, it's just that we keep running into file lock issues that messes up backups. To successfully backup Windows machine I need a huge exclude list, without it things always seem to be very troubled. If you backup Windows 2003 or later, you should: - snapshot your drives with shadow copy - this makes sure you don't have file in use problem - run rsyncd service as SYSTEM user - use the latest cwrsync with the latest _snapshot_ cygwin1.dll from http://cygwin.com/snapshots/ - it allows Cygwin programs to open files which are exclusively owned by other users -- Tomasz Chmielewski http://wpkg.org - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Tomasz Chmielewski wrote at about 11:42:22 +0100 on Sunday, October 26, 2008: Jeffrey J. Kosowsky schrieb: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. For me, Backuppc repeatedly hangs after backing up the first dozen or so files on any of my Windows machines. On any given machine, the hang always occurs on the same file but the specific file that hangs differs between machines. The file that it hangs on though seems to be a normal file (i.e. not corrupted). When I run 'rsync -avuxH windows machine:/c/ backup directory', the rsync process seems to copy over most directories (~12000/15000) but only 13 files before it too hangs. (Interestingly, the files copied over are not the same ones that get copied with Backuppc; also Backuppc only copies over a couple of directories (not 12,000) before hanging). I am using BackupPC-3.1.0 with rsync-2.6.9 The cygwin remote versions are: OpenSSH_4.5p1, OpenSSL 0.9.8d 28 Sep 2006 rsync version 2.6.6 protocol version 29 Has anybody figured out what is wrong with rsync/ssh and how to fix it? Are there other (free) versions of rsync/ssh for Windows that would work better? Your Windows version of rsync is very outdated. Oops I'm sorry. I'm actually using the following on the Windows machine: rsync version 3.0.4 protocol version 30 OpenSSH_5.1p1, OpenSSL 0.9.8i 15 Sep 2008 You may try the newest one from: www.itefix.no/cwrsync -- Tomasz Chmielewski http://wpkg.org - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
On Sun, Oct 26, 2008 at 4:49 AM, Steen Eugen Poulsen [EMAIL PROTECTED] wrote: Jeffrey J. Kosowsky skrev: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. I don't think it has anything to do with cygwin or how you backup windows machines, it's just that we keep running into file lock issues that messes up backups. I would tend to disagree because we have dozens of machines that backup successfully using rsyncd server (no ssh) but we ran into exactly the issues described when tunneling rsync in ssh. If it was a problem with locked files affecting rsync, we would see those problems in both instances. We've also tried backing up dummy directories that have no open files in them, and had that fail with rsync/ssh. To successfully backup Windows machine I need a huge exclude list, without it things always seem to be very troubled. We have a short exclude list, but we still get lots of failures on various files that should probably be excluded. We have no problems with the backups locking up on only certain files though. Brian Oborn - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
I find it more convenient to do includes rather than excludes. Set the RSYNC share to c:/Documents and Settings/ and then include Favorites, MyDocuments and Desktop seems to work pretty well. I just make it clear to users that anything not in those places will not be backed up. Of course if there are any particular requirements on one machine you can set that up on a per machine basis as required. RSYNCD seems to be the most reliable method for me. I don't bother with ssh. On Sun, 2008-10-26 at 16:53 -0600, Linux Punk wrote: On Sun, Oct 26, 2008 at 4:49 AM, Steen Eugen Poulsen [EMAIL PROTECTED] wrote: Jeffrey J. Kosowsky skrev: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. I don't think it has anything to do with cygwin or how you backup windows machines, it's just that we keep running into file lock issues that messes up backups. I would tend to disagree because we have dozens of machines that backup successfully using rsyncd server (no ssh) but we ran into exactly the issues described when tunneling rsync in ssh. If it was a problem with locked files affecting rsync, we would see those problems in both instances. We've also tried backing up dummy directories that have no open files in them, and had that fail with rsync/ssh. To successfully backup Windows machine I need a huge exclude list, without it things always seem to be very troubled. We have a short exclude list, but we still get lots of failures on various files that should probably be excluded. We have no problems with the backups locking up on only certain files though. Brian Oborn - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
Hi, Jeffrey J. Kosowsky wrote on 2008-10-26 09:51:06 -0400 [Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC]: Jeffrey J. Kosowsky schrieb: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. did you read [EMAIL PROTECTED] [1] ? [...] Oops I'm sorry. I'm actually using the following on the Windows machine: rsync version 3.0.4 protocol version 30 OpenSSH_5.1p1, OpenSSL 0.9.8i 15 Sep 2008 Apparently it works with that version for other people (but I'm just quoting, I don't backup Windoze ;-). Regards, Holger [1] http://sourceforge.net/mailarchive/[EMAIL PROTECTED] - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
I have posted this a few times but here it is again. Instead of installing cygwin completely, install Deltacopy instead. 1) It is rsync on windows via cygwin 2) it has a nice GUI 3) directorys available via rsync are settable via aliases. (Documents and Settings can be docs) 4) can use authentication to provide some basic security. 5) runs as a service 6) service can be run under a username that is capable of reading all files but not writing I use this extensively while backing up windows desktops. On Sun, Oct 26, 2008 at 9:11 PM, Holger Parplies [EMAIL PROTECTED] wrote: Hi, Jeffrey J. Kosowsky wrote on 2008-10-26 09:51:06 -0400 [Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC]: Jeffrey J. Kosowsky schrieb: I have googled and read a lot of posts about people having trouble with cygwin rsync/ssh but haven't seen any definitive solutions. did you read [EMAIL PROTECTED] [1] ? [...] Oops I'm sorry. I'm actually using the following on the Windows machine: rsync version 3.0.4 protocol version 30 OpenSSH_5.1p1, OpenSSL 0.9.8i 15 Sep 2008 Apparently it works with that version for other people (but I'm just quoting, I don't backup Windoze ;-). Regards, Holger [1] http://sourceforge.net/mailarchive/[EMAIL PROTECTED] - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] What is the latest on getting cygwin rsync/ssh to work for BackupPC
dan wrote at about 21:45:25 -0600 on Sunday, October 26, 2008: I have posted this a few times but here it is again. Instead of installing cygwin completely, install Deltacopy instead. 1) It is rsync on windows via cygwin 2) it has a nice GUI 3) directorys available via rsync are settable via aliases. (Documents and Settings can be docs) 4) can use authentication to provide some basic security. 5) runs as a service 6) service can be run under a username that is capable of reading all files but not writing I use this extensively while backing up windows desktops. - Can it coexist with a full cygwin install (I have cygwin installed already) or does it have conflicts with the cygwin1.dll (or other things? - Is it doing something different from cygwin rsync/ssh or is it just a nicer/cleaner UI? If so, what is different that makes it work better? Thanks - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
