Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-09-02 Thread dan
I might point out that if you rsync between two windows machine using the same version of cwrsync or deltacopy(which is basically still cwrsync but a different version) the transfer speeds are higher but not anywhere near a smb transfer. One solution might be to create a shadow copy, then explorer

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-09-01 Thread Jeffrey J. Kosowsky
Jacob Hydeman wrote at about 21:31:18 -0700 on Monday, August 31, 2009: > The simple file sharing is a requirement of the winexe service. You can read > more about it here: http://eol.ovh.org/winexe/ > > I am looking into the XP Home issue, and some suggest that you can change > permissions a

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-31 Thread Jacob Hydeman
-Original Message- From: Jeffrey J. Kosowsky [mailto:backu...@kosowsky.org] Sent: Saturday, August 29, 2009 7:40 PM To: General list for user discussion, questions and support Subject: Re: [BackupPC-users] BackupPC File::RsyncP issues Jacob Hydeman wrote at about 18:28:54 -0700 on

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-29 Thread Jeffrey J. Kosowsky
Jacob Hydeman wrote at about 18:28:54 -0700 on Thursday, August 20, 2009: > > > Jacob, could you make your specific rsync.exe + cygwin.dll available for > > testing? I have no experience compiling things in Windows and would > > most likely screw it up somehow. > > Here is a zip of the f

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-28 Thread Jeffrey J. Kosowsky
Michael Stowe wrote at about 14:48:16 -0500 on Thursday, August 20, 2009: > Sort of -- VSS/cygwin has a weird catch-22 where the shadow copy has to > exist before cygwin is launched or cygwin can't see the volume at all. In > other words, ssh can start a shadow copy, but then any launched rsync

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-28 Thread Jeffrey J. Kosowsky
Jim Leonard wrote at about 19:45:57 -0500 on Tuesday, August 18, 2009: > Holger Parplies wrote: > > first of all, where are you seeing these figures, and what are you > measuring? > > Rather than try to convince you of my competence, I will offer up these > benchmarks for the exact same e

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-28 Thread Michael Stowe
After downloading this and trying it out, I noticed that these errors went away: Remote[1]: rsync: readlink "..." (in C) failed: File name too long (91) So whatever version of rsync/cygwin that is, is well worth running for that reason alone. With that, I now have a complete copy of 100% of the

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jacob Hydeman
>> Please keep in mind that simple file sharing has to be turned off... a snag >> which cost me a lot in trouble shooting (not a problem if you're on a >> domain, this client is 80+ units on peer to peer) > I didn't know about this limitation, but thankfully have it off on all > of my machines si

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jim Leonard
Jacob Hydeman wrote: >> Jacob, could you make your specific rsync.exe + cygwin.dll available for >> testing? I have no experience compiling things in Windows and would >> most likely screw it up somehow. > > Here is a zip of the folder I use for each XP host: > http://www.itfresno.com/exe/backu

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jacob Hydeman
> Jacob, could you make your specific rsync.exe + cygwin.dll available for > testing? I have no experience compiling things in Windows and would > most likely screw it up somehow. Here is a zip of the folder I use for each XP host: http://www.itfresno.com/exe/backuppc.zip Minus the secrets fi

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jim Leonard
Les Mikesell wrote: > This probably isn't quite what you want but you can configure 4 different > hosts, > each with one of the shares, and then use the $Conf{ClientNameAlias} option > to > point three of them back to the real host. Normally you'd use this only if > you > wanted different sc

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Michael Stowe
Sort of -- VSS/cygwin has a weird catch-22 where the shadow copy has to exist before cygwin is launched or cygwin can't see the volume at all. In other words, ssh can start a shadow copy, but then any launched rsync can't see it, which makes it pointless. The first thing I tried was using remote

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Les Mikesell
Michael Stowe wrote: > It's faster, though I didn't do formal benchmarks. My own reason for > using it is to handle open files, documented below: > > http://www.goodjobsucking.com/?p=62 Is there some reason that wouldn't work as well using ssh for remote execution instead of winexe? Since it r

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Michael Stowe
It's faster, though I didn't do formal benchmarks. My own reason for using it is to handle open files, documented below: http://www.goodjobsucking.com/?p=62 > Jacob Hydeman wrote: >> I'm using winexe to launch it on the windows hosts. > > Thanks - I had wondered if that would work - and now I w

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Les Mikesell
Jacob Hydeman wrote: > I'm using winexe to launch it on the windows hosts. Thanks - I had wondered if that would work - and now I wonder how it compares speed-wise to sshd. -- Les Mikesell lesmiks...@gmail.com ---

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jacob Hydeman
Yes sir. -Original Message- From: Michael Stowe [mailto:mst...@chicago.us.mensa.org] Sent: Thursday, August 20, 2009 11:30 AM To: General list for user discussion,questions and support Subject: Re: [BackupPC-users] BackupPC File::RsyncP issues Out of morbid curiosity, are you doing so

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Michael Stowe
Out of morbid curiosity, are you doing so to use VSS services? > I'm using winexe to launch it on the windows hosts. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jacob Hydeman
I'm using winexe to launch it on the windows hosts. -Original Message- From: Les Mikesell [mailto:lesmikes...@gmail.com] Sent: Thursday, August 20, 2009 11:11 AM To: General list for user discussion,questions and support Subject: Re: [BackupPC-users] BackupPC File::RsyncP issues

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Les Mikesell
Jacob Hydeman wrote: > I only recompiled the cygwin dll. I can't remember the source of rsync that > I'm using since I tried many different ones... noting that none made a > difference in regards to speed. I also tried the packages here: > http://www.itefix.no/i2/node/11649 > > But they had the sa

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jacob Hydeman
Original Message- From: Les Mikesell [mailto:lesmikes...@gmail.com] Sent: Thursday, August 20, 2009 9:07 AM To: General list for user discussion,questions and support Subject: Re: [BackupPC-users] BackupPC File::RsyncP issues Jacob Hydeman wrote: > I had the same issue with backups running

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Les Mikesell
Jacob Hydeman wrote: > I had the same issue with backups running slowly over rsync from windows > hosts. The problem started after I upgraded cygwin to 1.7 in order to > support long file names. On each workstation I'd been getting 80% > utilization of a 100Mbit network card. After cygwin 1.7 I was

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-20 Thread Jacob Hydeman
org] Sent: Wednesday, August 19, 2009 2:14 PM To: General list for user discussion,questions and support Subject: Re: [BackupPC-users] BackupPC File::RsyncP issues I just gave this a shot, and although it was slightly faster, it was not significant enough to fall outside a margin of error (about two minu

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Les Mikesell
Jim Leonard wrote: > Les Mikesell wrote: >> performance on windows is just expected, and compensated for by running >> more instances concurrently if the server is waiting. > > Wait, I can run multiple backups concurrently? That would solve > problems; I've gotten nearly my full throughput by

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Jim Leonard
Les Mikesell wrote: > performance on windows is just expected, and compensated for by running > more instances concurrently if the server is waiting. Wait, I can run multiple backups concurrently? That would solve problems; I've gotten nearly my full throughput by running four rsyncs simulta

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Craig Barratt
Jim writes: > I initially used cygwin rsync; for the above test, I switched it out for > DeltaCopy's rsync. BOTH VERSIONS had this kind of crappy speed. Both > versions showed hardly any CPU or filesystem usage; they just simply run > slowly for a reason I can't figure out. The network isn't sl

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Michael Stowe
I just gave this a shot, and although it was slightly faster, it was not significant enough to fall outside a margin of error (about two minutes faster on a 40 minute backup.) If I were to speculate about a "culprit" for rsync's poor performance relevant to smb, I'd suggest that it's the many lay

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Les Mikesell
Tino Schwarze wrote: > On Wed, Aug 19, 2009 at 10:58:19AM -0500, Jim Leonard wrote: >> Tino Schwarze wrote: >>> I'd rule out the network. Samba might be doing fancy things to the TCP >>> level etc. Or you might try establishing an ssh tunnel to the Windows >>> host (or from Windows host to BackupPC

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Tino Schwarze
On Wed, Aug 19, 2009 at 10:58:19AM -0500, Jim Leonard wrote: > Tino Schwarze wrote: > > I'd rule out the network. Samba might be doing fancy things to the TCP > > level etc. Or you might try establishing an ssh tunnel to the Windows > > host (or from Windows host to BackupPC server using putty whic

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Les Mikesell
Craig Barratt wrote: > >> I believe Craig is researching other alternatives (a fuse FS to handle >> compression and deduplication, so BackupPC could, in fact, use native rsync). >> If that proves unviable, upgrading File::RsyncP to protocol version 30 would >> probably be next. But File::RsyncP is

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Craig Barratt
Holger writes: > I believe Craig is researching other alternatives (a fuse FS to handle > compression and deduplication, so BackupPC could, in fact, use native rsync). > If that proves unviable, upgrading File::RsyncP to protocol version 30 would > probably be next. But File::RsyncP is open source

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Jim Leonard
Tino Schwarze wrote: > I'd rule out the network. Samba might be doing fancy things to the TCP > level etc. Or you might try establishing an ssh tunnel to the Windows > host (or from Windows host to BackupPC server using putty which might be > easier), then point rsyncd to the local end of the tunne

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Tino Schwarze
On Wed, Aug 19, 2009 at 08:46:50AM -0500, Jim Leonard wrote: > > I would take a look at a network traffic dump - maybe something is bad > > there? More suspects: Windows firewall, some other firewall inbeteween? > > Did you try Windows rsyncd -> Windows rsync (to rule out some strange > > Linux vs

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Jim Leonard
Tino Schwarze wrote: > I would take a look at a network traffic dump - maybe something is bad > there? More suspects: Windows firewall, some other firewall inbeteween? > Did you try Windows rsyncd -> Windows rsync (to rule out some strange > Linux vs. Windows network stack issue)? It's not the net

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Jim Leonard
Les Mikesell wrote: > The usual killer with windows targets is an on-access virus scanner, but that > doesn't explain the SMB speed unless there is some kind of exception there. My test files weren't binaries or .zip files or anything that needed to be scanned. What are all of you seeing with r

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Les Mikesell
Tino Schwarze wrote: > >> > first of all, where are you seeing these figures, and what are you >>> measuring? >> Rather than try to convince you of my competence, I will offer up these >> benchmarks for the exact same endpoint machines and file (a 2 gigabyte >> uncompressable *.avi file that

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-19 Thread Tino Schwarze
Hi Jim, On Tue, Aug 18, 2009 at 07:45:57PM -0500, Jim Leonard wrote: > > first of all, where are you seeing these figures, and what are you > > measuring? > > Rather than try to convince you of my competence, I will offer up these > benchmarks for the exact same endpoint machines and file (a

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-18 Thread Holger Parplies
Hi, Jim Leonard wrote on 2009-08-18 17:00:05 -0500 [[BackupPC-users] BackupPC File::RsyncP issues]: > First off, I'm a happy user of BackupPC; I'm only posting because I have > an architecture question resulting in bad performance that I'm hoping > someone can answer. &

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-18 Thread Jim Leonard
Holger Parplies wrote: > first of all, where are you seeing these figures, and what are you measuring? Rather than try to convince you of my competence, I will offer up these benchmarks for the exact same endpoint machines and file (a 2 gigabyte uncompressable *.avi file that did NOT exist on

Re: [BackupPC-users] BackupPC File::RsyncP issues

2009-08-18 Thread Jim Leonard
Jim Leonard wrote: > As you can see, pollChild is called a ridiculously large number of > times, which is eating up nearly 70% of the CPU time trying to do a > backup. This is extremely inefficient and completely explains why my > backups are taking so long over rsync (the CPU spends most of it

[BackupPC-users] BackupPC File::RsyncP issues

2009-08-18 Thread Jim Leonard
First off, I'm a happy user of BackupPC; I'm only posting because I have an architecture question resulting in bad performance that I'm hoping someone can answer. I have a need to back up Windows clients. I got smb-based backups working very well, or so I thought -- no matter what I tried, I c