rsync backups are quite slow. over slow links this is more than made up for
by the decrease in network bandwidth used but on a high speed lan it is very
slow. I am on gigabit from my desktop to my backuppc server but rarely
exceed 6-8MB/s during a backup.
That makes sense some of the time as rsync will only be transfering checksum
data but when there is a large file being transfered it is still quite
slow. install iftop and watch a backup that has a new large file that is
not in an exsisting backup and you will not see the transfer speed at
anywhere near the LAN speed or hard disk speed because of I/O bottlenecks.
It would be nice to be able to use WindowsXP+'s index as a reference or have
the rsyncd on the windows machine maintain all the checksum information but
I know of no way and no plans to do this.
I/O is an obvious slow point here but rsync doesn't seem to be aware of I/O
bottlenecks. it seems to scan files at random rather than trying to scan
files in the on-disk-order on the client side and then cache disk writes on
the server side to overcome the I/O bottleneck. Another, possibly the
primary factor, in the I/O bottleneck is the way almost all modern
filesystems handle small file creation, which includes all files smaller
than the cluster size but also directories, softlinks, and hardlinks.
The arguement of storing files in a database has come up as a database is
better able to overcome the I/O deficites of the disk storage by some
intelegent caching but that would require more complex code. Alternatively
using a fuse-based SQL filesystem would help overcome the I/O issue by
taking advantage of MySQL's caching but fuse modules are notoriously
innefficient and the overhead would negate the benefit.
One functional solution is to use a filesystem that has the caching benefits
without any extra complexity. ZFS on BSD and Solaris fits the bill and
performs very very well with backuppc. Unfortunately, none of the BSDs,
Solaris, or openSolaris include backuppc and filling dependancies is just
beyond many users as a large number of backuppc users/admins are all about
Linux. In fact, ZFS can have cache disks in the storage pools and pre-write
data sequentially from the RAM cache for re-organization later, making ZFS
very effective at hiding the I/O bottleneck.
On Wed, May 21, 2008 at 4:19 AM, Kurt Jasper <[EMAIL PROTECTED]>
wrote:
> Ok,
>
> thanks for (all) the information, now I understand the concept of
> Rsync/SambaShareName, BackupFilesExlude and BackupFilesOnly.
> You really helped me out, Thanks :-)
>
> But unfortunately I've discovered another problem:
>
> The backups of my windows xp clients take forever (nearly), while the
> backup of the server (where backuppc is runnning) works fine. I've done
> some testing and figured out that it is NOT an network issue, since I
> get the full speed when executing rsync from the windows xp client.
>
> I have transferred the cygwin directory of one of the clients (approx.
> 300 MB):
>
> On Windows-Client:
> rsync -av -h --progress /cygdrive/c/cygwin [EMAIL PROTECTED]:~/
> => works fine!
>
> On Server:
> rsync -av -h --progress [EMAIL PROTECTED]:/cygdrive/c/cygwin ~/
> => takes very, verrrry long.
>
> Browsing the web, I've found out that it may have something to do with
> the Cygwin Enviroment Variables. I'm currently using the suggested
> settings: CYGWIN = ntsec tty.
>
> Question:
> 1) So what may be the problem?
> What is the best way to back up windows xp clients? I don't understand
> that running rsync from CLIENT-to-SERVER runs fine, but running from
> SERVER-to-CLIENT is really slow.
> 2) Is is better to use Samba to backup windows clients?
> Some clients are running XP (Home), so that I can not set custom file
> permissions to allow access only the backuppc-user not to other clients
> in the network.
> 3) Will rsyncd instead of rsync bring better performance?
>
> Kind regards
>
> - Kurt
>
> PS: To give at least something back to this community, I thought about
> writing down my experiences (from a beginners point of view), I hope
> this is the right place:
> http://backuppc.wiki.sourceforge.net/CommonConfigurationIssues
> If not feel free to flame me :-0
>
> Bowie Bailey wrote:
>
> > Your RsyncShareName is '/', but your are specifying exclusions for the
> > 'c' share. Try this instead:
> >
> > $Conf{BackupFilesExclude} = {
> > '/' => ['/Dokumente und Einstellungen/*/NTUSER.DAT', ....
> > };
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> BackupPC-users mailing list
> [email protected]
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
BackupPC-users mailing list
[email protected]
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
