Re: [BackupPC-users] speed up backups
Ralf Gross schrieb:
Les Mikesell schrieb:
On 5/26/2010 3:41 PM, Ralf Gross wrote:
Ralf Gross schrieb:
write(1, N\2\0\7\5\3lvs\r\0\0\0\r\0\0\0lvmiopversion8\5..., 594) = 594
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}
smells like a time out, but I don't know where. I found a couple of
messages
with similar output in the list archives, but none of them had a
solution yet.
*grr*
I only traced the Xfer PID, not the PID. BackupPC_dump seems to be
active and comparing the file list with the pool and I see high cpu
load.
I'm sure that I haven't seen that as I abortet the backup before.
Now I'll have will wait until tomorrow morning...
Until the 2nd full completes, the server side has to uncompress the
stored copy to compute the checkums on existing files. And there may be
some quirk about switching from tar to rsync that I've forgotten. Maybe
the 1st run will add the checksum cache for files you already have.
The full rsync is still running sind 5/26 21:00. I'll report back when
it's done.
Ok, the first rsync full backup (488) completed. It took 500min. longer than
the last tar full backup (482).
Backup TypeFilled Level Start Date Duration/mins Age/days
482 fullyes 0 5/19 02:05 3223.2 11.5
483 incrno 1 5/21 07:4989.6 9.2
484 incrno 2 5/22 03:05 136.4 8.4
485 incrno 3 5/23 03:05 119.1 7.4
486 incrno 4 5/24 03:05 111.4 6.4
487 incrno 1 5/25 03:05 165.9 5.4
488 fullyes 0 5/26 21:00 3744.2 3.7
489 incrno 1 5/29 12:15 394.1 1.1
490 incrno 2 5/30 03:05 190.8 0.4
I'm not sure if the checksum caching will compensate this in after the 3rd
backup. Anything else I could do to tune rsync?
Ralf
--
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] speed up backups
Ralf Gross wrote:
Ralf Gross schrieb:
Les Mikesell schrieb:
On 5/26/2010 3:41 PM, Ralf Gross wrote:
Ralf Gross schrieb:
write(1, N\2\0\7\5\3lvs\r\0\0\0\r\0\0\0lvmiopversion8\5..., 594) = 594
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
select(1, [0], [], NULL, {60, 0}
smells like a time out, but I don't know where. I found a couple of
messages
with similar output in the list archives, but none of them had a solution
yet.
*grr*
I only traced the Xfer PID, not the PID. BackupPC_dump seems to be
active and comparing the file list with the pool and I see high cpu
load.
I'm sure that I haven't seen that as I abortet the backup before.
Now I'll have will wait until tomorrow morning...
Until the 2nd full completes, the server side has to uncompress the
stored copy to compute the checkums on existing files. And there may be
some quirk about switching from tar to rsync that I've forgotten. Maybe
the 1st run will add the checksum cache for files you already have.
The full rsync is still running sind 5/26 21:00. I'll report back when
it's done.
Ok, the first rsync full backup (488) completed. It took 500min. longer than
the last tar full backup (482).
BackupTypeFilled Level Start Date Duration/mins Age/days
482 fullyes 0 5/19 02:05 3223.2 11.5
483 incrno 1 5/21 07:4989.6 9.2
484 incrno 2 5/22 03:05 136.4 8.4
485 incrno 3 5/23 03:05 119.1 7.4
486 incrno 4 5/24 03:05 111.4 6.4
487 incrno 1 5/25 03:05 165.9 5.4
488 fullyes 0 5/26 21:00 3744.2 3.7
489 incrno 1 5/29 12:15 394.1 1.1
490 incrno 2 5/30 03:05 190.8 0.4
I'm not sure if the checksum caching will compensate this in after the 3rd
backup. Anything else I could do to tune rsync?
You could force a full to start on Friday evening so weekly scheduling will
keep
the full runs on weekends if they take more than a night to complete.
Depending
on how much daily change you have, you might want to set incremental levels for
the intermediate runs.
A more extreme change would be to edit Rsync.pm to not add the --ignore-times
option on fulls. I haven't needed this myself yet but I think it would make a
big difference in speed - at the expense of not checking files for unlikely but
possible differences.
--
Les Mikesell
lesmikes...@gmail.com
--
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] speed up backups
On 27/05/10 12:17, Sorin Srbu wrote: -Original Message- From: Tyler J. Wagner [mailto:ty...@tolaris.com] Sent: Thursday, May 27, 2010 1:00 PM To: backuppc-users@lists.sourceforge.net; sorin.s...@orgfarm.uu.se Subject: Re: [BackupPC-users] speed up backups On Thursday 27 May 2010 11:13:29 Sorin Srbu wrote: Looked into file system optimization again, while waiting for a backup to finish. Seems like noatime is a recommended setting in /etc/fstab. If I already have defaults set for the backup-array mount, and add noatime, should I still keep defaults? Or is it implied that the defaults are not to be used anymore and therefore one can omit it in /etc/fstab? My file system-fu is not that strong... 8-/ Replace defaults with noatime or better, noatime,nodiratime. Then just remount, you don't need to reboot: mount -o remount / Replace / with the appropriate filesystem. I have a separate /var and /var/local for backuppc. Thanks. Just what I suspected. Nodiratime is new to me though. I can't say I've come upon it before. I'll try it at home first. 8-) Will using ext2 instead of ext3 speed up backuppc? cheers Chris -- Chris Dennis cgden...@btinternet.com Fordingbridge, Hampshire, UK -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] speed up backups
It's generally slower, so I'm going to go with no on this one. If filesystem is a bottleneck, you'd be better off with xfs or jfs. Will using ext2 instead of ext3 speed up backuppc? cheers Chris -- Chris Dennis cgden...@btinternet.com Fordingbridge, Hampshire, UK -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
