Re: [BackupPC-users] Imnproving backup speed
Hi all,
On 1/7/21 8:56 PM, Michael Stowe wrote:
On 2021-01-07 00:39, Sorin Srbu wrote:
Hello all!
Trying to improve the backup speed with BPC and looked into setting
noatime in fstab. [...]
What will BPC in particular do if noatime is set?
In short, it depends on your transport methods. Rsync will be fine.
Tar/smb? Not so much for incrementals, but fulls (of course) will be fine.
On the *client* side that's a more interesting question. As mentioned
before in this thread and as documented in the BackupPC docs, the
*server* side is okay with noatime as far as the BackupPC pool is concerned.
For clients, I have no idea about smb, but a vague one about tar.
Disclaimer: I don't use tar myself.
The GNU tar manpage is fairly explicit about atime issues, but the
subject is distributed in different sections:
In
https://www.gnu.org/software/tar/manual/html_section/tar_22.html#SEC42,
it recommends --atime-preserve=system for incremental backups, if
supported by the OS. This gently asks the system for not modifying
atimes; essentially a noatime request for an individual read.
In https://www.gnu.org/software/tar/manual/html_section/tar_69.html
however, the --atime-preserve=replace (which happens to be the default
variant for --atime-preserve) is documented to *not* play nicely with
incremental backups. This one resets the file stats after reading, but
the reset itself counts as metadata change, and accordingly the file
will be re-read on the next run. I'm not 100% sure where the timestamp
of the change is recorded, though. Also, in my tests, I could not
confirm that --atime-preserve=replace causes issues with incrementals;
but that's based on rather ad-hoc manual tests, not via BackupPC.
In any case, tar recommends an entirely different approach for
incremental dumps
(https://www.gnu.org/software/tar/manual/html_section/tar_39.html#SEC96), but
this is not feasible with BackupPC.
Also, note that --newer does not consider atime, but only ctime and
mtime
(https://www.gnu.org/software/tar/manual/html_section/tar_52.html#SEC116).
BackupPC by default uses --newer (aka --after-date) on incremental dumps
(see $Conf{TarIncrArgs} around
https://github.com/backuppc/backuppc/blob/master/conf/config.pl#L1136),
and accordingly has a warning in the config about not using
--atime-preserve[=replace]. However, it is questionable whether the same
warning also applies for --atime-preserve=system (assuming it's
supported on the client).
So from what I understand, the combination of --newer and
--atime-preserve=system (or noatime mounts) should be almost optimal.
And --newer-mtime + noatime should work, too, but with the usual
downside of --newer-mtime that metadata updates are not caught (e.g.,
permission changes).
Of course, that's assuming that there no other issues with using noatime
unrelated to the backups.
P.S.: https://backuppc.github.io/backuppc/BackupPC.html#Backup-basics is
not 100% accurate, claiming that incremental backups with tar rely on
mtime only; actually, the use of --newer implies that mtime *and* ctime
are relevant.
Cheers,
Alex
smime.p7s
Description: S/MIME Cryptographic Signature
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Imnproving backup speed
On Thu, 2021-01-07 at 20:30 +0100, Alexander Kobel wrote: > Hi Sorin, > > On 1/7/21 9:39 AM, Sorin Srbu wrote: > > Hello all! > > > > Trying to improve the backup speed with BPC and looked into setting noatime > > in fstab. > > > > But this article states some backup programs may bork if noatime is set. > > https://lonesysadmin.net/2013/12/08/gain-30-linux-disk-performance-noatime-nodiratime-relatime/ > > > > What will BPC in particular do if noatime is set? > > exactly what it's supposed to do. noatime or at least relatime (or > perhaps recently lazytime) is the recommended setting: > https://backuppc.github.io/backuppc/BackupPC.html#Optimizations A _general_, although related, question: Would tuned do me some good? I see that tuned-adm recommends using the profile virtual-guest, but would maybe performance-throughput be any better? Do any of you using BackupPC tweak your BPC-servers anything at all with tuned? -- Kind regards, Sorin Srbu Find my OpenPGP public key here: https://cloud.srbu.se/index.php/s/KeEsCCDsG7PZG7N signature.asc Description: This is a digitally signed message part ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Imnproving backup speed
On Fri, 2021-01-08 at 12:07 +1100, Adam Goryachev via BackupPC-users wrote: > On 8/1/21 06:30, Alexander Kobel wrote: > > On 1/7/21 9:39 AM, Sorin Srbu wrote: > > > What will BPC in particular do if noatime is set? > > > > exactly what it's supposed to do. noatime or at least relatime (or > > perhaps recently lazytime) is the recommended setting: > > https://backuppc.github.io/backuppc/BackupPC.html#Optimizations > > I think it depends on whether you are applying this setting change on > the BPC server, and specifically the BPC pool drive, or if you are > applying it to the clients and/or root FS of the BPC server. > > If you have a separate filesystem for the BPC pool, then using this > setting on that filesystem will not have any adverse impact, but will > likely reduce overhead. Changing this setting elsewhere will have the > documented impacts (and you would need to assess the results of those > impacts based on your own personal requirements (or provide a lot more > information for anyone else to comment on). The BPC server uses a separate drive for the backup pool, so is pretty much a schoolbook example. I went ahead and set relatime for the pool-drive. Thanks all for the hints! -- Kind regards, Sorin Srbu Find my OpenPGP public key here: https://cloud.srbu.se/index.php/s/KeEsCCDsG7PZG7N signature.asc Description: This is a digitally signed message part ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Imnproving backup speed
On 8/1/21 06:30, Alexander Kobel wrote: Hi Sorin, On 1/7/21 9:39 AM, Sorin Srbu wrote: Hello all! Trying to improve the backup speed with BPC and looked into setting noatime in fstab. But this article states some backup programs may bork if noatime is set. https://lonesysadmin.net/2013/12/08/gain-30-linux-disk-performance-noatime-nodiratime-relatime/ What will BPC in particular do if noatime is set? exactly what it's supposed to do. noatime or at least relatime (or perhaps recently lazytime) is the recommended setting: https://backuppc.github.io/backuppc/BackupPC.html#Optimizations I think it depends on whether you are applying this setting change on the BPC server, and specifically the BPC pool drive, or if you are applying it to the clients and/or root FS of the BPC server. If you have a separate filesystem for the BPC pool, then using this setting on that filesystem will not have any adverse impact, but will likely reduce overhead. Changing this setting elsewhere will have the documented impacts (and you would need to assess the results of those impacts based on your own personal requirements (or provide a lot more information for anyone else to comment on). Regards, Adam ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Imnproving backup speed
Hi Sorin, On 1/7/21 9:39 AM, Sorin Srbu wrote: Hello all! Trying to improve the backup speed with BPC and looked into setting noatime in fstab. But this article states some backup programs may bork if noatime is set. https://lonesysadmin.net/2013/12/08/gain-30-linux-disk-performance-noatime-nodiratime-relatime/ What will BPC in particular do if noatime is set? exactly what it's supposed to do. noatime or at least relatime (or perhaps recently lazytime) is the recommended setting: https://backuppc.github.io/backuppc/BackupPC.html#Optimizations Cheers, Alex smime.p7s Description: S/MIME Cryptographic Signature ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Imnproving backup speed
On 2021-01-07 00:39, Sorin Srbu wrote: Hello all! Trying to improve the backup speed with BPC and looked into setting noatime in fstab. But this article states some backup programs may bork if noatime is set. https://lonesysadmin.net/2013/12/08/gain-30-linux-disk-performance-noatime-nodiratime-relatime/ What will BPC in particular do if noatime is set? In short, it depends on your transport methods. Rsync will be fine. Tar/smb? Not so much for incrementals, but fulls (of course) will be fine. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
[BackupPC-users] Imnproving backup speed
Hello all! Trying to improve the backup speed with BPC and looked into setting noatime in fstab. But this article states some backup programs may bork if noatime is set. https://lonesysadmin.net/2013/12/08/gain-30-linux-disk-performance-noatime-nodiratime-relatime/ What will BPC in particular do if noatime is set? -- Kind regards, Sorin Srbu Find my OpenPGP public key here: https://cloud.srbu.se/index.php/s/KeEsCCDsG7PZG7N signature.asc Description: This is a digitally signed message part ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
