On Sun, Oct 24, 2021 at 10:48 AM Helmut Jarausch <jarau...@skynet.be> wrote: > > Finally I have come up with the following shell script for backing up > my /home directory, > comments are more than welcome, of course. > (/HBackUp contains a BTRFS file system) >
Is /home on btrfs? If not then something like this is probably your best bet (not sure if existing tools work, and I didn't carefully check your script for any missed error handling). If /home is on btrfs, just on a different filesystem, then you're probably much better off using something based on send/receive. This can let you very efficiently transfer snapshots incrementally and preserve COW between the copies (that is, unchanged data is deduplicated). While rsync is very efficient for network transfer or amount of data written, in order to detect changes it has to read all the inodes in both copies of the data. If you don't trust mtime in either copy then you have to further read all the data itself. That is a lot of read IO and CPU on both ends, though if they're on different hosts the network traffic is very efficient. Btrfs send/receive can determine what has changed between two snapshots very efficiently, as snapshots are b-trees that are cross-linked where there are no changes, so you only have to descend and then transfer branches that actually contain changes (similar to git). If only one file changes between snapshots the number of reads to find it scales logarithmically with the amount of metadata, while rsync scales linearly as it has to read all of it. Also, with btrfs the incremental change set can be prepared without any reference to the target (since both copies are on the source), so latency isn't a factor at all. You can just dump the incremental backups to serial files if you prefer, though that requires keeping them all as with normal incremental backups. If you receive them into a btrfs filesystem then you only need to retain one snapshot in common between the source and target to allow future incrementals. I think there are tools out there which already implement this for btrfs. I know such tools exist for zfs and I use one myself. I never used this with btrfs as it was immature at the time I switched over. -- Rich
