Re: Is there any way to restore/create hardlinks lost in incremental backups?
On Sun, Dec 13, 2020 at 11:59 AM Wayne Davison via rsync < rsync@lists.samba.org> wrote: > I should also mention that there are totally valid reasons why the dir > might be huge on day4. For instance, if someone changed the mode on the > files from 664 to 644 then the files cannot be hard-linked together even if > the file's data is unchanged. The same goes for differences in preserved > xattrs, acls, and ownership. In such a case you could decide that you > don't care about the change in meta info and tweak it on the earlier files > to match day4's files and then the suggested re-link command would decide > it could join them together. You'd probably then need to keep going and > re-link day5's pictures (since it was probably linking to the old day4's > pictures). > > ..wayne.. > I totally get why some folks would prefer to use rsync --link-dest for backups: It's very fast, and the backup itself is usable as a replacement filesystem. If you are open to trying something else though, there are probably several tools at https://stromberg.dnsalias.org/~strombrg/backshift/documentation/comparison/index.html that can backup permissions changes without needing to create a copy of the file data. Sadly, I don't know about most of the tools there, but I know that backshift wouldn't. Backshift is much slower than rsync, but also takes up quite a bit less storage space, even if you mv a large hierarchy or change all the file permissions in a hierarchy. HTH -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Is there any way to restore/create hardlinks lost in incremental backups?
I should also mention that there are totally valid reasons why the dir might be huge on day4. For instance, if someone changed the mode on the files from 664 to 644 then the files cannot be hard-linked together even if the file's data is unchanged. The same goes for differences in preserved xattrs, acls, and ownership. In such a case you could decide that you don't care about the change in meta info and tweak it on the earlier files to match day4's files and then the suggested re-link command would decide it could join them together. You'd probably then need to keep going and re-link day5's pictures (since it was probably linking to the old day4's pictures). ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Is there any way to restore/create hardlinks lost in incremental backups?
You could rsync the current day4 dir to a day4.new dir, and list all the prior days as --link-dest options. Make sure that you're using the same xatt/acl options as your official backup command (the options may or may not be present) so that you are preserving the same level of info as the backup. You also have the choice of copying the whole day4 dir or just the day4/pictures dir, as you see fit. For example: rsync -aiv --link-dest=../day1 --link-dest=../day2 --link-dest=../day3 day4/ day4.new/ mv day4 day4.bad mv day4.new day4 If you only want to reprocess the pictures subdir, just tweak the "day4/" arg to be "day4/pictures" (no trailing slash) and change the mv commands to deal with just that subdir. ..wayne.. On Thu, Dec 10, 2020 at 9:29 AM Chris Green via rsync wrote: > I run a simple self written incremental backup system using rsync's > --link-dest option. > > Occasionally, because I've moved things around or because I've done > something else that breaks things, the hard links aren't created as > they should be and I get a very space consuming backup increment. > > Is there any easy way that one can restore hard links in the *middle* > of a series? For example say I have:- > > day1/pictures > day2/pictures > day3/pictures > day4/pictures > day5/pictures > > and I notice that day4/pictures is using as much space as > day1/pictures but all the others are relatively small, i.e. > day2 day3 and day5 have correctly hard linked to the previous day but > day4 hasn't. > > It needs a tool that can scan day4, check a file is identical with the > one in day3 then hardlink it without losing the link from day5. > > There's jdupes but that does lose the link from day5 so you'd have to > apply it to all the directories after the one that's lost the links. > > > > -- > Chris Green > ยท > > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
