Hi list. Until recently I was using rdiff-backup for backing up our servers. But after a certain point, the rdiff-backup process would take days, use up huge amounts of CPU and RAM, and the debug logs were very opaque. And the rdiff-backup mailing lists weren't very helpful either.
So, for servers where that's a problem, I changed over to a hardlink snapshot-based aproach. That's been working fine for the past few weeks. But now, the backup server is running out of harddrive space and we need to check which backup histories is taking up the most space, so we can prune the oldest week or two from them. The problem now is, that 'du' takes days to run, and now generates 4-5 GB text files. This is of course caused by du having to now process dozens of additional snapshot directories, many of them with a large number of files. What I've been doing, is writing helper scripts to help prune the earlier directories. Something like this: 1) Compare files under snapshot1 & snapshot2. If any files under snapshot1 are hardlinks to the same files under snapshot2, then remove them from snapshot1, and add an entry to a text file (for possible later regeneration). 2) Remove empty directories (and add their details to a text file) 3) 7zip-compress the text files containing recovery info. 4) Possibly later (before empty directories): Remove hardlinks and symlinks from snapshot dirs, and add them to compressed text files. There's also scripts to reverse the above operations. Of course, it takes a while to get a complete snapshot of a given server from a few weeks back, but at least it's possible. I'm thinking that this is a lot of work, and there must be better ways of handling this kind of problem. I don't like reinventing the wheel. Given the above, are there any suggestions? eg: 1) Another tool similar to rdiff-backup, which has easier-to-understand logs. 2) A quicker way of running DU for my use case (huge number of hardlinks in directories) 3) Existing tools for managing hardlink-based snapshot directories etc. Thanks in advance. David. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
