Ahhh, I think I know what‘s going on now. I run Debian stable on all my systems so still on version 3, yes.
It’s been several years since I copied the files over, before the old disks were sent for shredding, so they’ve been sitting there as disabled hosts and I’ve only now got around to finally sorting it out. Reading back over the documentation and searching the list archive, I think what I did was to just copied (preserving hardlinks - I am very familiar with the issues of copying entire, but not partial, BackupPC stores having done this many times, and currently do so monthly for offsite/DR purposes) the pc directories, so although the duplicated files were still hardlinked together (as they came from the same backuppc filesystem) nothing I copied over will be in this system’s pool (this process was suggested on this mailing list circa. 2012). I know I followed someone else’s advice on this for precisely the reasons you describe and this is the first, and I expect will be the only time, I’ve tried to partially or selectively access a subset of data in a BackupPC store. What I was tying to do at the time was resurrect access to these specific hosts, so preserving hardlinks within there pc directories but not adding to the live system pool was sufficient (and within my understanding of the storage structure ;) ). You’re spot on that trying to merge the pools myself is beyond my current understanding of the storage structure, with I don’t “really really understand”. Assuming this is what I did, that makes sense of what I’m seeing now, if I’m getting an approx. 50% hit rate on files already in the pool, on this live BackupPC, from other hosts and their backups - 50% is higher than I would have presumed but still plausible. Having found the old mailing list post suggesting just copying the pc directories (with the caveat that it loses the benefits of pooling for new backups) I’m 90% sure this is what’s going on, I may try and confirm it tomorrow (if that’s right, nothing from these old pc directories will exist in the pool). I’ll have to modify my plan and work through one old host at a time, instead of restoring all the data at once then sorting it. I was expecting (having clearly forgotten how I got these old systems backups into my current BackupPC install) restoring all the old backups to my current desktop system would have negligible impact on the pool’s free space and I could tidy up the defunct pc’s configuration and pc directories before I’d made a significant dent in sorting through the restored files. Hopefully doing it a host at a time then removing that hosts configuration and pc directory will release enough space from files not shared between the old hosts to keep at least one backup copy (old or new) of all of the data as I go, just in case. Thanks for taking the time to reply. I wouldn’t have found the old message and realised the above hypothesis, which makes sense of the situation, without it. Laurence > On 12 Jan 2020, at 22:57, backu...@kosowsky.org wrote: > > Assuming you are using BackupPC 3.x... which is based on hard links... > You can't "just" copy over and merge backups... Rather there are links > and pool/cpool chains and several other complexities. > > I did write and post some routines that can do this by essentially > looking up each file, searching the pool for a match, and then either > replacing the 'pc' file with a hard link to an existing pool file or > creating a new link depending on whether the file already exists in > the pool. There are also several edge cases to be careful of. > This is of course a slow process and not recommended unless > you really really understand the structure of the backup storage and > know what you are doing (which clearly you don't :) > > > Laurence Hurst wrote at about 19:11:02 +0000 on Sunday, January 12, 2020: >> Hi I'm hoping someone can help me figure out what I've done and/or where >> my expectations are wrong. >> >> Bit of background: I've found some old disks that belonged to a BackupPC >> pool, which contained amongst other things copies of some coding >> projects I thought I'd lost the source for many years ago. I've copied >> them into my existing backuppc pool, along with the relevant pc >> directories and configurations, so I can access them. >> >> I've started restoring some of the file to a directory onto my current >> machine, using a direct restore. I was expecting to be able to restore >> the files, rescue what I needed and delete the original 'pc' from >> BackupPC without having much impact on the pool since all of the files >> are already in there, so interim backups of my current machine should >> just create new links to the files I've recovered via BackupPC's restore >> feature. However, when the next backup of my current machine happened >> only about 50% of the files were found as 'existing files', the other >> 50% came out as 'new files' (and now my pool disk is 95% full as nearly >> 200GB of data that should already be in the pool has been duplicated). >> >> This is where I'm confused: Why would files restored directly from >> BackupPC and not subsequently touched in any way (not even viewed or >> moved/renamed) not be found as existing files in the pool? Compression >> was turned on for both pools (the original and the new one). >> >> Any ideas? >> >> Thanks, >> >> Laurence >> >> >> _______________________________________________ >> BackupPC-users mailing list >> BackupPC-users@lists.sourceforge.net >> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users >> Wiki: http://backuppc.wiki.sourceforge.net >> Project: http://backuppc.sourceforge.net/ > > > _______________________________________________ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
