Hello, I've more then 6000 MD5 hashes, reported by BackupPC_refCountUpdate. I startet to create empty files with the hash, reported by BackupPC_refCountUpdate as missing. But than it will be reported as poolfile with wrong hash. BackupPC is stubborn🤣️.
Currently I'm working on a Perl programm: - reading all this error hashes from the LOG files - compare it with the global ../[c]pool/*/poolCnt references - searching the remaining hashes in the backups ../pc/<host>/<bckNr>/refCnt/poolCnt.*.* - search the hashes in the attrib- and index-files of the affected backups - print a list of files, affected by thus corrupt pool files - offer to cut this file from the attrib-file or delete the whole backup I expect to be ready and tested till end of september and I'll send it here into the mailing list. Br Matthias Am Montag, dem 03.08.2026 um 14:11 -0400 schrieb Ronald Colcernian: > Hi Ged, > > Sorry it took a while to get back to you. > I spend an amount of time running > BackuPC_refCountUpdate -h <host> -c -f -o 3 > on all of my backups as you pointed out in a previous email, they were > batched in 11 backups that > were very old and deletable. > > So now I'd like to clean up this problem. Here is one example of an actual > message: > 2026-08-03 01:22:41 admin : BackupPC_refCountUpdate: ERROR pool file > /bkpcdata/new_BackupPC_Data/cpool/1c/56/1d5621373a63a74a8872a46f02b40850 > has digest d41d8cd98f00b204e9800998ecf8427e > instead of 1d5621373a63a74a8872a46f02b40850 > Note that digest d41d8.....427e is the digest of an empty file. > > I have about 48 of these messages from the last three days. > I have researched each of these and they all point to a zero sized file. > > Understanding that, I ran the command find cpool -type f -size 0 | wc -l > and there are 798 of these zero sized files. > > I have noticed that the nightly while it complains about these cpool file(s), > it does not remedy > it. > > I've spend a while reading backuppc-users email list, and have not really got > a good answer, or > course of action. > > Questions: > 1) Can I remediate by deleting those files, would be easy with a short bash > script. > > 2) Is there a way to take the "has digest number" and find the backup it came > from? > and then delete that backup, perhaps > > 3) I could with a bash script change the instead of digest to the zero file > size digest. > > What is your best advice? > > Thank you in advance. > > Ron Colcernian > > > > An important note in my opinion these have accumulated to this extend as a > result of > years of running BackupPC, hard drive failures, rsync TopDir to a new > machine (that was brutal > BTW), conversion V3 to V4, and gross neglect. > > > > I ran find cpool > > > On Thu, Jul 23, 2026 at 6:41 PM G.W. Haywood <[email protected]> > wrote: > > Hi there, > > > > Long day today, sorry. > > > > On Thu, 23 Jul 2026, Ronald Colcernian wrote: > > > > > I have 1/2 dozen of these errors in the main BackupPC Log file. > > > > > > Run: I found the following recommendations: > > > > > > BackupPC_refCountUpdate -m -f -c > > > > > > *Questions:* > > > 1) Is this the proper tool to address this type of error? > > > > Not necessarily. I'd want to know *why* this has happened. This tool > > won't tell you that. When the problem is fixed, I'd want to know that > > it has stopped happening. It's a good tool for that. > > > > The errors are presumably generated and logged by the nightly checks. > > If you have $Conf{PoolNightlyDigestCheckPercent} at the default 1% per > > nightly run, each nightly run checks a random 1% of the pool files. > > If that's the case and you have half a dozen of the errors it could be > > that there are a lot more waiting to be discovered. I'd be more than > > a little uncomfortable with that. Have you done checks on the storage > > medium/filesystem(s)? I'd be wondering about their reliability. On > > the systems here I'm almost sure I've never seen this error message. > > I grepped the last seven years of logs on my busiest backup machine > > and found none. > > > > > if not what would be the better tool? > > > > You could change the 1% setting (if it is still at the default) to > > check files more quickly, or try the tool posted in the Wiki here: > > > > https://github.com/guille2306/backuppc_check_pool > > > > it will check all the pool if you want to. (Strictly speaking the cpool.) > > > > > 2) How long will it take using the tool? > > > I understand this is a function of the speed of the machine and disk > > > subsystem. > > > > Hmmmm... perhaps more to the point it will depend on how many pool > > files there are and how large they are. Very difficult to say, but > > you can at least get an idea using the 'progress' option. Probably > > with a 3TByte pool it won't be like greased lightning on *anything*. > > You can stop a test at any time with CTRL-C, it won't break things. > > That means you can try a test with one set of options and, if looks > > like it's going to take too long for you stop it and try again with > > different options. > > > > > I prefer that BackupPC is not shutdown for days. > > > > :) > > > > I'd probably try doing a chunk of checks each day, adjusting 'chunk' > > to suit the hardware performance and your waking hours. :/ > > > > > Can the tool be run by host? > > > > Yes. If you run it without any options it prints a help page: > > > > 8<-------------------------------------------------------------------------- > > > > $ /usr/local/BackupPC/bin/BackupPC_refCountUpdate > > Usage: > > BackupPC_refCountUpdate -h HOST [-c] [-f] [-F] [-o N] [-p] [-v] > > With no other args, updates count db on backups with poolCntDelta > > files > > and computes the host's total reference counts. Also builds refCnt > > for > > any >=4.0 backups without refCnts. > > -f - do an fsck on this HOST, which involves a rebuild of the > > last two backup refCnts. poolCntDelta files are ignored. > > Also forces fsck if requested by needFsck flag files > > in TopDir/pc/HOST/refCnt. Equivalent to -o 2. > > -F - rebuild all the >=4.0 per-backup refCnt files for this > > host. Equivalent to -o 3. > > -c - compare current count db to new db before replacing > > -o N - override $Conf{RefCntFsck}. > > -p - don't show progress > > -v - verbose > > Notes: in case there are legacy (ie: <=4.0.0alpha3) unapplied > > poolCntDelta > > files in TopDir/pc/HOST/refCnt then the -f flag is turned on. > > > > BackupPC_refCountUpdate -m [-f] [-p] [-c] [-r N-M] [-s] [-v] [-P phase] > > -m Updates main count db, based on each HOST > > -f - do an fsck on all the hosts, ignoring poolCntDelta files, > > and replacing each host's count db. Will wait for backups > > to finish if any are running. > > -F - rebuild all the >=4.0 per-backup refCnt files. > > -p - don't show progress > > -c - clean pool files > > -r N-M - process a subset of the main count db, 0 <= N <= M <= 255 > > -s - prints stats > > -v - verbose > > -P phase Phase from 0..15 each time we run BackupPC_nightly. Used > > to compute exact pool size for portions of the pool based > > on the phase and $Conf{PoolSizeNightlyUpdatePeriod}. > > > > 8<-------------------------------------------------------------------------- > > > > HTH > > > > _______________________________________________ > > BackupPC-users mailing list > > [email protected] > > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki: https://github.com/backuppc/backuppc/wiki > > Project: https://backuppc.github.io/backuppc/ _______________________________________________ BackupPC-users mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
