Hello,
I have a very old BackupPC server (coming up to 15 years, now
running 4.4.0). I've been migrating the entire pool to v4, basically
one backup per day (since the migration requires downtime). This has
been going fine for months, but recently, one of the backups exposed
some problems in the cpool that have me worried:
There's this:
BackupPC_refCountUpdate: ERROR pool file
/srv/backuppc/cpool/00/f6/01f7b4d296cfc54f260dbd3ab7f61d0a has digest
d41d8cd98f00b204e9800998ecf8427e instead of 01f7b4d296cfc54f260dbd3ab7f61d0a
and then there are errors like this:
BackupPC_refCountUpdate: missing pool file 58209505159c3b0e9cfd48099a53bc79
count 1
I am less worried about the second one, because in [this
message](https://adsm.org/lists/html/BackupPC-users/2017-03/msg00026.html),
Craig explains that these are reminiscents of a bug that's since
been fixed, and the problems will go away when the backups created
by the buggy software expire. Every run of `nightly` also reports
the exact same set of 6 such missing files.
But the former error is giving me a bit of a worry, and there's [an
open issue on Github](https://github.com/backuppc/backuppc/issues/412).
The way I read the error message is that there's a zero-length file
in the cpool stored under a hash that should indicate that there was
actually content. So has my filesystem truncated these files? It's
an ext4 filesystem, and it fsck's cleanly.
What I find especially unnerving is that there's a new set of these
files being reported on every run of `refCountUpdate -m -c -r0-255`
(without `-P`), without any overlap. This doesn't seem like just a
handful of pool files erroneously put in place due to a bug, but
more like an unknown set of these.
So I ran a `find cpool/?? -size 0` and it found around 1,800 of
them, though most of them have been created on 21 August 2021, with
only a few (ca. 17%) created on 14 Jan 2022. That's a bit of a
relief, and maybe it's not all that bad?
At the end of a refCountUpdate, I am shown this:
BackupPC_refCountUpdate total errors: 6
and those are the missing files (see above). The zero-sized files
stored with the wrong digest are not included.
Another glimmer of hope: there is only exactly one backup from 21
August 2021 in the database, so about 83% of the zero-sized,
misplaced files may well apply to this one very backup, incidentally
the initial backup of one of the youngest hosts in the collection.
I have a couple of questions:
1. Can someone explain what's going on, and hopefully put my mind to
rest?
2. Are these really just empty files for which a faulty digest was
calculated at some point in the past?
3. Why doesn't `refCountUpdate` report the same set of such files on
every run?
4. Why are these logged as errors, but not included in the total
error count?
5. How can I find out where in the backup hierarchies these files
are referenced?
6. Given that I know the precise backup that seems to be mostly
affected by this, is there anything else I can provide to help
diagnose this?
Thanks,
--
@martinkrafft | https://matrix.to/#/#madduck:madduck.net
"to me, vi is zen. to use vi is to practice zen. every command is
a koan. profound to the user, unintelligible to the uninitiated.
you discover truth everytime you use it."
-- reddy ät lion.austin.ibm.com
spamtraps: madduck.bo...@madduck.net
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
