Le ven. 17 juil. 2026 à 17:21, G.W. Haywood <[email protected]> a écrit :
> Hi there, > > On Fri, 17 Jul 2026, J?r?me Aug? wrote: > > > I'm currently configuring a new bare-metal BackupPC v4 server with Ubuntu > > 24.04 packages (backuppc `4.4.0-8build1` + backuppc-rsync > > `3.1.3.0-3ubuntu1`), for replacing our aging BackupPC v3 on Ubuntu 20.04, > > and I'm experiencing various errors and silent corruptions. > > I'm in touch with another user who is having trouble with any version > of rsync-bpc 3.1.3.x. In that case it appears to be a resource issue, > but until now it has seemed only to happen in a particularly odd set > of circumstances that include Docker on top of layers of filesystems > which I've never seen elsewhere. The user found that version 3.1.2.2 > is the most recent which works for him in that peculiar setting. If > you can install 3.1.2.2 to see if it helps I'd be interested to know, > but it might be better for you to install the latest rsync-bpc from a > tarball that I can send to you - I'd be very keen to know how you get > on with it. Alternatively, if you're up for it I can let you have a > debug version of 3.1.3 (or any other version) which will log a lot of > information on resource usage. That might shed more light directly on > your issue. Finally if you'd like to try the latest rsync-3.4.1rc1 I > can let you have the tarball. I did deal with some memory issues when > I was working on it, but it's had little testing other than my use of > it for my own systems for a few months. The version currently running > here was compiled on May 30th. Building from source should be easy, a > couple of config tweakes might be needed. > I guess you refer to this issue https://github.com/backuppc/rsync-bpc/issues/28? I'll see if I can try downgrading rsync-bpc to v3.1.2.2. Do you have a Github repo with this 3.4.1rc1 version? > > Hosts appears to be backed up correctly in the UI, but when I look at the > > XferLOG or browse the list of files from the UI, I see errors and missing > > files/dirs. > > It's disconcerting that you're seeing nothing in the UI to indicate > that there are problems with the backups - are you looking at the full > UI pages? One or two of them are rather long, and the existence of > Xfer errors might not be obvious until you scroll a way down the page. > Do you have emailed alerts set up? For your own peace of mind you > might want to set up something on the crontab to scan recent XferLOGs > for error messages nightly but this should all ideally be taken care > of by BackupPC itself of course. > That's exactly what I have set up, for catching "rsync_bpc:" messages. > > I have observed two cases of corruptions so far. > > One occurred during a network outage (at the beginning of the month) that > > left some files unreadable as seen with `BackupPC_tarCreate` with errors > > like `Unable to open file > > The messages "Unable to open file ..." and "Error: padding ..." are > expected if BackupPC_tarCreate runs into problems with its input data. > > > ... > > The second one occurred this evening, without any network outage, with a > > host showing many errors in the XferLOG like: > > - `G bpc_path_create: can't create f%2f/fetc (errno 2)` > > - `rsync_bpc: recv_generator: mkdir "/etc/NetworkManager" failed: No such > > file or directory (2)` > > - `G bpc_poolWrite_updateMatches: can't open candidate file > > /var/lib/backuppc/cpool/7c/1a/7d1acb7c259e7575f6d5f609c092cb8101 for > read` > > - `G bpc_attribCache_dirWrite: failed to write attributes for dir > > > f%2f/fusr/flib/fmodules/f5.15.0-156-generic/fkernel/fdrivers/fmedia/fplatform/fcadence/attrib > > ` > > > > `/var/log/kern.log` shows a segfault at 06:00: > > When I'm testing new code I see similar errors if rsync-bpc segfaults, > but it's not normally supposed to do that. :( IIRC the user that I > mentioned earlier sees instead that the problem rsync-bpc processes > get killed by the OOM killer. It might be worth looking at 'top' > while backups are running to see what's going on. On my backup > servers I routinely log the output of 'top -b -n1' to a file every > five minutes as a way of keeping an eye on memory consumption, rsync > can somethimes get a little heavy. You could try tweaking the kernel > memory management, e.g. /proc/sys/vm/overcommit_memory = 0 which will > would give you a hard memory allocation failure instead of some weird > error if things get out of hand with memory. > The memory and swap usage of this BackupPC server is already monitored, and I've not seen a spike in memory or swap usage during the night. And I see no OOM messages in dmesg or kern.log neither. > > ... > > What I also find strange, is that the XferLOG seems to be garbled with > what > > looks like multiples processes overwriting each other and truncated > lines, > > Just a shot in the dark: are you accidentally running more than one > BackupPC process or something like that? Again, that shouldn't happen. > That's what I also tought, but being a standard and default install with Ubuntu packages I doubt there could be an issue with multiple BackupPC instances... Last night I added a cron to log the server's process tree every minute and I see no concurrent executions of `BackupPC_dump`. However, as you mentionned in your reply, the only think that might explain this is `rsync-bpc` which seems to fork to execute two childs : one for connecting by SSH to launch the remote `rsync --server --sender` and one where it seems to re-execute himself (same command line) which might be the generator or the receiver part: --8<-- backuppc 940562 0.0 0.0 23320 15448 ? Ss 10:40 0:05 /usr/bin/perl /usr/share/backuppc/bin/BackupPC backuppc 964102 0.0 0.0 32784 24192 ? S 21:04 0:02 \_ /usr/bin/perl /usr/share/backuppc/bin/BackupPC_dump xxx.xxx backuppc 970611 1.1 0.6 257792 217700 ? S 22:37 0:55 \_ /usr/libexec/backuppc-rsync/rsync_bpc --bpc-top-dir /var/lib/backuppc --bpc-host-name xxx.xxx --bpc-share-name /var/opt --bpc-bkup-num 14 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 1173648 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh -l backup --rsync-path=sudo /usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --one-file-system --partial --log-format=log: %o %i %B %8U,%8G %9l %f%L --stats --timeout=72000 --exclude=nobackup xxx.xxx:/var/opt/ / backuppc 970612 10.1 0.0 19368 13512 ? S 22:37 8:05 \_ /usr/bin/ssh -l backup xxx.xxx sudo /usr/bin/rsync --server --sender -slHogDtprxe.iLsfxC backuppc 970613 68.1 0.6 401408 207388 ? R 22:37 54:16 \_ /usr/libexec/backuppc-rsync/rsync_bpc --bpc-top-dir /var/lib/backuppc --bpc-host-name xxx.xxx --bpc-share-name /var/opt --bpc-bkup-num 14 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 1173648 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh -l backup --rsync-path=sudo /usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --one-file-system --partial --log-format=log: %o %i %B %8U,%8G %9l %f%L --stats --timeout=72000 --exclude=nobackup xxx.xxx:/var/opt/ / -->8-- These garbled lines seems to be detected as errors and shown when one clicks on the "Errors" links in the "Xfer Error Summary" as they do not match the usual same/new/pool/del keyword lines? Here is a full "Errors" ouput in "Xfer Error Summary" from one of my hosts: --8<-- Running: /usr/libexec/backuppc-rsync/rsync_bpc --bpc-top-dir /var/lib/backuppc --bpc-host-name xxx.xxx --bpc-share-name / --bpc-bkup-num 14 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 1173601 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh\ -l\ backup --rsync-path=sudo\ /usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --one-file-system --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --timeout=72000 --exclude=var/log/lastlog --exclude=var/swap --exclude=swap --exclude=swap.img --exclude=nobackup xxx.xxx:/ / incr backup started for directory / Xfer PIDs are now 970510 This is the rsync child about to exec /usr/libexec/backuppc-rsync/rsync_bpc Xfer PIDs are now 970510,970513 xferPids 970510,970513 [ skipped 160 lines ] s/uaclient/__pycache__/event_logger.cpython-310.pyc [ skipped 13 lines ] lient/__pycache__/version.cpython-310.pyc [ skipped 4 lines ] ges/uaclient/api/__pycache__/exceptions.cpython-310.pyc [ skipped 3 lines ] ist-packages/uaclient/api/u/apt_news/__pycache__/__init__.cpython-310.pyc [ skipped 10 lines ] c.000000usr/lib/python3/dist-packages/uaclient/api/u/pro/attach/auto/full_auto_attach/__pycache__/__init__.cpython-310.pyc [ skipped 19 lines ] ib/python3/dist-packages/uaclient/api/u/pro/attach/magic/initiate/__pycache__/v1.cpython-310.pyc 3/dist-packages/uaclient/api/u/pro/detach/__pycache__/__init__.cpython-310.pyc packages/uaclient/api/u/pro/detach/__pycache__/v1.cpython-310.pyc [ skipped 18 lines ] n3/dist-packages/uaclient/api/u/pro/security/cves/__pycache__/v1.cpython-310.pyc [ skipped 955 lines ] 10.gz ion.log.11.gz ion.log.20.gz [ skipped 422 lines ] Done: 0 errors, 800 filesExist, 925593949 sizeExist, 790116431 sizeExistComp, 0 filesTotal, 0 sizeTotal, 120 filesNew, 913828217 sizeNew, 233004748 sizeNewComp, 1173645 inode Number of files: 241,313 (reg: 204,618, dir: 30,451, link: 6,140, special: 104) Number of created files: 33 (reg: 22, dir: 10, special: 1) Number of deleted files: 152 (reg: 131, dir: 12, link: 9) Number of regular files transferred: 1,181 Total file size: 16,444,281,572 bytes Total transferred file size: 1,843,060,054 bytes Literal data: 1,568,906,132 bytes Matched data: 270,516,034 bytes File list size: 1,384,242 File list generation time: 0.002 seconds File list transfer time: 0.000 seconds Total bytes sent: 3,092,291 Total bytes received: 1,576,139,498 sent 3,092,291 bytes received 1,576,139,498 bytes 11,240,083.91 bytes/sec total size is 16,444,281,572 speedup is 10.41 DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 210862 filesTotal, 16444281572 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 1173622 inode -->8-- > > ... > > I'm pondering switching to "tar" XferMethod to see if it helps... > > ... > > If you can do that it would be very interesting to know what happens - > will you be using remotely mounted filesystems? > After re-reading the documentation, it mentions that "tar incrementals are not able to detect deleted files, renamed files", which I'm not confident with, as superfluous files could cause unexpected behaviours on some restored systems... So I've kept the "rsync" method for the moment. Regards, Jérôme
_______________________________________________ BackupPC-users mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
