Implicit containing directory sometimes rejected as unrequested
Version: rsync-3.2.6-1.1.x86_64 from OpenSuSE Tumbleweed, get it from
https://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/rsync-3.2.6-1.1.x86_64.rpm
(or .src.rpm).
Symptom: In certain circumstances (see the reproducer script), rsync
from a remote source to a local destination and using a --files-from
list
sometimes rejects an implicit containing directory with this error
message:
ERROR: rejecting unrequested file-list name: usr/lib64
rsync error: protocol incompatibility (code 2) at flist.c(998) \
[Receiver=3.2.6]
Success or failure depends on arcane determinstic variations, and there
is a suspicion that the same input might produce different output at
different times. However, I repeated two variants 20 times a few secs
apart and they always worked or failed.
I've found these variant behaviors:
* With either sender version, receiver 3.2.5 always works and 3.2.6
sometimes fails.
* If both the source and destination are local, it always works.
* If the destination is remote, it works. Failures are seen with a
remote source.
* All my test cases have two implicit containing directories.
The first one has never been seen to fail but the second one does.
I haven't investigated if third or subsequent dirs would fail.
In one case where the second dir failed, exchanging the two filenames
led to both of them succeeding.
* I'm using rsync in a configuration management system and it needs some
options. If I remove -K -O --numeric-ids leaving only --rsh=ssh -a
--files-from, it fails or works equally.
* If I add --trust-sender then failing cases start working.
v3.2.5 has an addition to recognize if a rogue sender adds unrequested
toplevel names etc. (CVE-2022-29154) The option --trust-sender
disables the new paranoia. If this option is added to the execution
command line, spurious rejections disappear. Clearly the bugfixes in
file-list processing added in v3.2.6 had a bad interaction with the new
paranoia.
Attaching reproducer script rsyncbug.sh .
--
James F. Carter Email: j...@jfcarter.net
Web: http://www.math.ucla.edu/~jimc (q.v. for PGP key)
#!/bin/bash
# rsync-3.2.6-1.1.x86_64 shows the bug; rsync-3.2.5-1.1.x86_64 works perfectly.
# The --files-from list has just files, but one of the implicit containing
# directories will be rejected with this error message:
#ERROR: rejecting unrequested file-list name: usr/lib64
#rsync error: protocol incompatibility (code 2) at flist.c(998) \
# [Receiver=3.2.6]
# Receiver 3.2.5 works with either sender. Receiver 3.2.6 fails with either.
# Not every --files-from list sets off the bug.
: ${HOSTNAME:=$(uname -n)} # Name of this host for "remote" execution
j=/var/tmp/root.jimc # Directory to put the test files in
mkdir -p $j
cd $j
# Actual filenames that failed in the config management system
if /bin/true ; then # *** Turn on one of these
cat - > $j/filesfrom.ls < $j/filesfrom.ls < $fname
done < $j/filesfrom.ls
# Testing rsync...
rsync --rsh=ssh -a -K -O --numeric-ids --files-from=$j/filesfrom.ls $HOSTNAME:$j/src/ $j/dst/
echo "exit code = $?"
# Outcomes:
# "rsync --options $j/src/ $j/dst/" works perfectly on either file list.
# "rsync --options $HOSTNAME:$j/src/ $j/dst/" rejects dirname of 2nd file
# (lib/ OK, lib64/ rejected). This order and remote src are used for
# variant tests except as noted.
# But if lib64/ is before lib/ then both are transferred.
# "rsync --options $j/src/ $HOSTNAME:$j/dst/" transfers both files.
# Aargh, there's a suspicion that it may be intermittent! Got to test a lot
# of repetitions. These were a few seconds apart, using the actual filenames.
# 1 = failed on first dir, 2 = failed on 2nd dir, x = transferred both files.
# x x x x x x x x x x x x x x x x x x x x x (21 trials, remote dst)
# 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 (20 trials, remote src)
# If most options are removed, leaving only --rsh=ssh -a --files-from,
# it fails or succeeds equally. Failed with orig. filenames, remote src,
# /usr/lib64 second. Worked with simplified names.
# Big clue! v3.2.5 has an addition to recognize if a rogue sender adds
# unrequested toplevel names etc. (CVE-2022-29154) The option --trust-sender
# disables the new paranoia. If this option is added to the execution command
# line, spurious rejections disappear. Clearly the bugfixes in file-list
# processing added in v3.2.6 had a bad interaction with the new paranoia.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question abount pre-xfer exec
You only log you would like to to mount /backup, but the actual command is missing. You should also log errors, so something like /usr/bin/mount /backup >> /var/log/rsyncd.log 2>&1 would be adequate before your line to check what is mounted. Hope this helps Hardy Am 24.09.22 um 15:15 schrob dotdeb--- via rsync: I've been using rsync for years to backup my machines both at work and at home. These days I faced a new "challenge": at work I connect my laptop to a docking station with an external usb disk. I'd like to use this disk as a backup volume. I put my disk in /etc/fstab to be mounted at boot (with 'nofail' option to avoid errors when I'm at home). I have no problems if the laptop is booted after the connection to the docking station but, if I work at home, suspend the laptop and then go to work and connect it to the docking station and resume it, the /backup volume will not be mounted automatically. I found that rsyncd.conf can execute scripts before and I tried to create a script to be executed (as early stage or pre transfer? a bit confuser about it) to check if /backup is mounted and mount it if not. I verified that the script is executed (I put there some debugging "echo" sent to the log file) but the mount command within it does not mount anything. Here it is the rsyncd.conf ## read only = false write only = false usechroot = true uid = 0 gid = 0 early exec = /tmp/test-pre-exec [rsync-backup-xxx] comment Local rsync-backup of xxx path = /backup/xxx log file = /var/log/rsyncd.log ## and the script /tmp/test-pre-exec ## #!/bin/sh echo -n "executing pre-xfer script ..." >> /var/log/rsyncd.log if ! grep -qs '/backup ' /proc/mounts then echo -n "mounting /backup ..." >> /var/log/rsyncd.log /usr/bin/mount >> /var/log/rsyncd.log fi echo " done" >> /var/log/rsyncd.log exit 0 ## -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question abount pre-xfer exec
You aren't logging any stderr. That is where any error messages would go. Add some 2>&1. Also, mount has a -v On 9/24/22 09:15, dotdeb--- via rsync wrote: I've been using rsync for years to backup my machines both at work and at home. These days I faced a new "challenge": at work I connect my laptop to a docking station with an external usb disk. I'd like to use this disk as a backup volume. I put my disk in /etc/fstab to be mounted at boot (with 'nofail' option to avoid errors when I'm at home). I have no problems if the laptop is booted after the connection to the docking station but, if I work at home, suspend the laptop and then go to work and connect it to the docking station and resume it, the /backup volume will not be mounted automatically. I found that rsyncd.conf can execute scripts before and I tried to create a script to be executed (as early stage or pre transfer? a bit confuser about it) to check if /backup is mounted and mount it if not. I verified that the script is executed (I put there some debugging "echo" sent to the log file) but the mount command within it does not mount anything. Here it is the rsyncd.conf ## read only = false write only = false usechroot = true uid = 0 gid = 0 early exec = /tmp/test-pre-exec [rsync-backup-xxx] comment Local rsync-backup of xxx path = /backup/xxx log file = /var/log/rsyncd.log ## and the script /tmp/test-pre-exec ## #!/bin/sh echo -n "executing pre-xfer script ..." >> /var/log/rsyncd.log if ! grep -qs '/backup ' /proc/mounts then echo -n "mounting /backup ..." >> /var/log/rsyncd.log /usr/bin/mount >> /var/log/rsyncd.log fi echo " done" >> /var/log/rsyncd.log exit 0 ## -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
question abount pre-xfer exec
I've been using rsync for years to backup my machines both at work and at home. These days I faced a new "challenge": at work I connect my laptop to a docking station with an external usb disk. I'd like to use this disk as a backup volume. I put my disk in /etc/fstab to be mounted at boot (with 'nofail' option to avoid errors when I'm at home). I have no problems if the laptop is booted after the connection to the docking station but, if I work at home, suspend the laptop and then go to work and connect it to the docking station and resume it, the /backup volume will not be mounted automatically. I found that rsyncd.conf can execute scripts before and I tried to create a script to be executed (as early stage or pre transfer? a bit confuser about it) to check if /backup is mounted and mount it if not. I verified that the script is executed (I put there some debugging "echo" sent to the log file) but the mount command within it does not mount anything. Here it is the rsyncd.conf ## read only = false write only = false usechroot = true uid = 0 gid = 0 early exec = /tmp/test-pre-exec [rsync-backup-xxx] comment Local rsync-backup of xxx path = /backup/xxx log file = /var/log/rsyncd.log ## and the script /tmp/test-pre-exec ## #!/bin/sh echo -n "executing pre-xfer script ..." >> /var/log/rsyncd.log if ! grep -qs '/backup ' /proc/mounts then echo -n "mounting /backup ..." >> /var/log/rsyncd.log /usr/bin/mount >> /var/log/rsyncd.log fi echo " done" >> /var/log/rsyncd.log exit 0 ## -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
