[SCM] The rsync repository. - branch master updated
The branch, master has been updated
via f9e29dfb More NEWS updates.
via 591de7ce Fix compile w/o openssl; disable sha256 & sha512 for
--checksum.
via c8c62775 Avoid test -e.
via 46884e4f Fix a link.
from 97e02bf2 Some "use chroot" improvements.
https://git.samba.org/?p=rsync.git;a=shortlog;h=master
- Log -
commit f9e29dfb0912f2689ad10a7ba14ebb3ed43cf153
Author: Wayne Davison
Date: Sun Sep 25 12:49:45 2022 -0700
More NEWS updates.
commit 591de7ce5c80ed20ccf096772f11ddca082cdf41
Author: Wayne Davison
Date: Sun Sep 25 12:42:09 2022 -0700
Fix compile w/o openssl; disable sha256 & sha512 for --checksum.
commit c8c627756a607818fd92ac603ea63dcab6da33ef
Author: Wayne Davison
Date: Tue Sep 20 21:50:07 2022 -0700
Avoid test -e.
commit 46884e4ff665db1756ed20313c48f5ae4c2bf185
Author: Wayne Davison
Date: Tue Sep 20 00:12:49 2022 -0700
Fix a link.
---
Summary of changes:
NEWS.md| 26 --
checksum.c | 28 +++-
mkgitver | 2 +-
3 files changed, 36 insertions(+), 20 deletions(-)
Changeset truncated at 500 lines:
diff --git a/NEWS.md b/NEWS.md
index 63191360..0db49925 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -4,11 +4,15 @@
### BUG FIXES:
-- Fixed the validating of remote filter rules.
+- Fixed the client-side validating of the remote sender's filtering behavior.
+
+- More fixes for the "unrequested file-list name" name, including a copy of
+ "/" with `--relative` enabled and a copy with a lot of related paths with
+ `--relative` enabled (often derived from a `--files-from` list).
- When rsync gets an unpack error on an ACL, mention the filename.
-- Avoid oversetting sanitize_paths when a daemon is serving "/" (even if
+- Avoid over-setting sanitize_paths when a daemon is serving "/" (even if
"use chroot" is false).
### ENHANCEMENTS:
@@ -35,10 +39,15 @@
converted. Newer rsync versions will provide more complete info than older
versions.
-- The [`use chroot`](#rsyncd.conf) daemon parameter now defaults to "unset" so
- that rsync can test if chrooting works and decide to proceed with a sanitized
- copy if chroot is not supported (e.g., for a non-root daemon). Explicitly
- setting it to true or false (on or off) behaves the same way as before.
+- The [`use chroot`](rsyncd.conf.5#use_chroot) daemon parameter now defaults to
+ "unset" so that rsync can use chroot when it works and decide to proceed with
+ a sanitized copy when chroot is not supported (e.g., for a non-root daemon).
+ Explicitly setting it to true or false (on or off) behaves the same way as
+ before.
+
+- The `--fuzzy` option was optimized a bit to try to cut down on the amount of
+ computations when considering a big pool of files. The simple heuristic from
+ Kenneth Finnegan resuled in about a 2x speedup.
### PACKAGING RELATED:
@@ -62,6 +71,11 @@
as the root user (though rsync should behave the same even with the value
unset, a little extra paranoia doesn't hurt).
+- I've noticed that some packagers haven't installed support/nameconvert for
+ users to use in their chrooted rsync configs. Even if it is not installed
+ as an executable script (to avoid a python3 dependency) it would be good to
+ install it with the other rsync-related support scripts.
+
--
# NEWS for rsync 3.2.6 (9 Sep 2022)
diff --git a/checksum.c b/checksum.c
index b5363bca..071db0e8 100644
--- a/checksum.c
+++ b/checksum.c
@@ -59,12 +59,6 @@ struct name_num_item valid_checksums_items[] = {
{ CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL },
#ifdef SHA_DIGEST_LENGTH
{ CSUM_SHA1, NNI_EVP, "sha1", NULL },
-#endif
-#ifdef SHA256_DIGEST_LENGTH
- { CSUM_SHA256, NNI_EVP, "sha256", NULL },
-#endif
-#ifdef SHA512_DIGEST_LENGTH
- { CSUM_SHA512, NNI_EVP, "sha512", NULL },
#endif
{ CSUM_NONE, 0, "none", NULL },
{ 0, 0, NULL, NULL }
@@ -100,15 +94,16 @@ struct name_num_item implied_checksum_md5 =
{ CSUM_MD5, NNI_BUILTIN, "md5", NULL };
struct name_num_item *xfer_sum_nni; /* used for the transfer checksum2
computations */
-const EVP_MD *xfer_sum_evp_md;
int xfer_sum_len;
struct name_num_item *file_sum_nni; /* used for the pre-transfer --checksum
computations */
-const EVP_MD *file_sum_evp_md;
int file_sum_len, file_sum_extra_cnt;
#ifdef USE_OPENSSL
+const EVP_MD *xfer_sum_evp_md;
+const EVP_MD *file_sum_evp_md;
EVP_MD_CTX *ctx_evp = NULL;
#endif
+
static int initialized_choices = 0;
struct name_num_item *parse_csum_name(const char *name, int len)
@@ -147,9 +142,9 @@ struct name_num_item *parse_csum_name(const char *name, int
len)
return nni;
}
+#ifdef USE_OPENSSL
static const EVP_MD *csum_evp_md(struct name_num_item *nni)
{
-#ifdef
Re: ignore mtime - or any metadata
I can't say I have tried it but -pog would all also result in duplicating files (assuming they are different). On 9/25/22 14:49, Hardy via rsync wrote: Thanks Kevin, -a implies rlptgoD and I obviously used it by comfort and habit. Omitting -t (and perhaps also gop, not really needed) might solve my problem? Will give it a try. Am 25.09.22 um 20:30 schrob Kevin Korb via rsync: -a is telling rsync to copy the metadata. --size-only is telling rsync to skip the contents of files that are the same time. Without --link-dest rsync would just update the metadata on the target. However, with --link-dest rsync is being told to make the same file with 2 different metadatas. It can only do so by duplicating the file (effectively the same as --copy-dest). On 9/25/22 14:26, Hardy via rsync wrote: Hi, SHORT: rsync shall ignore differences in meta (inode) data, but does not even with --size-only. -- If in combination with --link-dest. Is there any flag I missed? EXPLANATION: I use rsync -avbuH --size-only --stats --delete-excluded --link-dest=/path/to/last "user@system:Shared/Data" /path/to/now to sync backup-style and use the --link-dest option to minimize traffic (and space) and still have a full backup each time. So far nothing special. Now it happens that at the source side data changes are moderate, but meta information (like atime, mtime) do change a lot. I didn't find a solution to make rsync ignore these differences to use a link. --size-only and -t (implied by -a) work ok with a simple syncs, but obviously (for me) it does not in combination with --link-dest. I can understand the motivation behind this: rsync is not allowed to change meta data of the existing inode, nor can it neglect the change - or can it? Hardy -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: ignore mtime - or any metadata
Thanks Kevin, -a implies rlptgoD and I obviously used it by comfort and habit. Omitting -t (and perhaps also gop, not really needed) might solve my problem? Will give it a try. Am 25.09.22 um 20:30 schrob Kevin Korb via rsync: -a is telling rsync to copy the metadata. --size-only is telling rsync to skip the contents of files that are the same time. Without --link-dest rsync would just update the metadata on the target. However, with --link-dest rsync is being told to make the same file with 2 different metadatas. It can only do so by duplicating the file (effectively the same as --copy-dest). On 9/25/22 14:26, Hardy via rsync wrote: Hi, SHORT: rsync shall ignore differences in meta (inode) data, but does not even with --size-only. -- If in combination with --link-dest. Is there any flag I missed? EXPLANATION: I use rsync -avbuH --size-only --stats --delete-excluded --link-dest=/path/to/last "user@system:Shared/Data" /path/to/now to sync backup-style and use the --link-dest option to minimize traffic (and space) and still have a full backup each time. So far nothing special. Now it happens that at the source side data changes are moderate, but meta information (like atime, mtime) do change a lot. I didn't find a solution to make rsync ignore these differences to use a link. --size-only and -t (implied by -a) work ok with a simple syncs, but obviously (for me) it does not in combination with --link-dest. I can understand the motivation behind this: rsync is not allowed to change meta data of the existing inode, nor can it neglect the change - or can it? Hardy -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: ignore mtime - or any metadata
-a is telling rsync to copy the metadata. --size-only is telling rsync to skip the contents of files that are the same time. Without --link-dest rsync would just update the metadata on the target. However, with --link-dest rsync is being told to make the same file with 2 different metadatas. It can only do so by duplicating the file (effectively the same as --copy-dest). On 9/25/22 14:26, Hardy via rsync wrote: Hi, SHORT: rsync shall ignore differences in meta (inode) data, but does not even with --size-only. -- If in combination with --link-dest. Is there any flag I missed? EXPLANATION: I use rsync -avbuH --size-only --stats --delete-excluded --link-dest=/path/to/last "user@system:Shared/Data" /path/to/now to sync backup-style and use the --link-dest option to minimize traffic (and space) and still have a full backup each time. So far nothing special. Now it happens that at the source side data changes are moderate, but meta information (like atime, mtime) do change a lot. I didn't find a solution to make rsync ignore these differences to use a link. --size-only and -t (implied by -a) work ok with a simple syncs, but obviously (for me) it does not in combination with --link-dest. I can understand the motivation behind this: rsync is not allowed to change meta data of the existing inode, nor can it neglect the change - or can it? Hardy -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
ignore mtime - or any metadata
Hi, SHORT: rsync shall ignore differences in meta (inode) data, but does not even with --size-only. -- If in combination with --link-dest. Is there any flag I missed? EXPLANATION: I use rsync -avbuH --size-only --stats --delete-excluded --link-dest=/path/to/last "user@system:Shared/Data" /path/to/now to sync backup-style and use the --link-dest option to minimize traffic (and space) and still have a full backup each time. So far nothing special. Now it happens that at the source side data changes are moderate, but meta information (like atime, mtime) do change a lot. I didn't find a solution to make rsync ignore these differences to use a link. --size-only and -t (implied by -a) work ok with a simple syncs, but obviously (for me) it does not in combination with --link-dest. I can understand the motivation behind this: rsync is not allowed to change meta data of the existing inode, nor can it neglect the change - or can it? Hardy -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Implicit containing directory sometimes rejected as unrequested
The git code should have this fixed. Feel free to build it yourself, or try the executable in the tar file from here: https://download.samba.org/pub/rsync/binaries/opensuse-15-x86_64/ ..wayne.. On Sat, Sep 24, 2022 at 5:02 PM jimc via rsync wrote: > Version: rsync-3.2.6-1.1.x86_64 from OpenSuSE Tumbleweed, get it from > > https://download.opensuse.org/repositories/openSUSE:/Factory/standard/x86_64/rsync-3.2.6-1.1.x86_64.rpm > (or .src.rpm). > > Symptom: In certain circumstances (see the reproducer script), rsync > from a remote source to a local destination and using a --files-from > list > sometimes rejects an implicit containing directory with this error > message: > ERROR: rejecting unrequested file-list name: usr/lib64 > rsync error: protocol incompatibility (code 2) at flist.c(998) \ > [Receiver=3.2.6] > > Success or failure depends on arcane determinstic variations, and there > is a suspicion that the same input might produce different output at > different times. However, I repeated two variants 20 times a few secs > apart and they always worked or failed. > > I've found these variant behaviors: > * With either sender version, receiver 3.2.5 always works and 3.2.6 >sometimes fails. > * If both the source and destination are local, it always works. > * If the destination is remote, it works. Failures are seen with a >remote source. > * All my test cases have two implicit containing directories. >The first one has never been seen to fail but the second one does. >I haven't investigated if third or subsequent dirs would fail. >In one case where the second dir failed, exchanging the two filenames >led to both of them succeeding. > * I'm using rsync in a configuration management system and it needs some >options. If I remove -K -O --numeric-ids leaving only --rsh=ssh -a >--files-from, it fails or works equally. > * If I add --trust-sender then failing cases start working. > > v3.2.5 has an addition to recognize if a rogue sender adds unrequested > toplevel names etc. (CVE-2022-29154) The option --trust-sender > disables the new paranoia. If this option is added to the execution > command line, spurious rejections disappear. Clearly the bugfixes in > file-list processing added in v3.2.6 had a bad interaction with the new > paranoia. > > Attaching reproducer script rsyncbug.sh . > > -- > James F. Carter Email: j...@jfcarter.net > Web: http://www.math.ucla.edu/~jimc (q.v. for PGP key) > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
