** Description changed: + **NOTE: This ticket has been moved/altered. Original problem referred to + rsync behaviour with libxxhash0. New subject is the changed ABI of + libxxhash0.** + + Original-Subject: rsync 3.2.x in Groovy depends on broken libxxhash + 0.7.x + + Actual problem: libxxhash0 in Focal exposes 128-bit xxhash (xxh128) + symbols which behave differently on versions of libxxhash0 in Groovy and + above. + + In xxhash 0.7.x there is this notice: + + https://github.com/Cyan4973/xxHash/blob/173e50be0509c6fb6c1bb74d95049ef61d7fdced/xxhash.h#L465-L476 + + > The XXH3 algorithm is still in development. + > The results it produces may still change in future versions. + > + > Results produced by v0.7.x are not comparable with results + > from v0.7.y. However, the API is completely stable, and it + > can safely be used for ephemeral data (local sessions). + > + > Avoid storing values in long-term storage until the + > algorithm is finalized. XXH3's return values will be + > officially finalized upon reaching v0.8.0. + + Unfortunately these ended up as usable symbols in xxhash 0.7.x in + Debian: + + > XXH128@Base 0.7.0 + (etc..) + + This means that those linking against libxxhash0 would get a binary + dependency on 0.7.0. But behaviour of xxh128 differs between 0.7 and + 0.8. So if you're linking against xxh128 from 0.8.x, it should binary- + depend on >=0.8 and not >=0.7. (And if possible, you should not use + xxh128 at all in 0.7.x.) + + The following Debian changeset fixes this for newer libxxhash0 builds: + + https://github.com/norbusan/debian- + xxhash/commit/b51053c86952dd13573b08344101172c88de847f + + > debian/libxxhash0.symbols + > - XXH128@Base 0.7.0 + > + XXH128@Base 0.8.0 + + But, this is not fully resolved until: + + - that Debian change lands in libxxhash0 in Ubuntu; + - packages linking against libxxhash0 (e.g. rsync) are rebuilt against this newer package, so the Depends is bumped to >=0.8; + - xxh128 in libxxhash0 in Focal is clearly marked as ephemeral (or removed altogether; this applies to xxh128 symbols libxxhash0 and an `xxh128sum` binary in the xxhash package) + + Cheers, + Walter Doekes + OSSO B.V. + + + ---vvv--- Original ticket below ---vvv--- + + **Problem** - $ rsync root@focal-system:/etc/.pwd.lock . - ERROR: .pwd.lock failed verification -- update discarded. - rsync error: some files/attrs were not transferred (see previous errors) - (code 23) at main.c(1816) [generator=3.2.3] + $ rsync root@focal-system:/etc/.pwd.lock . + ERROR: .pwd.lock failed verification -- update discarded. + rsync error: some files/attrs were not transferred + (see previous errors) + (code 23) at main.c(1816) [generator=3.2.3] - - $ rsync root@focal-system:/etc/.pwd.lock . --debug=all - opening connection using: ssh -l root focal-system rsync --server --sender \ - -e.LsfxCIvu . /etc/.pwd.lock (10 args) - (Client) Protocol versions: remote=31, negotiated=31 - Client negotiated checksum: xxh128 - ... - + $ rsync root@focal-system:/etc/.pwd.lock . --debug=all + opening connection using: ssh -l root focal-system rsync \ + --server --sender \ + -e.LsfxCIvu . /etc/.pwd.lock (10 args) + (Client) Protocol versions: remote=31, negotiated=31 + Client negotiated checksum: xxh128 + ... **Cause** - focal-system# dpkg -l | grep -E 'libxxhash|rsync' - ii libxxhash0:amd64 0.7.3-1 amd64 - ii rsync 3.2.3-2ubuntu1 amd64 - + focal-system# dpkg -l | grep -E 'libxxhash|rsync' + ii libxxhash0:amd64 0.7.3-1 amd64 + ii rsync 3.2.3-2ubuntu1 amd64 **Why this affects only us and not more people?** On Ubuntu/Focal, there is no rsync 3.2.3, only 3.1.3-8. But because we need the lz4 compression support we've fetched a newer rsync (from Groovy). However: the rsync 3.2.3 depends on libxxhash0 0.7.1+, while in fact it needs 0.8+. + **Details** - **Details** - - On a Ubuntu/Focal system we have installed a rsync 3.2.3 package from Ubuntu/Groovy because we need the lz4 compression support. - + On a Ubuntu/Focal system we have installed a rsync 3.2.3 package from + Ubuntu/Groovy because we need the lz4 compression support. focal-system# apt-cache show rsync Package: rsync ... Version: 3.2.3-2ubuntu1 Depends: lsb-base, libacl1 (>= 2.2.23), libc6 (>= 2.15), - liblz4-1 (>= 0.0~r130), libpopt0 (>= 1.14), libssl1.1 (>= 1.1.0), - libxxhash0 (>= 0.7.1), libzstd1 (>= 1.3.8), zlib1g (>= 1:1.1.4) + liblz4-1 (>= 0.0~r130), libpopt0 (>= 1.14), libssl1.1 (>= 1.1.0), + libxxhash0 (>= 0.7.1), libzstd1 (>= 1.3.8), zlib1g (>= 1:1.1.4) ... - Alongside this we had libxxhash0 0.7.3-1 from Focal: focal-system# apt-cache policy libxxhash0 libxxhash0: - Installed: 0.7.3-1 - Candidate: 0.7.3-1 - Version table: - *** 0.7.3-1 500 - 500 http://ARCHIVE/ubuntu focal/universe amd64 Packages - 100 /var/lib/dpkg/status - + Installed: 0.7.3-1 + Candidate: 0.7.3-1 + Version table: + *** 0.7.3-1 500 + 500 http://ARCHIVE/ubuntu focal/universe amd64 Packages + 100 /var/lib/dpkg/status According to the dependencies, this should work. But the combination does not, as this quote from the rsync maintainer would tell you: https://github.com/WayneD/rsync/issues/122#issuecomment-737690913 > Yeah, Cyan4973 could have told you that the 128-bit xxhash only > just stabilized in its 0.8.0 release, so anything older than > that isn't compatible. - **The fix** As the maintainer points out, version 0.7 is not stable (= broken for our intents and purposes) and thus not fit for use with rsync 3.2. I would argue that it's a good idea to bump the dependency of rsync 3.2.3 on Groovy to libxxhash0>=0.8 After all, in Groovy there is a libxxhash0 0.8.0-1ubuntu1.20.10.1, so that would not be a problem. And it would fix issues for those mixing and matching packages. - Thanks! Walter Doekes OSSO B.V. - (*) possible patch: $ diff -pu debian/control{.orig,} --- debian/control.orig 2021-07-08 09:56:57.646861644 +0200 +++ debian/control 2021-07-08 09:57:38.499029903 +0200 @@ -8,7 +8,7 @@ Build-Depends: debhelper-compat (= 13), - libacl1-dev, - libpopt-dev, - liblz4-dev, + libacl1-dev, + libpopt-dev, + liblz4-dev, - libxxhash-dev, + libxxhash-dev (>= 0.8), - libzstd-dev, - zlib1g-dev, - libssl-dev + libzstd-dev, + zlib1g-dev, + libssl-dev
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1934992 Title: rsync 3.2.x in Groovy depends on broken libxxhash 0.7.x To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/xxhash/+bug/1934992/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
