Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
On Sat, Feb 11, 2023 at 08:48:55AM +0100, Paul Gevers wrote: > Hi, > > Disclaimer, I may be misunderstanding how things work, because I only judge > it on observations and some comments in threads here and there. > > On 11-02-2023 00:43, Adrian Bunk wrote: > > I would be curious whether there is any technical reason why most > > Go libraries are binary-all but most Rust libraries are binary-any. > > > > Both choices look reasonable to me, but different ecosystems in Debian > > seem to have made different decisions in the same situation. > > I think go libraries are compiled statically into the other go binary > packages, so the whole go world needs to be recompiled for security (or > other) issues. >... Go libraries are source files like Rust libraries. > Paul cu Adrian
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Hi, Disclaimer, I may be misunderstanding how things work, because I only judge it on observations and some comments in threads here and there. On 11-02-2023 00:43, Adrian Bunk wrote: I would be curious whether there is any technical reason why most Go libraries are binary-all but most Rust libraries are binary-any. Both choices look reasonable to me, but different ecosystems in Debian seem to have made different decisions in the same situation. I think go libraries are compiled statically into the other go binary packages, so the whole go world needs to be recompiled for security (or other) issues. IIUC some/most rust libraries are source only, so it's the packages that build with rust that need to be binNMU-able. I have no experience with rust whatsoever, but to me as a bystander, the claim of Jonas does make sense. However, I agree with bunk: deviating from the ecosystem (in Debian) has a price and also I wonder if that price (non-maintainer human resources) might be high even _if_ the solution is superior. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
On Fri, Feb 10, 2023 at 11:32:21AM +0100, Jonas Smedegaard wrote: >... > Reduced size in the archive: Avoids multiple identical copies of library > source code >... I remember the 1990s when archive size was a problem, but a few years ago we had the first file > 2 GB in the archive (AFAIR an .orig.tar.gz) and that was not a problem except for some fast fix needed somewhere for files > INT_MAX. > Reduced complexity of APT: Avoids replicated packaging metadata > > Avoids certain types of impossible-to-satisfy scenarios: >... > Yes, I am aware that the Rust team packages arch-all code as arch-any > packages, but I am unaware that their reasoning is well documented > anywhere. >... A scarce resource is human time. There are several people who know how Rust packages in Debian work. They can quickly make changes to any normal Rust package. If you package your Rust packages differently, then it becomes harder and more time-consuming if anyone else ever has to touch one of your packages (e.g. a security update for rust-rustls). When your packages are "better" than the normal packages in an ecosystem, you have low benefits when most packages in the ecosystem stay "worse", while having the high costs associated with having different ways of packaging in the ecosystem. Better benefits and fewer costs would be if you try to convince the ecosystem as a whole to change, and in any case use for your own packages whatever the ecosystem is doing. > - Jonas cu Adrian
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
On Fri, Feb 10, 2023 at 03:10:03PM -0700, Sam Hartman wrote: > > "Jonas" == Jonas Smedegaard writes: > > > Jonas> Yes, I am aware that the Rust team packages arch-all code as > Jonas> arch-any packages, but I am unaware that their reasoning is > Jonas> well documented anywhere. The only reason I was aware of > Jonas> when I did the switch was that Debian has a convenient > Jonas> processing of binNMUs but annoyingly require source-only > Jonas> releases for rebuilding arch-all packages. > > But the bin nmu issue is a huge deal. I'm not sure for your case. But > Rust is statically linked, and so it is more likely than average to need > bin nmus. So if your packages will need to be rebuilt when their > dependencies change (I am not sure if this is true for packages just > containing crate source code), then moving from arch any to arch all > makes things significantly more difficult for SRM, RT, security team, > and people doing QA woork. > I'd definitely recommend reaching out to Adrian Bunk on this issue. Built-Using in binary-all packages is a problem. Extra-Source-Only means we might even have our mirrors serve sources that were removed from unstable years ago due to "not distributable". No matter whether or not Built-Using is used (which should only be used for licence compliance), the rebuild question might come up for binary-all packages that contain binaries for an architecture that is encoded in the package name (e.g. cross compiler libraries or microcontroller firmware like firmware-microbit-micropython). There are rare cases where binary-all python packages have to be rebuilt during every python transition, e.g. scripts that start with #!/usr/bin/python3.11 And then there's the special PITA of the gcc-*-cross* packages that also make invalid assumptions like that the binary-any packages are not built after the binary-all packages from the same source package (otherwise the binary-any packages might have dependencies that cannot be fulfilled). None of the above seems to apply to the package in question. There is the real problem that all binaries that have been built using rust-rustls might need rebuilding in security or stable when rust-rustls gets a security or other important fix, but for that it does not make any difference whether librust-rustls-dev is binary-any or binary-all. > If on the other hand your packages aren't going to need to be rebuilt > when their dependencies change, it may not be a big deal. I would be curious whether there is any technical reason why most Go libraries are binary-all but most Rust libraries are binary-any. Both choices look reasonable to me, but different ecosystems in Debian seem to have made different decisions in the same situation. cu Adrian
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Quoting Paul Gevers (2023-02-10 21:14:52) > On 09-02-2023 23:59, Jonas Smedegaard wrote: > > I can only interpret that as the test environment on thos arches being > > broken. Please don't punish the package for that :-( > > I think your interpretation it wrong. librust-rustls-dev Depends on > librust-ring-0.16+default-dev which is only available on amd64, arm64, > armel, armhf and i386. The test used to be skipped on ppc64el and s390x, > because you had skip-not-installable enabled, so the results were > neutral. You removed the skip-not-installable restriction, so now the > test fails. The regression is your own making, don't blame the test > environment. Ah, I am very sorry for throwing blame with stuff I don't understand. I shall stop doing that! > Having said that, as I mentioned before I regret I added the > skip-not-installable restriction, although it has it's use of giving > maintainers some control. I'll add the hint shortly. > > > There seems to be same problem for rust-oxhttp, and rust-hyper-rustls. > > Do you want a separate bugreport for each, or is it fine just mentioning > > it here? > > I'll have a look and if they're fine, I'll handle them too. Thanks! - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ * Sponsorship: https://ko-fi.com/drjones [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Quoting Sam Hartman (2023-02-10 23:10:03) [concerns over binNMUing due to static linking snipped: irrelevant!] > If on the other hand your packages aren't going to need to be rebuilt > when their dependencies change, it may not be a big deal. librust-*-dev packages contain no static linking. I am unaware of this being a big deal (besides those hickups in infrastructure when switching from arch-any to arch-all which as I understand it is a bug somewhere which just hasn't been shaken out because a move like the one I did is rare). - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ * Sponsorship: https://ko-fi.com/drjones [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
> "Jonas" == Jonas Smedegaard writes: Jonas> Yes, I am aware that the Rust team packages arch-all code as Jonas> arch-any packages, but I am unaware that their reasoning is Jonas> well documented anywhere. The only reason I was aware of Jonas> when I did the switch was that Debian has a convenient Jonas> processing of binNMUs but annoyingly require source-only Jonas> releases for rebuilding arch-all packages. But the bin nmu issue is a huge deal. I'm not sure for your case. But Rust is statically linked, and so it is more likely than average to need bin nmus. So if your packages will need to be rebuilt when their dependencies change (I am not sure if this is true for packages just containing crate source code), then moving from arch any to arch all makes things significantly more difficult for SRM, RT, security team, and people doing QA woork. I'd definitely recommend reaching out to Adrian Bunk on this issue. If on the other hand your packages aren't going to need to be rebuilt when their dependencies change, it may not be a big deal. signature.asc Description: PGP signature
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Hi Jonas, On 09-02-2023 23:59, Jonas Smedegaard wrote: I can only interpret that as the test environment on thos arches being broken. Please don't punish the package for that :-( I think your interpretation it wrong. librust-rustls-dev Depends on librust-ring-0.16+default-dev which is only available on amd64, arm64, armel, armhf and i386. The test used to be skipped on ppc64el and s390x, because you had skip-not-installable enabled, so the results were neutral. You removed the skip-not-installable restriction, so now the test fails. The regression is your own making, don't blame the test environment. Having said that, as I mentioned before I regret I added the skip-not-installable restriction, although it has it's use of giving maintainers some control. I'll add the hint shortly. There seems to be same problem for rust-oxhttp, and rust-hyper-rustls. Do you want a separate bugreport for each, or is it fine just mentioning it here? I'll have a look and if they're fine, I'll handle them too. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Quoting Sebastian Ramacher (2023-02-10 10:30:53) > Control: tags -1 moreinfo > Control: severity -1 normal > > On 2023-02-09 23:59:25 +0100, Jonas Smedegaard wrote: > > Package: release.debian.org > > Severity: important > > > > ckage src:rust-rustls is not migrating to testing because tests fail for > > s390x and ppc64el. On both arches the failure is that the test > > environment thinks it needs to install an arch-specific > > librust-rustls-dev to satisfy a virtual dependency pointing back to > > itself, which is (since recently) an arch-all package. > > > > I can only interpret that as the test environment on thos arches being > > broken. Please don't punish the package for that :-( > > > > There seems to be same problem for rust-oxhttp, and rust-hyper-rustls. > > Do you want a separate bugreport for each, or is it fine just mentioning > > it here? > > What's the added benefit of the conversion to Architecture: all > packages? Currently there does not seem to be a concensus to do this > switch for rust packages and doing them shortly before the soft freeze > apparently just generates work for the RT. Reduced size in the archive: Avoids multiple identical copies of library source code Reduced complexity of APT: Avoids replicated packaging metadata Avoids certain types of impossible-to-satisfy scenarios: https://github.com/sagebind/threadfin/issues/3 is an example of a complexity which due to differences in how crates.io and APT handles variety in architecture support caused a dependency of src:rust-threadfin to be impossible to migrate to testing because a test-dependency was unavailable on a lesser-used architecture and could not (easily) be skipped for single architectures. That concrete example was luckily solved now before the freeze, but would have been quite problematic to handle afte the freeze, and was the trigger for my decisison to switch all of the Rust libaries I maintain to use the format which is generally better understood in Debian context: arch-all code should be packaged as arch-all packages. Yes, I am aware that the Rust team packages arch-all code as arch-any packages, but I am unaware that their reasoning is well documented anywhere. The only reason I was aware of when I did the switch was that Debian has a convenient processing of binNMUs but annoyingly require source-only releases for rebuilding arch-all packages. I certainly had no intention of causing additional burden for the release team, and only learned about the hickups in infrastructure now experienced when I did the switch. On the contrary, as explained above, one of my reasons for doing the switch was to _reduce_ the risk of release team headaches _after_ the freeze. - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ * Sponsorship: https://ko-fi.com/drjones [x] quote me freely [ ] ask before reusing [ ] keep private
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Control: tags -1 moreinfo Control: severity -1 normal On 2023-02-09 23:59:25 +0100, Jonas Smedegaard wrote: > Package: release.debian.org > Severity: important > > ckage src:rust-rustls is not migrating to testing because tests fail for > s390x and ppc64el. On both arches the failure is that the test > environment thinks it needs to install an arch-specific > librust-rustls-dev to satisfy a virtual dependency pointing back to > itself, which is (since recently) an arch-all package. > > I can only interpret that as the test environment on thos arches being > broken. Please don't punish the package for that :-( > > There seems to be same problem for rust-oxhttp, and rust-hyper-rustls. > Do you want a separate bugreport for each, or is it fine just mentioning > it here? What's the added benefit of the conversion to Architecture: all packages? Currently there does not seem to be a concensus to do this switch for rust packages and doing them shortly before the soft freeze apparently just generates work for the RT. Cheers > > - Jonas > > -- Sebastian Ramacher
Bug#1030957: release.debian.org: please have rust-rustls ignore CI tests for s390x and ppc64el
Package: release.debian.org Severity: important -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 ckage src:rust-rustls is not migrating to testing because tests fail for s390x and ppc64el. On both arches the failure is that the test environment thinks it needs to install an arch-specific librust-rustls-dev to satisfy a virtual dependency pointing back to itself, which is (since recently) an arch-all package. I can only interpret that as the test environment on thos arches being broken. Please don't punish the package for that :-( There seems to be same problem for rust-oxhttp, and rust-hyper-rustls. Do you want a separate bugreport for each, or is it fine just mentioning it here? - Jonas -BEGIN PGP SIGNATURE- iQIzBAEBCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAmPlesoACgkQLHwxRsGg ASHWew/+P4ARdVeceSslT016FEdGE8y5oQ0Ee/gcNqLFQN7U/xwLLvyFq2PDcIWH uwAJUL4QVipEwQd1J7AzpCuWly5ac7G1iQB/wAslAs9SbFeoQ2rOhZNQZv6Qf9vj OkLB1v1hDkGgiQAjwDSc7mUhn7O/p/o0FZwlbtcoMAFm5se8rWptrMvmcUy9AweR PJkc08guOQD8+Fn28BwbEduprHCqPwwdVWzgMZwNT7ID10TTl7nuY365HvguOa5z iHcctNoQd4Sm3wQMp3YaZp0LFa7MpZH6wvf4mNGheKzLsn0HZuRaVd0gJBo5MJCD OZ9izaiicGGgE+2QGOLoomT9/UT6K6xsfkTJnDnXjyOYH4FmPNorCNlfz/dvKtEo PdAvoxE9gO+sMXFUuxc7HJZAxV1zOo85WGBoB8Bjfb0sSX9+KC/GyULLdVEGJ0su zKzLh99jQRhLXPYifBt9JpstLWdg7WGOd1GEvNW0J8/lpQoC627TDF4hutLb4KFl Ffzl9Y91e1oI59L5mO8BfYezjJe8VifFM4QR2kQqGUe/7fIUVRFvNGis/FKX67wn b2/t0rBJymqkdrz+hoavmV/iO3iBiNjDCLxak3VW6d8CVsfkn3mHVZeJ4UQU9qyu 05j7I0D1rSagheRAjpfVyfC3yIMMz1/GaFQGOqbJIimJhQ7mNhc= =GkQb -END PGP SIGNATURE-
