Bug#1064856: dpkg: New xz-utils print warnings on stderr
On 2024-02-26 20:46:43 [+0100], Guillem Jover wrote: > > > Ignoring stderr could be a workaround, but I'd need to do something as > > > well for the libdpkg code and the perl code calling xz, which will get > > > very annoying. > > > > > > This is also going to get in the way of migrating both xz and dpkg > > > (which seems like would need to be uploaded today for the time64 > > > transition). > > > > > > Or perhaps that warning could be disabled for now in Debian until things > > > are sorted out with upstream? > > > > Falling back to -T1 in order to keep the previous is not an option? > > I'm not sure I understand what you are saying. Do you mean dpkg would > fallback to pass -T1 (maybe you mean -T+1, otherwise we might get > unreproducible output due to switching to single-threaded)? And > fallback on what condition? Yes. I *think* that error came from the decompress part but I'm not sure. > Ah, I think you mean to pass -T+1 to the xz invocations for the test > suite, right, that could workaround the issue there indeed. Thanks, I > think I'll do that for now. Thank you. > > Let me try to sell this "we move this warning to verbose" to upstream in > > the meantime… > > That would actually be great! > > > > (Had not seen this test suite failure yet, as I've got xz on hold due > > > to the breaks on pristine-tar, but would probably stumble over it > > > during the release process anyway I guess, so thanks for the heads up!) > > > > This poped up on xz debci only armel and armhf. > > Perhaps I'll not see it in my local tree then, but I think the dpkg > failure will at least block xz migration, but thinking about it now, > probably not dpkg's migration. So this might not be blocking for dpkg. > (Sorry if this sounded alarming, but didn't want to add new roadblocks > to the time64 transition from the dpkg side! :D) Understood ;) > Thanks, > Guillem Sebastian
Bug#1064856: dpkg: New xz-utils print warnings on stderr
On Mon, 2024-02-26 at 20:10:20 +0100, Sebastian Andrzej Siewior wrote: > On 2024-02-26 19:23:58 [+0100], Guillem Jover wrote: > > > | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the > > > memory usage limit of 1400 MiB > > > | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the > > > memory usage limit of 1400 MiB > > > | 89s 4. deb-format.at:511: FAILED (deb-format.at:518) > > > > > > Allowing output on stderr would be a possible tix. > > > > Hmm, that's warning is unfortunate, and a quick check at the xz code > > didn't reveal a way to only suppress it. :/ > > > > For dpkg, I'd like to either not get the warning or being able to tell xz > > that even if the threads are reduced, that's ok and it should not warn > > about that (but that would then require depending on a new enough > > version supporting that, perhaps that could be suppresses instead via > > an envvar) so that? > > Could poke upstream. Thanks! > > Ignoring stderr could be a workaround, but I'd need to do something as > > well for the libdpkg code and the perl code calling xz, which will get > > very annoying. > > > > This is also going to get in the way of migrating both xz and dpkg > > (which seems like would need to be uploaded today for the time64 > > transition). > > > > Or perhaps that warning could be disabled for now in Debian until things > > are sorted out with upstream? > > Falling back to -T1 in order to keep the previous is not an option? I'm not sure I understand what you are saying. Do you mean dpkg would fallback to pass -T1 (maybe you mean -T+1, otherwise we might get unreproducible output due to switching to single-threaded)? And fallback on what condition? Ah, I think you mean to pass -T+1 to the xz invocations for the test suite, right, that could workaround the issue there indeed. Thanks, I think I'll do that for now. > Let me try to sell this "we move this warning to verbose" to upstream in > the meantime… That would actually be great! > > (Had not seen this test suite failure yet, as I've got xz on hold due > > to the breaks on pristine-tar, but would probably stumble over it > > during the release process anyway I guess, so thanks for the heads up!) > > This poped up on xz debci only armel and armhf. Perhaps I'll not see it in my local tree then, but I think the dpkg failure will at least block xz migration, but thinking about it now, probably not dpkg's migration. So this might not be blocking for dpkg. (Sorry if this sounded alarming, but didn't want to add new roadblocks to the time64 transition from the dpkg side! :D) Thanks, Guillem
Bug#1064856: dpkg: New xz-utils print warnings on stderr
On 2024-02-26 19:23:58 [+0100], Guillem Jover wrote: > Hi! Hi Guillem, > > | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the > > memory usage limit of 1400 MiB > > | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the > > memory usage limit of 1400 MiB > > | 89s 4. deb-format.at:511: FAILED (deb-format.at:518) > > > > Allowing output on stderr would be a possible tix. > > Hmm, that's warning is unfortunate, and a quick check at the xz code > didn't reveal a way to only suppress it. :/ > > For dpkg, I'd like to either not get the warning or being able to tell xz > that even if the threads are reduced, that's ok and it should not warn > about that (but that would then require depending on a new enough > version supporting that, perhaps that could be suppresses instead via > an envvar) so that? Could poke upstream. > Ignoring stderr could be a workaround, but I'd need to do something as > well for the libdpkg code and the perl code calling xz, which will get > very annoying. > > This is also going to get in the way of migrating both xz and dpkg > (which seems like would need to be uploaded today for the time64 > transition). > > Or perhaps that warning could be disabled for now in Debian until things > are sorted out with upstream? Falling back to -T1 in order to keep the previous is not an option? Let me try to sell this "we move this warning to verbose" to upstream in the meantime… > (Had not seen this test suite failure yet, as I've got xz on hold due > to the breaks on pristine-tar, but would probably stumble over it > during the release process anyway I guess, so thanks for the heads up!) This poped up on xz debci only armel and armhf. > Thanks, > Guillem Sebastian
Bug#1064856: dpkg: New xz-utils print warnings on stderr
Hi! On Mon, 2024-02-26 at 18:57:32 +0100, Sebastian Andrzej Siewior wrote: > Package: dpkg > Version: 1.22.4 > Severity: important > xz-utils 5.6.0 has been uploaded to unstable. A changed behaviour of > `xz' is now that mutlti threaded compress/ decompression is now enabled > by default. This in turn leads to warnings if the requested amount of > memory exceeds the available amount. A snippet from > > https://ci.debian.net/data/autopkgtest/testing/armel/d/dpkg/43341232/log.gz > > | 88s > /tmp/autopkgtest-lxc.dumkcbm0/downtmp/build.4CO/src/src/at/deb-format.at:518: > | 88s # Extract the base members > | 88s xz -c control.tar >control.tar.xz > | 88s xz -c data.tar >data.tar.xz > | 88s > | 89s --- /dev/null 2024-02-26 09:29:33.669234548 + > | 89s +++ > /tmp/autopkgtest-lxc.dumkcbm0/downtmp/autopkgtest_tmp/src/at/testsuite.dir/at-groups/4/stderr >2024-02-26 09:30:58.601386838 + > | 89s @@ -0,0 +1,2 @@ > | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the > memory usage limit of 1400 MiB > | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the > memory usage limit of 1400 MiB > | 89s 4. deb-format.at:511: FAILED (deb-format.at:518) > > Allowing output on stderr would be a possible tix. Hmm, that's warning is unfortunate, and a quick check at the xz code didn't reveal a way to only suppress it. :/ For dpkg, I'd like to either not get the warning or being able to tell xz that even if the threads are reduced, that's ok and it should not warn about that (but that would then require depending on a new enough version supporting that, perhaps that could be suppresses instead via an envvar) so that? Ignoring stderr could be a workaround, but I'd need to do something as well for the libdpkg code and the perl code calling xz, which will get very annoying. This is also going to get in the way of migrating both xz and dpkg (which seems like would need to be uploaded today for the time64 transition). Or perhaps that warning could be disabled for now in Debian until things are sorted out with upstream? (Had not seen this test suite failure yet, as I've got xz on hold due to the breaks on pristine-tar, but would probably stumble over it during the release process anyway I guess, so thanks for the heads up!) Thanks, Guillem
Bug#1064856: dpkg: New xz-utils print warnings on stderr
Package: dpkg Version: 1.22.4 Severity: important xz-utils 5.6.0 has been uploaded to unstable. A changed behaviour of `xz' is now that mutlti threaded compress/ decompression is now enabled by default. This in turn leads to warnings if the requested amount of memory exceeds the available amount. A snippet from https://ci.debian.net/data/autopkgtest/testing/armel/d/dpkg/43341232/log.gz | 88s /tmp/autopkgtest-lxc.dumkcbm0/downtmp/build.4CO/src/src/at/deb-format.at:518: | 88s # Extract the base members | 88s xz -c control.tar >control.tar.xz | 88s xz -c data.tar >data.tar.xz | 88s | 89s --- /dev/null 2024-02-26 09:29:33.669234548 + | 89s +++ /tmp/autopkgtest-lxc.dumkcbm0/downtmp/autopkgtest_tmp/src/at/testsuite.dir/at-groups/4/stderr 2024-02-26 09:30:58.601386838 + | 89s @@ -0,0 +1,2 @@ | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB | 89s +xz: Reduced the number of threads from 16 to 8 to not exceed the memory usage limit of 1400 MiB | 89s 4. deb-format.at:511: FAILED (deb-format.at:518) Allowing output on stderr would be a possible tix. Sebastian
