Your message dated Sun, 30 Jan 2022 23:44:49 +0100
with message-id <[email protected]>
and subject line Re: Bug#1003536: Fwd: Re: Kernel related problem (randomly
failing tests), where to discuss?
has caused the Debian Bug report #1003536,
regarding iwd: Tests can be dependent on loaded kernel modules
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1003536: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1003536
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: iwd
Version: 1.21-1
Severity: normal
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
When version 1.21-1 was uploaded, the buildd failed for all the release
architecures. Looking into several of them, I found that they failed on
the same test:
"test-dpp: unit/test-dpp.c:114: test_key_derivation: Assertion `m' failed."
I went then on OFTC:/#iwd to ask whether they'd (instantly) know what
may have caused it. While an immediate cause wasn't found, it did bring
some interesting aspects to light.
One of the upstream maintainers ran a build and reported that 22 tests ran
and they all succeeded. On buildd 1 test out of *20* failed.
One of the great features (IMO) of iwd is that it reuses functionality
available in the kernel. But that also means it is dependent on the
kernel configuration.
In Debian kernels the rule is to build kernel modules as loadable
modules (=m) as much as possible and not builtin (=y).
One of the triggers for a kernel module to get loaded is when the kernel
detects a certain piece of hardware present on the system, like a soundcard
or a HWRNG. AFAIK this works quite well.
But it also means that the loaded kernel modules differ from machine to
machine as their hardware differs. Or in case of Virtual Machines, which
(virtual) hardware is configured in that VM.
Another trigger (AFAIK) is when a certain process needs a certain kernel
module, it gets loaded (ondemand).
And the user also has the possibility to explicitly load kernel modules,
either through a direct `modprobe` or via `/etc/modules` (f.e.)
All this means that a test on my laptop with 5.15.0-2-amd kernel may
succeed, while that same test may fail on my PC with the same kernel.
For example I noticed SHA512 kernel modules were loaded on my laptop,
while they were not on my PC.
The buildds run Debian Stable with a 5.10.0-10 kernel, which is already
different, but its hardware is very likely also different.
Different architectures also have different kernel configurations and
it's quite likely that caused the same test to succeed for several other
(non-release) architectures. And as they're different machines, the
hardware is also different and therefor the loaded kernel modules as
well.
It can also mean that a build succeeds or fails because of other things
that happen concurrently or previously on *that same buildd machine*,
just because another build/process triggered a certain kernel module to
be loaded, which may stay loaded until that buildd machine gets rebooted.
The (high) preference is that builds are reproducible and not dependent
on variable hardware/kernel config/other builds that may have happened.
One possible solution could be making sure certain kernel modules are
loaded before all or a certain test is run. This page may be helpful:
https://iwd.wiki.kernel.org/gettingstarted#kernel_dependencies
This would need to be implemented/applied on the Debian side as Debian
has its own kernel configuration (per architecture).
Another possibility is to conditionally run a test based on whether a
kernel module is loaded. Note that checking `lsmod` is insufficient as
a kernel module can be builtin, like CONFIG_CRYPTO_SHA256 is.
I don't know what's possible or desirable to address the issues I've
raised above (and there may be other issues as well), but I hope this
bug report is a useful start for such a discussion.
Cheers,
Diederik
PS: Based on the discussion I had on IRC, the following patch was
proposed (and applied) upstream:
https://lists.01.org/hyperkitty/list/[email protected]/thread/USXHQD27OI7RJEC52MPJLNGQCKEBBJKO/
- -- System Information:
Debian Release: bookworm/sid
APT prefers testing
APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 5.15.0-2-amd64 (SMP w/4 CPU threads)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8),
LANGUAGE=en_US:en
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages iwd depends on:
ii init-system-helpers 1.61
ii libc6 2.33-2
ii libell0 0.47-1
ii libreadline8 8.1.2-1
Versions of packages iwd recommends:
ii dbus [dbus-system-bus] 1.12.20-3
ii wireless-regdb 2021.08.28-1
iwd suggests no packages.
- -- no debconf information
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQT1sUPBYsyGmi4usy/XblvOeH7bbgUCYd2ktQAKCRDXblvOeH7b
bs5+AP951nPaCbTer/pyoPKWm4ffryUoJ76T684YM8N7Yjdl3gEAmPrWnBwVggsC
2UUHABQGjE8ba9T0Xt0vEROFalDW9QY=
=X4I3
-----END PGP SIGNATURE-----
--- End Message ---
--- Begin Message ---
Quoting Diederik de Haas (2022-01-26 16:12:13)
> I asked on the debian-kernel ML about this and got the following
> response:
[...]
> From: Ben Hutchings <[email protected]>
[...]
> You shouldn't run any tests like this at build time.
>
> For autopkgtests, if you set the needs-root and isolation-machine
> restrictions then the tests will run as root on a VM. But currently
> neither salsa-ci nor ci.debian.net implements this, so those tests
> will be skipped.
>
> Another option in autopkgtests is to depend on qemu and start the VM
> yourself. This is not easy to do, but I implemented it for initramfs-
> tools.
Thanks, Diederik, for your thorough investigation, and thanks, Ben, for
the suggestions.
I choose to consider this not an issue for the Debian packaging of IWD,
and close this bugreport.
Feel free to disagree, and feel welcome to continue posting comments to
this bugreport: a closed bugreport is still open for discussion (for
some time - and later locking is done to avoid spam, unlocking remain an
option).
- Jonas
--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/
[x] quote me freely [ ] ask before reusing [ ] keep private
signature.asc
Description: signature
--- End Message ---
