Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Cyril Brulebois]

Marco d'Itri  (2024-04-26):
> On Apr 26, Michael Tokarev  wrote:
> 
> > So, should I disable module utils in busybox-udeb now?
> I think so.

I haven't gotten any requests / seen any reasons to keep it; so, yes,
please feel free to remove it whenever is convenient for you.

> > Is kmod udeb ready and used in d-i already, or does it need some
> > prep first?
> AFAIK it works.

Absolutely, that's been working since the small xz-utils tweak (the udeb
addition, not the backdoor thing).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Michael Tokarev]

Ok,

I'm removing whole modutils from busybox udeb (besides depmod, this is
lsmod, insmod, rmmod, and modprobe).  All these are provided by
kmod-udeb as far as I can see (as symlinks to kod).

--
GPG Key transition (from rsa2048 to rsa4096) since 2024-04-24.
New key: rsa4096/61AD3D98ECDF2C8E  9D8B E14E 3F2A 9DD7 9199  28F1 61AD 3D98 
ECDF 2C8E
Old key: rsa2048/457CE0A0804465C5  6EE1 95D1 886E 8FFB 810D  4324 457C E0A0 
8044 65C5
Transition statement: http://www.corpit.ru/mjt/gpg-transition-2024.txt

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Marco d'Itri]

On Apr 26, Michael Tokarev  wrote:

> So, should I disable module utils in busybox-udeb now?
I think so.

> Is kmod udeb ready and used in d-i already, or does it need some
> prep first?
AFAIK it works.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Michael Tokarev]

09.04.2024 16:48, Cyril Brulebois wrote:

Marco d'Itri  (2024-04-09):

Yes. Nowadays kmod has many more features related to compressed modules
and verification of signatures.
Can we agree that kmod should provide these programs for d-i?
Or can the d-i maintainers just tell us what they want?


I meant to come back to this after experimenting, then things happened…

I picked kmod at the time because it worked, and because busybox didn't
work, which I summed up in:
   
https://salsa.debian.org/installer-team/debian-installer/-/commit/450daf0bd24ee94d4f466ab65908c079ef795145

(plus follow-up commit, woopsie
   
https://salsa.debian.org/installer-team/debian-installer/-/commit/69777be465c5d0210d16159a456ab88535513a07
)

I'm fine with sticking to kmod regarding module support in d-i. I'm not
sure we should keep support in two different modules, so dropping it
from busybox would work for me. Others might have different views on
this, though.


So, should I disable module utils in busybox-udeb now?
Wanted to spare some time on busybox, this bug report come in.

Is kmod udeb ready and used in d-i already, or does it need some
prep first?

Thanks,

/mjt

--
GPG Key transition (from rsa2048 to rsa4096) since 2024-04-24.
New key: rsa4096/61AD3D98ECDF2C8E  9D8B E14E 3F2A 9DD7 9199  28F1 61AD 3D98 
ECDF 2C8E
Old key: rsa2048/457CE0A0804465C5  6EE1 95D1 886E 8FFB 810D  4324 457C E0A0 
8044 65C5
Transition statement: http://www.corpit.ru/mjt/gpg-transition-2024.txt

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Cyril Brulebois]

Hi,

Marco d'Itri  (2024-04-09):
> Yes. Nowadays kmod has many more features related to compressed modules 
> and verification of signatures.
> Can we agree that kmod should provide these programs for d-i?
> Or can the d-i maintainers just tell us what they want?

I meant to come back to this after experimenting, then things happened…

I picked kmod at the time because it worked, and because busybox didn't
work, which I summed up in:
  
https://salsa.debian.org/installer-team/debian-installer/-/commit/450daf0bd24ee94d4f466ab65908c079ef795145

(plus follow-up commit, woopsie
  
https://salsa.debian.org/installer-team/debian-installer/-/commit/69777be465c5d0210d16159a456ab88535513a07
)

I'm fine with sticking to kmod regarding module support in d-i. I'm not
sure we should keep support in two different modules, so dropping it
from busybox would work for me. Others might have different views on
this, though.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Marco d'Itri]

On Jan 06, Michael Tokarev  wrote:

> Yes, some utils in busybox aren't as good as regular implementations. For
Yes. Nowadays kmod has many more features related to compressed modules 
and verification of signatures.
Can we agree that kmod should provide these programs for d-i?
Or can the d-i maintainers just tell us what they want?

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Michael Tokarev]

06.01.2024 11:40, Helmut Grohne:

On Sat, Jan 06, 2024 at 09:01:12AM +0100, Helmut Grohne wrote:

I also recommend to establish QA for all udebs to automatically detect,
report and address such conflicts as they evidently cause undefined
behaviour otherwise. That can be as simple as collecting file lists of
all udebs and comparing them.


This seems like a more generic problem. I downloaded all amd64 udebs and
the following files (normalized to account for aliasing) pose a
conflict:


From this list, only a few utilities are from busybox, namely wget and module
utilities (depmod/insmod/lsmod/modinfo/modprobe/rmmod).

My initial plan - with regular busybox package and with busybox udeb - is to
provide most things in busybox, so that other packages don't need to ship
udeb packages and the whole thing (be it d-i or initrd) is small.

Yes, some utils in busybox aren't as good as regular implementations. For
example, I just found out busybox's xz does not perform compression, only
decompression (-d option is mandatory).  Or #1003757 - missing functionality
in busybox ip.  Still, overall, it is enough for most things.  BTW, it looks
like with compressed kernel modules, busybox m-i-t needs some (albiet minor)
tweaks (it works but kernel produces warnings when busybox tries to load a
module).

Unfortunately this didn't work out for one reason or another.  One of the
reasons is perhaps #921556, where original util does more than needed but
busybox didn't implement the unnecessary functionality.

This needs to be thought about at a more general level. Including initrd
stuff (if we still need it, instead of relying on mkosi-initrd).  I use
my own initrd for a good reason, and this one does not include 2 or even
3 libc as debian does..

/mjt

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Helmut Grohne]

On Sat, Jan 06, 2024 at 09:01:12AM +0100, Helmut Grohne wrote:
> I also recommend to establish QA for all udebs to automatically detect,
> report and address such conflicts as they evidently cause undefined
> behaviour otherwise. That can be as simple as collecting file lists of
> all udebs and comparing them.

This seems like a more generic problem. I downloaded all amd64 udebs and
the following files (normalized to account for aliasing) pose a
conflict:

./bin/netcfg
./bin/ptom
./bin/wget
./etc/console-setup/remap.inc
./lib/base-installer.d/40netcfg
./lib/finish-install.d/55netcfg-copy-config
./sbin/depmod
./sbin/insmod
./sbin/lsmod
./sbin/modinfo
./sbin/modprobe
./sbin/rmmod
./usr/share/console-setup/charmap_functions.sh
./usr/share/fonts/truetype/noto/NotoSansSinhala-Bold.ttf
./usr/share/fonts/truetype/noto/NotoSansSinhala-Regular.ttf
./usr/share/fonts/truetype/noto/NotoSerifGujarati-Bold.ttf
./usr/share/fonts/truetype/noto/NotoSerifGujarati-Regular.ttf
./usr/share/keymaps/i386/qwerty/it.kmap.gz
./usr/share/keymaps/i386/qwerty/jp106.kmap.gz
./usr/share/keymaps/i386/qwertz/mac-usb-de-latin1-nodeadkeys.kmap.gz
./usr/share/keymaps/i386/qwertz/mac-usb-de-latin1.kmap.gz
./usr/share/keymaps/i386/qwertz/mac-usb-de_CH.kmap.gz
./usr/share/keymaps/i386/qwertz/mac-usb-fr_CH-latin1.kmap.gz
./usr/share/keymaps/i386/qwertz/mac-usb-pt-latin1.kmap.gz
./usr/share/keymaps/include/compose.latin1.inc.gz

So for busybox and kmod, it's not just depmod, but all the others as
well. It's also busybox-udeb vs wget-udeb and a bunch of others though.
All of the aliased ones will explode sooner or later.

Helmut

Bug#1060134: kmod-udeb vs busybox-udeb: agree on who ships depmod

[Helmut Grohne]

Package: kmod-udeb,busybox-udeb
Severity: serious
Justification: file conflict
X-Debbugs-Cc: Cyril Brulebois , debian-b...@lists.debian.org

Hi Cyril,

On Sat, Jan 06, 2024 at 04:31:44AM +0100, Cyril Brulebois wrote:
> d-i daily builds now FTBFS everywhere due to the merge-usr step, with
> the following error:
> 
> merge-usr "./tmp/cdrom/tree"
> error: merge target 'usr//sbin/depmod' is a symlink
> 
> It would be nice if you could investigate and suggest what fix(es) would
> be best here. I suppose it could be some fallout from the kmod upload
> which included some changes of yours, hence the ping.
> 
>   
> https://tracker.debian.org/news/1491892/accepted-kmod-31-1-source-into-unstable/

Thank you for the report. In investigating it, I note that reproducing
the failure was unexpectedly difficult. debian-installer fails to build
from source in unstable for unrelated reasons. I request that you update
the package more frequently and keep it in a buildable state to ease QA
efforts of others.

It turns out that /sbin/depmod points to /bin/busybox and
/usr/sbin/depmod points to ../bin/kmod. As such, merge-usr does not have
a good idea of what to do and rightly refuses to continue. These two
files are to be considered a file conflict according to the Debian
policy as they only differ in aliasing. I argue that this was a bug
before merged-/usr and merged-/usr just makes it visible now. The
underlying conflict has existed earlier and was resolved arbitrarily
dependent on the order of unpacks.

The real solution here is to agree on whether busybox-udeb or kmod-udeb
is supposed to provide depmod and only ship it once. Since there is no
other consumer of udebs, it does not make sense to install it in both.

Cyril, please reassign the bug to the right package (i.e. the one that
should not be providing depmod).

I also recommend to establish QA for all udebs to automatically detect,
report and address such conflicts as they evidently cause undefined
behaviour otherwise. That can be as simple as collecting file lists of
all udebs and comparing them.

Helmut