Bug#1032831: #1032831 broken loginscreen when switching virtual console forth and back to tty7 out of an active kde session

[Holger Wansing]

Control: reassign -1 sddm


On Sun, 12 Mar 2023 13:01:27 +0100, digital...@gmx.de wrote:
> Package: installation-reports
> 
> 
> (Please provide enough information to help the Debian
> maintainers evaluate the report efficiently - e.g., by filling
> in the sections below.)
> 
> Boot method: USB
> Image version: 
> https://cdimage.debian.org/cdimage/bookworm_di_alpha2/amd64/iso-cd/
> debian-bookworm-DI-alpha2-amd64-netinst.iso
> Date: <06.03.2023>
> 
> Machine: its an acer travel mate 7750G laptop but i have this problem also on 
> a getac s410
> laptop
> Partitions: 
> 
> 
> Base System Installation Checklist:
> [O] = OK, [E] = Error (please elaborate below), [ ] = didn't try it
> 
> Initial boot:   [O]
> Detect network card:[O]
> Configure network:  [O]
> Detect media:   [O]
> Load installer modules: [O]
> Clock/timezone setup:   [O]
> User/password setup:[O]
> Detect hard drives: [O]
> Partition hard drives:  [O]
> Install base system:[O]
> Install tasks:  [O]
> Install boot loader:[O]
> Overall install:[E]
> 
> Comments/Problems:
> 
> I did two new installations of debian 12 on two different laptops(listed 
> above), this
> problem i have on both on them.
> If i start my machine i get the normal login screen of sddm on tty7.
> Then i can login and the kde loading screen start and after that i have a 
> normal kde
> session.
> But if i then change witch strg alt f2(for example) the virtual console and 
> back to tty7 i am
> logged out of my active kde session and get a sddm login screen with a 
> "pre-typed greyed"
> password.Its the same image just before the KDE loading screen appears after 
> you type
> your pw and hit enter like on a normal login.
> 
> I can only move my mouse around and pressing any key does nothing,no 
> interaction with
> the login screen is possible, but i can still switch tty back to like tty2.



digital...@gmx.de replied:
> If i choose in the sddm login Plasma (X11) this Problem does not appear but if
> i change it to Plasma (Wayland) the problem is just a described.



-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Processed: Re: #1032831 broken loginscreen when switching virtual console forth and back to tty7 out of an active kde session

[Debian Bug Tracking System]

Processing control commands:

> reassign -1 sddm
Bug #1032831 [installation-reports] broken loginscreen when switching virtual 
console forth and back to tty7 out of an active kde session
Bug reassigned from package 'installation-reports' to 'sddm'.
Ignoring request to alter found versions of bug #1032831 to the same values 
previously set
Ignoring request to alter fixed versions of bug #1032831 to the same values 
previously set

-- 
1032831: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1032831
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1076099: debian-installer: LVM's activation broken for volumes with the dm-integrity feature

[Cyril Brulebois]

Hi Franco,

Franco  (2024-07-10):
> In a VirtualBox virtual machine I've converted a LVM volume with the 
> following options:
> 
> ~# lvconvert --raidintegrity y --raidintegritymode bitmap ...
> 
> When I boot in rescue-mode that machine using the
> "debian-12.5.0-amd64-DVD-1.iso" iso, I follow all configuration steps
> until D-I asks me which device I want to mount as root filesystem (for
> running a shell into), then it happens that the volume is shown but if
> selected an error message it appears on the screen (red) that it says:
> 
> ---
> No such device
> The device you entered for your root file system (/dev/vg0/lv1) does not 
> exist. Please try again
> ---
> 
> Then I choose to run a shell without mount a root filesystem and I try to 
> manually activate the LVM's volume, but it fails with this error message:
> 
> ~# vgchange -a y vg0
> modprobe: FATAL: Module dm-integrity not found in directory 
> /lib/modules/6.1.0-18-amd64
>   /sbin/modprobe failed: 1
>   Can't process LV vg0/lv1_rimage_0: integrity target support missing from 
> kernel?
>   0 logical volume(s) in volume group "vg0" now active
> 
> The kernel version of D-I is:
> 
> ~# uname -a
> Linux bw12 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) 
> x86_64 GNU/Linux
> 
> I suggest to add "dm-integrity" module to kernel image enabling the
> configuration entry CONFIG_DM_INTEGRITY=m and possibly add the
> "dm-integrity" module's name to /etc/initramfs-tools/modules file in
> order to automatically activate at boot time the LVM volumes with the
> integrity feature enabled.

The first bit has been here for a very long while (linux 4.17.3-1,
back in 2018), but we would need dm-integrity to be added to linux's
debian/installer/modules/md-modules for that to become available in
the installer.

Cc-ing the kernel team for the time being, I have no clue whether it
makes sense to have that included for everyone, or if that's solving
a corner case (as far as I understand it you enabled a feature that's
not turned on by default). {Making,Keeping} rescue mode {more,} useful
is a worthy goal but it's really not meant to compete with a dedicated
rescue-oriented image (or even general-purpose images like live ones).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1076099: debian-installer: LVM's activation broken for volumes with the dm-integrity feature

[Franco]

Source: debian-installer
Version: 20230607+deb12u6
Severity: normal
Tags: d-i
X-Debbugs-Cc: martelli...@gmail.com

Dear Maintainer,

In a VirtualBox virtual machine I've converted a LVM volume with the following 
options:

~# lvconvert --raidintegrity y --raidintegritymode bitmap ...

When I boot in rescue-mode that machine using the 
"debian-12.5.0-amd64-DVD-1.iso" iso, I follow all configuration steps until D-I 
asks me which device I want to mount as root filesystem (for running a shell 
into), then it happens that the volume is shown but if selected an error 
message it appears on the screen (red) that it says:

---
No such device
The device you entered for your root file system (/dev/vg0/lv1) does not exist. 
Please try again
---

Then I choose to run a shell without mount a root filesystem and I try to 
manually activate the LVM's volume, but it fails with this error message:

~# vgchange -a y vg0
modprobe: FATAL: Module dm-integrity not found in directory 
/lib/modules/6.1.0-18-amd64
  /sbin/modprobe failed: 1
  Can't process LV vg0/lv1_rimage_0: integrity target support missing from 
kernel?
  0 logical volume(s) in volume group "vg0" now active

The kernel version of D-I is:

~# uname -a
Linux bw12 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) 
x86_64 GNU/Linux

I suggest to add "dm-integrity" module to kernel image enabling the 
configuration entry CONFIG_DM_INTEGRITY=m and possibly add the "dm-integrity" 
module's name to /etc/initramfs-tools/modules file in order to automatically 
activate at boot time the LVM volumes with the integrity feature enabled.

Possible workaround: use a live iso (debian-live-12.5.0-amd64-kde.iso) and 
manually activate the LVM volume with the command:

~# vgchange -a y volume_group_name

Cheers



-- System Information:
Debian Release: 12.6
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 
'stable'), (100, 'bookworm-fasttrack'), (100, 'bookworm-backports-staging')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.90 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=it_IT.UTF-8, LC_CTYPE=it_IT.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)

Bug#783950: marked as done (d-i.debian.org: PO spellchecking broken)

[Debian Bug Tracking System]

Your message dated Sat, 11 May 2024 11:20:22 +0200
with message-id <3ee6f845-bcb8-4c4f-859a-e7ce46bfe...@mailbox.org>
and subject line Re: d-i.debian.org: PO spellchecking broken
has caused the Debian Bug report #783950,
regarding d-i.debian.org: PO spellchecking broken
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
783950: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=783950
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: d-i.debian.org
Severity: normal

The set of scripts that generate D-I l10n spellchecking statistics are
apparently broken since the d-i.debian.org host was upgraded to
Jessie. Output follows.

The master script that generates the following is:
d-i@dillon:/srv/d-i.debian.org/home/l10n/material/packages.cvs1/debian-installer$
 cat ~/bin/d-i_spellcheck 
#!/bin/sh
cd ${HOME}/trunk/scripts/l10n/l10n-spellcheck/cfg
svn -q update
./update_po.sh
for i in man l1 l2 l3 ; do
#echo Spellchecking level $i
./scheck.sh $i
done

Sadly, I'm not sure that anyone but Davide Viti who wrote the original
set of scripts, can fix this...:-)

- Forwarded message from Cron Daemon  -

Date: Thu, 30 Apr 2015 03:21:51 +
From: Cron Daemon 
To: bubu...@dillon.debian.org, k...@dillon.debian.org
Subject: Cron  $HOME/bin/d-i_spellcheck ; ~/bin/push-www
X-CRM114-Status: Good  ( pR: 14.9657 )

Error: The language "el" is not known. This is probably because: the file 
"/usr/share/aspell/el.dat" can not be opened for reading.
Error: The file "/tmp/fileJxXALD" is not in the proper format.
Error: The language "fi" is not known. This is probably because: the file 
"/usr/share/aspell/fi.dat" can not be opened for reading.
Error: No word lists can be found for the language "fi".
Error: The language "hu" is not known. This is probably because: the file 
"/usr/share/aspell/hu.dat" can not be opened for reading.
Error: The file "/tmp/file3Sl4iA" is not in the proper format.
Error: The language "pt_PT" is not known. This is probably because: the file 
"/usr/share/aspell/pt_PT.dat" can not be opened for reading.
Error: The file "/tmp/fileUIdsQ1" is not in the proper format.
Error: The language "ro" is not known. This is probably because: the file 
"/usr/share/aspell/ro.dat" can not be opened for reading.
Error: The file "/tmp/file35GqWl" is not in the proper format.
Error: The language "ru" is not known. This is probably because: the file 
"/usr/share/aspell/ru.dat" can not be opened for reading.
Error: The file "/tmp/fileonFXd8" is not in the proper format.
Error: The language "sv" is not known. This is probably because: the file 
"/usr/share/aspell/sv.dat" can not be opened for reading.
Error: The file "/tmp/fileloE2Q7" is not in the proper format.
Error: The language "tl" is not known. This is probably because: the file 
"/usr/share/aspell/tl.dat" can not be opened for reading.
Error: The file "/tmp/filebVYCcl" is not in the proper format.
Error: The language "en" is not known. This is probably because: the file 
"/usr/share/aspell/en.dat" can not be opened for reading.
Error: The file "/tmp/fileudTxQU" is not in the proper format.
Error: The language "am" is not known. This is probably because: the file 
"/usr/share/aspell/am.dat" can not be opened for reading.
Error: The file "/tmp/fileMBYqZb" is not in the proper format.
Error: The language "ar" is not known. This is probably because: the file 
"/usr/share/aspell/ar.dat" can not be opened for reading.
Error: The file "/tmp/filepXr5H3" is not in the proper format.
Error: The language "bg" is not known. This is probably because: the file 
"/usr/share/aspell/bg.dat" can not be opened for reading.
Error: The file "/tmp/fileEiMT0M" is not in the proper format.
Error: The language "bn" is not known. This is probably because: the file 
"/usr/share/aspell/bn.dat" can not be opened for reading.
Error: The file "/usr/lib/aspell/bn.rws" is not in the proper format. 
Incompatible hash function.
Error: The language "ca" is not known. This is probably because: the file 
"/usr/share/aspell/ca.dat" can not be opened for reading.
Error: The file "/tmp/fileqIP8Md" is not in the proper format.
Error: The langu

Re: Bug#1066071: mtdev: broken shlibs, leading to uninstallable udebs

[Cyril Brulebois]

Hi,

Cyril Brulebois  (2024-03-12):
> Your NMU broke this package's shlibs.
> 
> Before:
> 
> libmtdev 1 libmtdev1
> udeb: libmtdev 1 libmtdev1-udeb
> 
> After:
> 
> libmtdev 1 libmtdev1t64
> 
> At the moment, at least the following package is broken:
> 
> The following packages have unmet dependencies:
>  libinput10-udeb : Depends: libmtdev1t64 but it is not installable

No response in 1+ month, the package isn't ready to migrate anyway since
it's waiting on dpkg but also regressing on 32-bit arms.

Source debdiff attached for the NMU, which I've verified to generate
appropriate shlibs, leading a rebuilt src:libinput10 to have appropriate
dependencies as far as libinput10-udeb is concerned.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
diff -Nru mtdev-1.1.6/debian/changelog mtdev-1.1.6/debian/changelog
--- mtdev-1.1.6/debian/changelog	2024-02-28 21:51:50.0 +0100
+++ mtdev-1.1.6/debian/changelog	2024-04-15 09:51:44.0 +0200
@@ -1,3 +1,12 @@
+mtdev (1.1.6-1.2) unstable; urgency=medium
+
+  * Non-maintainer upload.
+  * Fix shlibs entry for the udeb: set DEB_DH_MAKESHLIBS_ARGS_libmtdev1t64
+instead of setting DEB_DH_MAKESHLIBS_ARGS_libmtdev1, to match the
+rename of the library (Closes: #1066071).
+
+ -- Cyril Brulebois   Mon, 15 Apr 2024 09:51:44 +0200
+
 mtdev (1.1.6-1.1) unstable; urgency=medium
 
   * Non-maintainer upload.
diff -Nru mtdev-1.1.6/debian/rules mtdev-1.1.6/debian/rules
--- mtdev-1.1.6/debian/rules	2020-05-24 06:38:08.0 +0200
+++ mtdev-1.1.6/debian/rules	2024-04-15 09:50:23.0 +0200
@@ -9,7 +9,7 @@
 distribution	:= $(shell lsb_release -is)
 LDFLAGS += -Wl,-z,defs -Wl,--as-needed
 
-DEB_DH_MAKESHLIBS_ARGS_libmtdev1 = --add-udeb=libmtdev1-udeb
+DEB_DH_MAKESHLIBS_ARGS_libmtdev1t64 = --add-udeb=libmtdev1-udeb
 
 DEB_INSTALL_MANPAGES_mtdev-tools = debian/mtdev-test.1
 


signature.asc
Description: PGP signature

Bug#923091: marked as done (base-installer: Allow installing w/o the broken merged-usr-via-symlinks)

[Debian Bug Tracking System]

Your message dated Tue, 19 Mar 2024 12:31:36 +0100
with message-id <20240319113136.w3x36ynyk4jjv...@shell.thinkmo.de>
and subject line Re: Bug#923091: base-installer: Allow installing w/o the 
broken merged-usr-via-symlinks
has caused the Debian Bug report #923091,
regarding base-installer: Allow installing w/o the broken 
merged-usr-via-symlinks
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
923091: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923091
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: base-installer
Version: 1.187
Severity: wishlist

Hi!

The current base-installer uses the default debootstrap settings
which end up unconditionally installing systems with the
merged-usr-via-symlinks deployment method which is broken by design,
please see:

  <https://lists.debian.org/debian-devel/2019/02/msg00251.html>

I'm aware the original request to change the debootstrap default got
unfortunately moved to the tech-ctte. :(

But regardless of that outcome I'd like to request to have a way to
install using debootstrap's --no-merged-usr option, to not have to
install from stretch and then upgrade to buster, or having to drop into
a shell and do manual stuff from within the installer.

In addition it would be also nice if that option was passed whenever
/usr is not on its own partition, because then the properties from
the merged-/usr concept are not relevant anymore, but we get all the
downsides of the broken deployment method.

If this was to be applied for buster, I'd be happy to provide patches.

Otherwise I guess I might need to end up looking to generate
alternative netinst somewhere else or something. :(

Thanks,
Guillem
--- End Message ---
--- Begin Message ---
On Sun, Feb 24, 2019 at 03:10:17AM +0100, Guillem Jover wrote:
> I'm aware the original request to change the debootstrap default got
> unfortunately moved to the tech-ctte. :(

And ctte made an decision.  So there is no bug here.  Closing.

Bastian

-- 
Violence in reality is quite different from theory.
-- Spock, "The Cloud Minders", stardate 5818.4--- End Message ---

Bug#1066074: ntfs-3g: broken shlibs (deb and udeb)

[Cyril Brulebois]

Source: ntfs-3g
Version: 1:2022.10.3-1.1
Severity: serious
Tags: d-i
Justification: broken shlibs
X-Debbugs-Cc: debian-boot@lists.debian.org, Benjamin Drung 

Hi,

Here's what debian/libntfs-3g89t64/DEBIAN/shlibs looks like after
a build:

libntfs-3g 89 libntfs-3g89
udeb: libntfs-3g 89 libntfs-3g89

That doesn't match binaries shipped by the source package.


See debian/rules:

SONAME = $(shell objdump -p debian/tmp/lib/*/libntfs-3g.so.*.* | awk -Fso. 
'/SONAME/ { print $$2 }')

[…]

override_dh_makeshlibs:
dh_makeshlibs --add-udeb=ntfs-3g-udeb -Vlibntfs-3g$(SONAME)


In the previous version we had:

libntfs-3g 89 libntfs-3g89
udeb: libntfs-3g 89 ntfs-3g-udeb

Adding 't64' at the end of the dh_makeshlibs line quoted above gives:

libntfs-3g 89 libntfs-3g89t64
udeb: libntfs-3g 89 ntfs-3g-udeb

which certainly looks much better. I haven't performed any rebuild test
for the reverse dependencies of the library, nor runtime tests on the
d-i side (other packages are broken right now). The Depends field of
the udeb looks correct now though:

-Depends: libc6-udeb (>= 2.37), libntfs-3g89, fuse3-udeb
+Depends: libc6-udeb (>= 2.37), fuse3-udeb


I'll leave it up to the 64-bit time_t transition drivers to choose how
to address this issue (add t64 on the SONAME line, or just in the
dh_makeshlibs override, or something else), and to track down packages
that might have been rebuilt against the broken library.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#1066071: mtdev: broken shlibs, leading to uninstallable udebs

[Cyril Brulebois]

Source: mtdev
Version: 1.1.6-1.1
Severity: serious
Tags: d-i
Justification: broken shlibs
X-Debbugs-Cc: debian-boot@lists.debian.org, Benjamin Drung 

Hi,

Your NMU broke this package's shlibs.

Before:

libmtdev 1 libmtdev1
udeb: libmtdev 1 libmtdev1-udeb

After:

libmtdev 1 libmtdev1t64

At the moment, at least the following package is broken:

The following packages have unmet dependencies:
 libinput10-udeb : Depends: libmtdev1t64 but it is not installable


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#731859: Still broken

[Ken Sharp]

variant=fakechroot still fails in testing, but it apparently fails in a 
different way. (Do I need a new bug report?)


`fakeroot fakechroot debootstrap --variant=fakechroot testing testing` 
fails.


Whereas `fakeroot fakechroot debootstrap --variant=minbase testing 
testing` succeeds.


But at this point I have to ask what the purpose of variant=fakechroot 
is in the first place, as it can only be run with `fakeroot fakechroot` 
anyway.


$ debootstrap --variant=fakechroot testing testing
E: debootstrap can only run as root

$ fakeroot debootstrap --variant=fakechroot testing testing 


E: This variant requires fakechroot environment to be started

So the only way to run a debootstrap is to either use `fakeroot 
fakechroot` with variant=minbase, or as root.


debootstrap 1.0.134


OpenPGP_signature.asc
Description: OpenPGP digital signature

Bug#772523: preseeding get_domain using DHCP is broken

[Raphaël Halimi]

(sorry, accidental Ctrl+Enter...)

With Bookworm, it totally broke.

If the preseeding happens after netcfg (url=...), when setting the 
hostname from the kernel parameters, d-i keeps it, but does not get the 
domain from DHCP as before; only setting both a hostname and a domain 
name makes things work (but now keeps the domain from the kernel 
parameters, which may be more appropriate).


If the preseeding happens before netcfg (file=...), whatever hostname 
and/or domain is set in the kernel paramaters, and whatever domain the 
DHCP sends, d-i uses the values from the preseed file 
(unassigned-hostname and unassigned-domain).


So, currently, the only way to make things work with an installation 
medium, is to unset both netcfg/get_hostname and netcfg/get_domain in 
the preseed file, and set hostname and domain in the kernel parameters.


(for convenience, I attached a text file in markdown format with the 
results of these tests).


This is a big change from the behavior of previous netcfg versions (I 
use preseeding since Wheezy, both with PXE or installation media), and 
this new behavior makes things more complicated: before, we just 
configured bogus values in the preseed file, and if the machine was not 
registered in the DNS but the DHCP provided a valid domain name, 
specifying a hostname in the kernel parameters was sufficient. Now, we 
have to specify both the hostname and the domain name in the kernel 
command line (think of non-QWERTY keyboards...), and this makes me 
consider this a severe regression.


It also partly contradicts the various documentations (like the ones 
mentioned above).


I sincerely hope this will be fixed in a forthcoming point release.

Feel free to ask me if you need to test stuff, I have a suitable 
environment to test preseeding for both PXE or installation media.


Best regards,

--
Raphaël HalimiBullseye


netcfg before preseed (url=...)


### cmdline: none
hostname=debian, no domain name

### cmdline: hostname=
hostname from cmdline, domain from DHCP

### cmdline: hostname=, domain=
hostname from cmdline, domain from DHCP (different from cmdline)

preseed before netcfg (file=...)


### cmdline: none
hostname and domain from preseed file (unassigned-hostname, unassigned-domain)

### cmdline: hostname=
hostname from cmdline, domain from DHCP

### cmdline: hostname=, domain=
hostname from cmdline, domain from DHCP (different from cmdline)

Bookworm


netcfg before preseed (url=...)


### cmdline: none
hostname=debian, no domain name

### cmdline: hostname=
hostname from cmdline, no domain

### cmdline: hostname=, domain=
hostname and domain from cmdline

preseed before netcfg (file=...)


### cmdline: none
hostname and domain from preseed file (unassigned-hostname, unassigned-domain)

### cmdline: hostname=
hostname and domain from preseed file (unassigned-hostname, unassigned-domain)

### cmdline: hostname=, domain=
hostname and domain from preseed file (unassigned-hostname, unassigned-domain)

Bug#772523: preseeding get_domain using DHCP is broken

[Raph]

Dear developers,

I confirm that something broke between Bullseye and Bookworm (IIRC it 
worked even with Bookworm RC2) regarding netcfg's behavior when the 
host's IP address can't be found in the DNS.


I also think it's a different bug from the one originally reported 
(which were for Jessie). Maybe a new one should be created for this big 
regression in Bookworm.


The behavior also changes according to when the preseeding happens: 
before netcfg (when installation is performed from an installation media 
and the preseed file is specified with kernel parameter file=...) or 
after netcfg (when installation is performed from a PXE server and the 
preseed file is specified with kernel parameter url=...).


Note that when IP address (assigned by DHCP) is found in the DNS, 
everything works as before, whether netcfg is performed before or after 
the preseeding.


I performed a number of tests with a VM and the IP address missing from 
the DNS, and the preseed file containing "d-i netcfg/get_hostname string 
unassigned-hostname" and "d-i netcfg/get_domain string 
unassigned-domain" (as suggested in [1]).


[1] https://www.debian.org/releases/bookworm/example-preseed.txt

With Bullseye, if the preseeding happens after netcfg (url=...), without 
hostname or domain kernel parameters, d-i chooses "debian" as hostname 
and sets no domain name. With a hostname set in kernel parameters, d-i 
keeps it and sets the domain name from DHCP. With both hostname and 
domain set in kernel parameters, d-i keeps the hostname, but still sets 
the domain name from DHCP, ignoring the one provided in kernel parameters.


If the preseeding happens before netcfg (file=...), without hostname or 
domain kernel parameters, d-i uses the ones from the preseed file 
(unassigned-hostname and unassigned-domain). When hostname or hostname 
and domain are specified in kernel parameters, it behaves as above 
(hostname from kernel parameters and domain from DHCP).


This behavior is consistent with the aforementioned example preseed 
file, and also with documentation in [2] (and particularly B.2.3, "Auto 
mode").


[2] https://www.debian.org/releases/stable/i386/apbs02.en.html

With Bookworm, everything











--
Raph

Bug#772523: [Re] preseeding get_domain using DHCP is broken

[Daniel Dehennin]

Hello.

I just tried with bookworm net installer and things changed since
bullseye.

On a bullseye, netcfg use the domain returned by DHCP which is not the
case on bookworm.

I think that the rDNS check before using the DHCP domain should be
removed.

It looks like on bullseye, domain was preseeded from DHCP because
netcfg/get_hostname was considered a valid FQDN which is not the case on
bookworm.

I got logs from both installations (attached to this email) and here are
the relevant logs:

#+begin_src diff
--- netcfg-bullseye.log 2023-08-23 11:18:55.0 +0200
+++ netcfg-bookworm.log 2023-08-23 11:18:55.0 +0200
@@ -1,4 +1,4 @@
-frontend: --> SET netcfg/get_hostname bullseye
+frontend: --> SET netcfg/get_hostname bookworm
 frontend: --> METAGET netcfg/get_hostname type
 frontend: --> FSET netcfg/get_hostname seen true
 frontend: --> SET netcfg/choose_interface auto
@@ -11,7 +11,7 @@
 debconf: --> METAGET debian-installer/netcfg/title Description
 main-menu: INFO: Menu item 'netcfg' selected
 debconf: --> SETTITLE debian-installer/netcfg/title
-netcfg: INFO: Starting netcfg v.1.176
+netcfg: INFO: Starting netcfg v.1.187
 debconf: --> GET netcfg/enable
 debconf: --> GET netcfg/disable_autoconfig
 debconf: --> SET netcfg/use_autoconfig true
@@ -144,11 +150,11 @@
 netcfg: DEBUG: State is now 2
 netcfg: DEBUG: State is now 5
 debconf: --> GET netcfg/hostname
-netcfg: INFO: DHCP hostname: "bullseye"
-netcfg: DEBUG: bullseye is a valid FQDN
-debconf: --> SET netcfg/get_hostname bullseye
-netcfg: DEBUG: Preseeding domain from global: eole.lan
-debconf: --> SET netcfg/get_domain eole.lan
+netcfg: DEBUG: Using DNS to try and obtain default hostname
+netcfg: DEBUG: Getting default hostname from rDNS lookup of autoconfigured 
address 192.168.230.136
+netcfg: DEBUG: getnameinfo() returned -2 (Name or service not known)
+netcfg: DEBUG: Getting default hostname from rDNS lookup of autoconfigured 
address fe80::c0ff:fea8:e6ea
+netcfg: DEBUG: getnameinfo() returned -2 (Name or service not known)
 debconf: --> INPUT high netcfg/get_hostname
 debconf: --> GET netcfg/get_hostname
 netcfg: DEBUG: State is now 6
#+end_src

-- 
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6  2AAD CC1E 9E5B 7A6F E2DF

frontend: --> SET netcfg/get_hostname bullseye
frontend: --> METAGET netcfg/get_hostname type
frontend: --> FSET netcfg/get_hostname seen true
frontend: --> SET netcfg/choose_interface auto
frontend: --> METAGET netcfg/choose_interface type
frontend: --> FSET netcfg/choose_interface seen true
debconf: --> METAGET debian-installer/netcfg/title Description
debconf: --> METAGET debian-installer/netcfg/title Description
debconf: --> METAGET debian-installer/netcfg/title Description
debconf: --> METAGET debian-installer/netcfg/title Description
debconf: --> METAGET debian-installer/netcfg/title Description
main-menu: INFO: Menu item 'netcfg' selected
debconf: --> SETTITLE debian-installer/netcfg/title
netcfg: INFO: Starting netcfg v.1.176
debconf: --> GET netcfg/enable
debconf: --> GET netcfg/disable_autoconfig
debconf: --> SET netcfg/use_autoconfig true
debconf: --> GET netcfg/disable_dhcp
netcfg: WARNING **: Couldn't read Wpasupplicant pid file, not trying to kill.
debconf: --> GET netcfg/choose_interface
netcfg: INFO: Could not find valid BOOTIF= entry in /proc/cmdline
netcfg: INFO: Taking down interface ens3
debconf: --> METAGET netcfg/internal-ens description
debconf: <-- 10 netcfg/internal-ens doesn't exist
debconf: --> METAGET netcfg/internal-unknown-iface description
debconf: --> INPUT medium netcfg/use_autoconfig
debconf: --> GET netcfg/use_autoconfig
netcfg: INFO: Taking down interface lo
netcfg: INFO: Activating interface ens3
netcfg: DEBUG: State is now 0
netcfg: DEBUG: Want link on ens3
debconf: --> INPUT low netcfg/link_wait_timeout
debconf: --> GET netcfg/link_wait_timeout
netcfg: INFO: Waiting time set to 3
debconf: --> SUBST netcfg/link_detect_progress interface ens3
debconf: --> PROGRESS START 0 12 netcfg/link_detect_progress
netcfg: INFO: ethtool-lite: ens3: carrier up
netcfg: INFO: Found link on ens3
netcfg: DEBUG: Commencing network autoconfiguration on ens3
netcfg: DEBUG: rdnssd started; PID: 550
debconf: --> PROGRESS START 0 12 netcfg/ipv6_link_local_wait_title
netcfg: DEBUG: nc_v6_interface_configured(ens3, scope local)
netcfg: DEBUG: Running ip addr show ens3 to look for address
netcfg: DEBUG: ip line: 2: ens3:  mtu 1500 
qdisc pfifo_fast qlen 1000
netcfg: DEBUG: ip line: link/ether 02:00:c0:a8:e6:ec brd ff:ff:ff:ff:ff:ff
netcfg: DEBUG: ip line: inet6 fe80::c0ff:fea8:e6ec/64 scope link tentative 
netcfg: DEBUG: ip line:valid_lft forever preferred_lft forever
netcfg: DEBUG: nc_v6_interface_configured(ens3, scope local)
netcfg: DEBUG: Running ip addr show ens3 to look for address
netcfg: DEBUG: ip line: 2: ens3:  mtu 1500 
qdisc pfifo_fast qlen 1000
netcfg: DEBUG: ip line: link/ether 02:00:c0:a8:e6:ec brd ff:ff:ff:ff:ff

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Cyril Brulebois]

Control: tag -1 - patch

Hi,

Thanks for the proposed patch but as discussed elsewhere it seemed too
risky to force 32 bpp on everyone, so I went for what looked like the
least risky (adding bochs.ko and cirrus.ko, manually and for the time
being).

Ben Hutchings  (2023-05-14):
> I think the problem is this GRUB has native drivers for Bochs and
> Cirrus that reprogram the framebuffer bit depth, and the kernel is then
> confused about what the bit depth is supposed to be.  With QXL, GRUB
> doesn't have a native driver so it doesn't reconfigure the framebuffer.

I've spent some time trying to reproduce these issues under UEFI but
without Secure Boot, and I failed. So I've moved to learning how to sign
a Linux kernel (certutil, pesign, mokutil, etc.), and I've added some
debugging information in various places.

Under Secure Boot, with the default QEMU driver (std aka. bochs),
initialization happens via:

drivers/firmware/efi/libstub/x86-stub.c

and its setup_graphics() that grabs the screen info part of boot params
and starts by zero-ing it:

si = &boot_params->screen_info;
memset(si, 0, sizeof(*si));

before trying efi_setup_gop() and setup_uga() in turn; the former being
current, the latter being the old standard.

Moving on to:

drivers/firmware/efi/libstub/gop.c

we see that its efi_setup_gop() calls setup_gop(), which in turn calls
find_gop(). That last one gets hold of a suitable GOP pointer:
  
https://uefi.org/specs/UEFI/2.10/12_Protocols_Console_Support.html#graphics-output-protocol

The rest of setup_gop() then uses information contained within that
structure to derive all relevant information, filling the screen_info
structure. That structure is then trusted by efifb, which can do nothing
else but fail miserably…

The si (screen_info) is set starting here:
  
https://elixir.bootlin.com/linux/v6.1.27/source/drivers/firmware/efi/libstub/gop.c#L534

Adding some debug, here's what I get with GRUB set to 800x600x24:

info->version: 0
info->horizontal_resolution: 1024
info->vertical_resolution: 768
info->pixel_format: 1
  info->pixel_information.red_mask: 0
  info->pixel_information.green_mask: 0
  info->pixel_information.blue_mask: 0
  info->pixel_information.reserved_mask: 0
info->pixels_per_scan_line: 1024

Let's see:

 - Of course width, height, and pixels_per_scan_line are incorrect.

 - pixel_format 1 means PIXEL_BGR_RESERVED_8BIT_PER_COLOR aka
   PixelBlueGreenRedReserved8BitPerColor in the spec, which means:

   A pixel is 32-bits and byte zero represents blue, byte one
   represents green, byte two represents red, and byte three is
   reserved. This is the definition for the physical frame buffer.
   The byte values for the red, green, and blue components represent
   the color intensity. This color intensity value range from a
   minimum intensity of 0 to maximum intensity of 255.

 - And masks are all 0.

So for this particular GRUB configuration to work, I've verified that
fixing all those fields was leading to a correct display via efifb
(having dropped bochs.ko to stick to efifb):

info->horizontal_resolution = 800;
info->vertical_resolution   = 600;
info->pixels_per_scan_line  = 800;

info->pixel_format = PIXEL_BIT_MASK;
info->pixel_information.red_mask  = 0x00ff;
info->pixel_information.green_mask= 0xff00;
info->pixel_information.blue_mask = 0x00ff;
info->pixel_information.reserved_mask = 0x;

Setting PIXEL_BIT_MASK means masks become relevant, and bits set in
those are added to determine the actual color depth, instead of an
hardcoded 32, giving me (and efifb) 24. And even:

efifb: mode is 800x600x24, linelength=2400, pages=1
efifb: scrolling: redraw
efifb: Truecolor: size=0:8:8:8, shift=0:16:8:0

instead of the dreaded:

efifb: mode is 1024x768x32, linelength=4096, pages=1
efifb: scrolling: redraw
efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0

Now, where to go from here? It seems pretty clear to me at this point
that the Linux kernel only relies on information that was obtained via
that GOP pointer, and does its best afterward.

The way the function call works seems pretty similar to what happens in
GRUB, so I'd think that the problem is likely *not* in the kernel, but
rather:

 - GRUB fails to set mode information properly.

 - OVMF drops the ball and return some default information.


> Unfortunately, with Secure Boot we have to use a monolithic GRUB build
> so I can't easily exclude video_bochs and video_cirrus to see if that
> improves matters.

Applying my new pesign skills on GRUB is the next step, but I have to
spend some time on another topic before Bookworm… It it possible that
trying to build a debug-enabled OVMF package might yield interesting
results, since AFAIUI that's the one implementing the back and forth…
If that's indeed the case, it should be easy to see what's written by
GRUB vs. what's read by Linux?

Processed: Re: Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Debian Bug Tracking System]

Processing control commands:

> tag -1 - patch
Bug #1036019 [debian-installer] debian-installer: Broken X display with QEMU 
under UEFI with cirrus and std graphics
Removed tag(s) patch.

-- 
1036019: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1036019
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Ben Hutchings]

On Sun, 2023-05-14 at 19:40 +0200, Ben Hutchings wrote:
[...]
> This works for me with all the QEMU graphics devices.  But I haven't
> tested on real hardware.

Now tested successfully on 2 custom desktops:

- Asus P8Z68-V LX motherboard, Intel Core i5 2500 CPU, integrated GPU
- ASRock B450 PRO4, AMD Ryzen 5 3600 CPU, Radeon RX580 GPU

and 2 laptops:

- Lenovo ThinkPad T420, Intel Core i5 2nd gen CPU, integrated GPU
- Lenovo ThinkPad T460, Intel Core i5 6th gen CPU, integrated GPU

Ben.

-- 
Ben Hutchings
Absolutum obsoletum. (If it works, it's out of date.) - Stafford Beer


signature.asc
Description: This is a digitally signed message part

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Ben Hutchings]

Control: tag -1 patch

On Sun, 2023-05-14 at 00:21 +0200, Ben Hutchings wrote:
[...]
> So I suppose there's a regression in either efifb or fbdev_drv.

I'm not spotting any functional changes in fbdev or the submodules it
depends on between bullseye and bookworm.  So this implicates either
efifb or, as you mentioned, GRUB.

> > Via QEMU, under BIOS and UEFI, results are:
> > 
> >   +-+-+-+-+
> >   |  Graphics   |  Bullseye 11.7  |  Bookworm RC 2  |  Daily builds   |
> >   +-+++++++
> >   | |  BIOS  |  UEFI  |  BIOS  |  UEFI  |  BIOS  |  UEFI  |
> >   +-+++++++
> >   | |   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
> >   | -vga std|   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
> >   | -vga cirrus |   OK   |   OK   |   OK   |  KO-S  |   OK   |  KO-S  |
> >   | -vga qxl|   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
> >   | -vga virtio |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
> >   | -vga vmware |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
> >   +-+++++++
> 
> I started testing with QEMU and OVMF from unstable, and I'm instead
> seeing Xorg failing to start in the same cases you see glitches.  The
> relevant error message seems to be this one:
> http://codesearch.debian.net/show?file=xorg-server_2%3A21.1.7-3%2Fhw%2Fxfree86%2Ffbdevhw%2Ffbdevhw.c&line=504
[...]

I tested with QEMU from bullseye and OVMF from unstable, and again I
saw Xorg failing to start, rather than glitches.  Weird.

I also patched the kernel to report the internal screen_info structure
and the fb_var_screeninfo structure passed in and out of
FBIOPUT_VSCREENINFO.  The key difference is:

- With -vga qxl, screen_info says 32 bpp, X wants 32 bpp, the kernel
  agrees with that.
- With -vga std or -vga cirrus screen_info says 24 bpp, X wants 32
  bpp, and the kernel says 24 bpp.

I think the problem is this GRUB has native drivers for Bochs and
Cirrus that reprogram the framebuffer bit depth, and the kernel is then
confused about what the bit depth is supposed to be.  With QXL, GRUB
doesn't have a native driver so it doesn't reconfigure the framebuffer.

Unfortunately, with Secure Boot we have to use a monolithic GRUB build
so I can't easily exclude video_bochs and video_cirrus to see if that
improves matters.

But what does works for me is:

--- a/build/boot/x86/grub/grub-efi.cfg
+++ b/build/boot/x86/grub/grub-efi.cfg
@@ -5,7 +5,7 @@ else
 fi
 
 if loadfont $font ; then
-  set gfxmode=800x600
+  set gfxmode=800x600x32
   set gfxpayload=keep
   insmod efi_gop
   insmod efi_uga
--- END ---

A full patch is attached.

This works for me with all the QEMU graphics devices.  But I haven't
tested on real hardware.


Ben.

-- 
Ben Hutchings
Absolutum obsoletum. (If it works, it's out of date.) - Stafford Beer
From 49a5e562850e3ae4f64ed2d61bd582d8adedc393 Mon Sep 17 00:00:00 2001
From: Ben Hutchings 
Date: Sun, 14 May 2023 19:17:45 +0200
Subject: [PATCH] Always use 32 bpp for GRUB EFI graphical menu (Closes:
 #1036019)

---
 build/boot/x86/grub/grub-efi.cfg | 2 +-
 debian/changelog | 4 
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/build/boot/x86/grub/grub-efi.cfg b/build/boot/x86/grub/grub-efi.cfg
index 0a9a67d48..14708c7bc 100644
--- a/build/boot/x86/grub/grub-efi.cfg
+++ b/build/boot/x86/grub/grub-efi.cfg
@@ -5,7 +5,7 @@ else
 fi
 
 if loadfont $font ; then
-  set gfxmode=800x600
+  set gfxmode=800x600x32
   set gfxpayload=keep
   insmod efi_gop
   insmod efi_uga
diff --git a/debian/changelog b/debian/changelog
index 4624187fe..6be6864b5 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,8 +1,12 @@
 debian-installer (20230428) UNRELEASED; urgency=medium
 
+  [ Cyril Brulebois ]
   * Bump Linux kernel ABI to 6.1.0-9.
   * Switch source format from 1.0 to 3.0 (native).
 
+  [ Ben Hutchings ]
+  * Always use 32 bpp for GRUB EFI graphical menu (Closes: #1036019)
+
  -- Cyril Brulebois   Thu, 27 Apr 2023 22:52:15 +0200
 
 debian-installer (20230427) unstable; urgency=medium


signature.asc
Description: This is a digitally signed message part

Processed: Re: Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Debian Bug Tracking System]

Processing control commands:

> tag -1 patch
Bug #1036019 [debian-installer] debian-installer: Broken X display with QEMU 
under UEFI with cirrus and std graphics
Added tag(s) patch.

-- 
1036019: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1036019
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Cyril Brulebois]

Cyril Brulebois  (2023-05-14):
> Also, I should note that while my focus was on netboot-gtk mini.iso
> (because it's much quicker to rebuild/tweak than a netinst image), I'm
> replicating those results with the netinst images:
[…]
>  - Bookworm RC 1 has a “text-like” GRUB, all good.
>  - Bookworm RC 2 has a “graphical” GRUB, issues!

While adjusting my “nasty” approach to make sure it would build on all
three modified archs (amd64, arm64, i386), it occurred to me that:

 - Of course the trivial patch wouldn't work, because some builds aren't
   “pure GTK” builds, like cdrom-xen, and that one would also need the
   686-pae flavour on i386.

 - Of course it wouldn't work on arm64 either, since that one doesn't
   ship vboxvideo.ko.

 - And more importantly, we have the fb-modules udeb in various places,
   including for builds that aren't about the graphical installer…


And at this point, it seems fair to say that at least the Linux kernel
isn't perfect, as problems show up even without X in the picture!

 - With Bookworm RC 1 netinst amd64 (again under UEFI), switch from
   default “Graphical install” to “Install”: the text installer shows
   up with both std and cirrus.

 - With Bookworm RC 2 netinst amd64 (again under UEFI), switch from
   default “Graphical install” to “Install”: the screen is garbled
   with std, split with cirrus.


This is easily confirmed:

 - Triggering a debian-installer “netboot” build (not “netboot-gtk”):
   the resulting mini.iso exhibits the same problems as Bookwork RC 2
   using “Install”, with both std and cirrus.

 - Patching that “netboot” build to benefit from the extra DRM modules
   makes those issues go away, with both std and cirrus.

 - Alternatively, not patching the “netboot” build but reverting the same
   patch as mentioned before makes those issues go away, with both std and
   cirrus:
 
https://salsa.debian.org/installer-team/debian-installer/-/commit/a4dc8c0fe7ad1a0c1506125ad9985f78819a1bb2


For the very short term (RC 3), I think I'll implement the following:

 1. Consider archs with the graphical installer (that's been my main
focus until a few hours ago, when I started realizing the console
without X was also impacted), even if other archs include fb-modules
as well.
This means: amd64, arm64, i386. Those happen to also do EFI/SB.

 2. Hardcode list of of modules to be added:
  drm_shmem_helper.ko
  drm_ttm_helper.ko
  drm_vram_helper.ko
  tiny/bochs.ko
  tiny/cirrus.ko
  ttm/ttm.ko
  vboxvideo/vboxvideo.ko [!arm64, i.e. amd64 and i386 only]

 3. For each of these 3 archs, deploy each of these modules. Do that for
each build that includes drm.ko (which should be synonymous with
fb-modules being deployed, given drm.ko is mandatory in the common
fb-modules file, included from the arch-specific ones in src:linux),
and do that without a condition on GTK detection or /usr/bin/Xorg's
presence.

This should be targeted enough (touching 3 archs, two of which are getting
a lot of attention; leaving all others entirely untouched), yet generic
enough to work around issues that show up in both text and graphical
versions of the installer, by patching all relevant builds (netboot,
netboot-gtk, those used by debian-cd, etc.).

I'll push a v2 of my nasty branch once I've performed some clean-up and
some more testing.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Cyril Brulebois]

Hi Ben,

Thanks for all those details!

Ben Hutchings  (2023-05-14):
> > 
> >   +-+-+-+-+
> >   |  Graphics   |  Bullseye 11.7  |  Bookworm RC 2  |  Daily builds   |
> >   +-+++++++
> >   | |  BIOS  |  UEFI  |  BIOS  |  UEFI  |  BIOS  |  UEFI  |
> >   +-+++++++
> >   | |   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
> >   | -vga std|   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
> >   | -vga cirrus |   OK   |   OK   |   OK   |  KO-S  |   OK   |  KO-S  |
> >   | -vga qxl|   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
> >   | -vga virtio |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
> >   | -vga vmware |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
> >   +-+++++++
> 
> I started testing with QEMU and OVMF from unstable, and I'm instead
> seeing Xorg failing to start in the same cases you see glitches.  The
> relevant error message seems to be this one:
> http://codesearch.debian.net/show?file=xorg-server_2%3A21.1.7-3%2Fhw%2Fxfree86%2Ffbdevhw%2Ffbdevhw.c&line=504

Checking RC 1, I'm seeing OK results for both `-vga std` (or no options)
and `-vga cirrus`. I should note GRUB itself is “text-like” with RC 1,
while it's “graphical” with RC 2.

Reverting the following commit in debian-installer.git and building a
netboot-gtk image against unstable gives me a working graphical
installer with `-vga std` (or no options) and `-vga cirrus`. I didn't
check the rest of the matrix though.
  
https://salsa.debian.org/installer-team/debian-installer/-/commit/a4dc8c0fe7ad1a0c1506125ad9985f78819a1bb2

So it looks to me the GRUB config fix uncovered a pre-existing bug, and
the linux version bump (6.1.20-1 → 6.1.20-2) between RC 1 and RC 2 isn't
a factor (xserver-xorg-* udebs didn't change).

Interestingly, switching to the bullseye branch and cherry-picking the
same GRUB config fix there, and rebuilding d-i against current bullseye,
I'm getting exactly the same problem: KO-G for std, KO-S for cirrus!

So it looks like this might be a rather old issue, rather than a
regression during the Bookworm release cycle.


Also, I should note that while my focus was on netboot-gtk mini.iso
(because it's much quicker to rebuild/tweak than a netinst image), I'm
replicating those results with the netinst images:
 - Bullseye has a “text-like” GRUB, all good.
 - Bookworm RC 1 has a “text-like” GRUB, all good.
 - Bookworm RC 2 has a “graphical” GRUB, issues!

> > Questions
> > =
> > 
> >  - Is it really to be expected that X and standard drivers would regress
> >this way when moving from Bullseye to Bookworm?
> 
> No.
> 
> >  - Or is it expected to require specific kernel modules while that wasn't
> >the case before? I've discovered this in VM environments, but maybe
> >similar things could be happening on bare metal as well, and maybe
> >some more modules should be considered for inclusion?
> 
> No.
> 
> >  - Is it acceptable to just bundle bochs, cirrus, and vboxvideo for the
> >time being (i.e. RC 3, RC 4, 12.0.0), be it via the nasty approach
> >or via a proper linux fb-modules inclusion?
> >  - Or does shipping those few modules risk breaking the kernel and/or X
> >on other platforms? (I'd definitely hope not!)
> 
> I would not expect so.  They get used on the installed system, so they
> probably work.

Copy all!

Note for a further session: instead of debugging d-i itself, it should
be possible to reproduce those issues in the installed system, by
keeping only a specific list of kernel modules and X drivers. Of course,
that means having GRUB in “graphical” mode as well (a quick check
suggests installing desktop-base, without plymouth*, is sufficient for
that part).

As a very quick experiment, I tried:
 - installing xfce4 and desktop-base;
 - rebooting;
 - X doesn't start directly, one needs to run startxfce4 from the
   console.

Then:
 - manually removing all X drivers except fbdev_drv.so;
 - manually removing both tiny/ drivers (bochs and cirrus);
 - rebuilding the initramfs;
 - rebooting.

This gives me the following:
 - std: black screen, not even seeing a console prompt;
 - cirrus: “garbled/split” screen symptoms in the console, and in X;
 - qxl: all good in the console and in X.

Interestingly, purging desktop-base gets me back to a “text-only” GRUB
prompt, but both std and cirrus are exhibiting “garbled/split” screen
symptoms in the console and in X.

I'll stop here, I just wanted to confirm one could reproduce those
issues within the installed system, which should almost always be a
debug-friendlier environment than d-i…

> > Proposal plan for d-i (Bookworm RC 3, RC 4, and 12.0.0)
> > =
> > 
> > Unless I received strong negative feedback before Monday (May 15th),
>

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Ben Hutchings]

On Sat, 2023-05-13 at 10:22 +0200, Cyril Brulebois wrote:
[...]
> Kernel-side
> ===
> 
> The fb-modules udeb hasn't changed much since 4+ years, with some DRM
> modules getting added alongside existing ones, leading to the following
> contents in Bullseye (5.10.178-3):
[...]
> Those contents are defined via those files in linux.git:
> 
> kibi@tokyo:~/debian-kernel/linux.git (sid=)$ cat 
> debian/installer/modules/amd64/fb-modules
> #include 
> 
> vesafb ?
> vga16fb
> 
> kibi@tokyo:~/debian-kernel/linux.git (sid=)$ cat 
> debian/installer/modules/fb-modules
> # We don't include all DRM drivers here as on many platforms we can
> # call system firmware to get hold of a simple framebuffer

To expand on this comment, in the case of UEFI boot the efifb driver
should provide a simple framebuffer, and on BIOS vesafb should do it. 
Those are both built-in on x86, and efifb is also built-in on arm64 and
armhf.


[...]
> X-side
> ==

Both of the kernel drivers are old-style framebuffer drivers so in
Xorg, the appropriate generic driver is "fbdev", not "modesetting".

> Now, we know that the contents of xserver-xorg-core-udeb have changed a
> little between Bullseye and Bookworm (#1035014), but that doesn't seem
> to be a factor here.
> 
> I've tested 3 netboot/gtk/mini.iso to assess the situation:
> 
>  - mini-20210731+deb11u8.iso from Bullseye 11.7
>  - mini-20230427.iso from D-I Bookworm RC 2
>  - mini-daily.isofrom D-I daily builds (downloaded today)
> 
> If people want to replicate those tests, they're available at:
>   https://people.debian.org/~kibi/bug-drm-vs-uefi/
> 
> Or:
> 
> wget 
> https://deb.debian.org/debian/dists/bullseye/main/installer-amd64/20210731+deb11u8/images/netboot/gtk/mini.iso
>  -O mini-20210731+deb11u8.iso
> wget 
> https://deb.debian.org/debian/dists/bookworm/main/installer-amd64/20230427/images/netboot/gtk/mini.iso
>  -O mini-20230427.iso
> wget https://d-i.debian.org/daily-images/amd64/daily/netboot/gtk/mini.iso 
> -O mini-daily.iso

These all include fbdev_drv.so, and Xorg.log shows that the fbdev
driver is being used.

So I suppose there's a regression in either efifb or fbdev_drv.

> Via QEMU, under BIOS and UEFI, results are:
> 
>   +-+-+-+-+
>   |  Graphics   |  Bullseye 11.7  |  Bookworm RC 2  |  Daily builds   |
>   +-+++++++
>   | |  BIOS  |  UEFI  |  BIOS  |  UEFI  |  BIOS  |  UEFI  |
>   +-+++++++
>   | |   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
>   | -vga std|   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
>   | -vga cirrus |   OK   |   OK   |   OK   |  KO-S  |   OK   |  KO-S  |
>   | -vga qxl|   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
>   | -vga virtio |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
>   | -vga vmware |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
>   +-+++++++

I started testing with QEMU and OVMF from unstable, and I'm instead
seeing Xorg failing to start in the same cases you see glitches.  The
relevant error message seems to be this one:
http://codesearch.debian.net/show?file=xorg-server_2%3A21.1.7-3%2Fhw%2Fxfree86%2Ffbdevhw%2Ffbdevhw.c&line=504

[...]
> Questions
> =
> 
>  - Is it really to be expected that X and standard drivers would regress
>this way when moving from Bullseye to Bookworm?

No.

>  - Or is it expected to require specific kernel modules while that wasn't
>the case before? I've discovered this in VM environments, but maybe
>similar things could be happening on bare metal as well, and maybe
>some more modules should be considered for inclusion?

No.

>  - Is it acceptable to just bundle bochs, cirrus, and vboxvideo for the
>time being (i.e. RC 3, RC 4, 12.0.0), be it via the nasty approach
>or via a proper linux fb-modules inclusion?
>  - Or does shipping those few modules risk breaking the kernel and/or X
>on other platforms? (I'd definitely hope not!)

I would not expect so.  They get used on the installed system, so they
probably work.



[...]
> Proposal plan for d-i (Bookworm RC 3, RC 4, and 12.0.0)
> =
> 
> Unless I received strong negative feedback before Monday (May 15th),
> I plan on including the nasty approach in RC 3, and to revert it
> altogether in RC 4 if big bad regressions are reported:
>   
> https://salsa.debian.org/installer-team/debian-installer/-/commit/9fceca63273d0b501ea64d7b719acafc93a5b7fa
> 
> As a side note, keeping the bundling in src:debian-installer for the
> next few weeks makes us autonomous: we can enable and disable those
> extra modules without requiring a new linux upload… so it's nasty but I
> actually thought about the few advantages we were getting out of

Bug#1036019: debian-installer: Broken X display with QEMU under UEFI with cirrus and std graphics

[Cyril Brulebois]

Package: debian-installer
Version: 20230427
Severity: important
X-Debbugs-Cc: debian-...@lists.debian.org, debian-ker...@lists.debian.org, 
debia...@lists.debian.org

Hi everyone,

I'm reaching out to all the aforementioned teams because I know nothing
about UEFI, kernel-side DRM modules, or X drivers, and I'd like to get
some feedback here.

If you need a TL;DR, you can skip to “Proposal plan for d-i”, which is
about my plans for the very next few hours, unless someone tells me the
proposal is crazy, unsafe, etc.


Backstory
=

Since we've been hitting and/or (re)discovering UEFI-specific issues
lately (#1033913), I decided to spend some time extending my usual
tests, traditionally run under QEMU with default settings, therefore
booted under BIOS, to also run them under UEFI (meaning also testing
Secure Boot without having to switch to baremetal).

I've been kindly pointed by regular image testers to the following page:
  https://wiki.debian.org/SecureBoot/VirtualMachine

But I was a little shocked to discover a broken X display when booting
under UEFI! It seems I'm not the only one since that page has the
following, even if there are no references to any bug reports:

-vga virtio - The Debian installer seems to have difficulties
  working with the standard VGA driver (and virtio
  should anyway have better performance) 

The test setup is described at the very end of this report, with my
current test target being specifically netboot/gtk/mini.iso for amd64.


Kernel-side
===

The fb-modules udeb hasn't changed much since 4+ years, with some DRM
modules getting added alongside existing ones, leading to the following
contents in Bullseye (5.10.178-3):

./lib/modules/5.10.0-22-amd64/kernel/drivers/gpu/drm/drm_kms_helper.ko
./lib/modules/5.10.0-22-amd64/kernel/drivers/gpu/drm/drm.ko
./lib/modules/5.10.0-22-amd64/kernel/drivers/gpu/drm/virtio/virtio-gpu.ko
./lib/modules/5.10.0-22-amd64/kernel/drivers/media/cec/core/cec.ko
./lib/modules/5.10.0-22-amd64/kernel/drivers/video/fbdev/vga16fb.ko
./lib/modules/5.10.0-22-amd64/kernel/drivers/video/vgastate.ko
./lib/modules/5.10.0-22-amd64/kernel/drivers/virtio/virtio_dma_buf.ko

and the following contents in Bookworm (6.1.27-1):

./lib/modules/6.1.0-9-amd64/kernel/drivers/gpu/drm/drm_kms_helper.ko
./lib/modules/6.1.0-9-amd64/kernel/drivers/gpu/drm/drm.ko
./lib/modules/6.1.0-9-amd64/kernel/drivers/gpu/drm/drm_shmem_helper.ko
./lib/modules/6.1.0-9-amd64/kernel/drivers/gpu/drm/virtio/virtio-gpu.ko
./lib/modules/6.1.0-9-amd64/kernel/drivers/video/fbdev/vga16fb.ko
./lib/modules/6.1.0-9-amd64/kernel/drivers/video/vgastate.ko
./lib/modules/6.1.0-9-amd64/kernel/drivers/virtio/virtio_dma_buf.ko

Those contents are defined via those files in linux.git:

kibi@tokyo:~/debian-kernel/linux.git (sid=)$ cat 
debian/installer/modules/amd64/fb-modules
#include 

vesafb ?
vga16fb

kibi@tokyo:~/debian-kernel/linux.git (sid=)$ cat 
debian/installer/modules/fb-modules
# We don't include all DRM drivers here as on many platforms we can
# call system firmware to get hold of a simple framebuffer

drm
drm_kms_helper
virtio-gpu ?


X-side
==

Now, we know that the contents of xserver-xorg-core-udeb have changed a
little between Bullseye and Bookworm (#1035014), but that doesn't seem
to be a factor here.

I've tested 3 netboot/gtk/mini.iso to assess the situation:

 - mini-20210731+deb11u8.iso from Bullseye 11.7
 - mini-20230427.iso from D-I Bookworm RC 2
 - mini-daily.isofrom D-I daily builds (downloaded today)

If people want to replicate those tests, they're available at:
  https://people.debian.org/~kibi/bug-drm-vs-uefi/

Or:

wget 
https://deb.debian.org/debian/dists/bullseye/main/installer-amd64/20210731+deb11u8/images/netboot/gtk/mini.iso
 -O mini-20210731+deb11u8.iso
wget 
https://deb.debian.org/debian/dists/bookworm/main/installer-amd64/20230427/images/netboot/gtk/mini.iso
 -O mini-20230427.iso
wget https://d-i.debian.org/daily-images/amd64/daily/netboot/gtk/mini.iso 
-O mini-daily.iso


Via QEMU, under BIOS and UEFI, results are:

  +-+-+-+-+
  |  Graphics   |  Bullseye 11.7  |  Bookworm RC 2  |  Daily builds   |
  +-+++++++
  | |  BIOS  |  UEFI  |  BIOS  |  UEFI  |  BIOS  |  UEFI  |
  +-+++++++
  | |   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
  | -vga std|   OK   |   OK   |   OK   |  KO-G  |   OK   |  KO-G  |
  | -vga cirrus |   OK   |   OK   |   OK   |  KO-S  |   OK   |  KO-S  |
  | -vga qxl|   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
  | -vga virtio |   OK   |   OK   |   OK   |   OK   |   OK   |   OK   |
  | -vga

Bug#792456: encrypted swap broken in last several debian releases

[Matt Taggart]

The last few debian releases (buster and bullseye at least) configuring 
encrypted swap has been broken for me. I don't know if it's the same as 
what the user in #792456 is seeing, but likely.


I have most recently tested it in Bookworm RC1 and here is what I did:
* run by hand partitioning
* create 3 partitions: boot, swap, lvm
* configure encryption: enable encryption for swap and lvm partitions
  * configure swap to use serpent and random passphrase
  * configure lvm partition to use serpent and key
  * finish
* mark lvm encrypted partition to be used for lvm, configure lvm, setup 
root and var partition

* configure boot partition to be ext4 mounted at /boot
* finish partitioning
* receive error about failure to setup swap

Sorry for the short report, I will try to redo it and capture logs, but 
I wanted to report this ASAP.


My workaround has been to allocate the partition but mark it do not use 
and then finish the setup by hand after install.


Thanks,

--
Matt Taggart
m...@lackof.org

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

On Fri, 21 Apr 2023 at 12:25:29 +0200, Guilhem Moulin wrote:
> Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 
> 2:2.6.1-4~deb12u1,
> graphical install), 1024M RAM:
> 
>   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  10
>   Memory: 223780
>   Threads:2
>   root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  8
>   Memory: 490598
>   Threads:2
> 
> Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 
> 2:2.6.1-4~deb12u1,
> text install), 1024M RAM:
> 
>   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  8
>   Memory: 294302
>   Threads:2
>   root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  8
>   Memory: 490598
>   Threads:2
> 
> Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 
> 2:2.6.1-4~deb12u1,
> text install), 2048M RAM:
> 
>   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  4
>   Memory: 590553
>   Threads:2
>   root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  4
>   Memory: 1005926
>   Threads:2
> 
> Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 
> 2:2.6.1-4~deb12u1,
> text install), 4096M RAM:
> 
>   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  4
>   Memory: 613826
>   Threads:2
>   root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<   root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
>   PBKDF:  argon2id
>   Time cost:  4
>   Memory: 1048576
>   Threads:2
>
> […]
> * I was surprised to see the memory cost settle at ~550-600M on systems
>  with a decent amount of RAM in d-i.  Would have expected to see 1G
>  here just like after running `cryptsetup luksConvertKey` in the
>  normal system.

libargon2-1-udeb bug filed at #1034696.  For the sake of completion, here are
updated benchmark results after injecting src:argon2=0~20171227-0.3+deb12u1
(debdiff attached to the aforementioned bug) into the ISO:

Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 
2:2.6.1-4~deb12u1
+ argon2 0~20171227-0.3+deb12u1, graphical install), 1024M RAM:

root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF:  argon2id
Time cost:  19
Memory: 219508
Threads:2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<

signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

On Fri, 21 Apr 2023 at 13:02:24 +0200, Cyril Brulebois wrote:
> Summing up some out-of-band brainstorming about what “a bit crippled”
> means, it might just be libargon2-1-udeb's being built without pthread
> support:
>
> https://salsa.debian.org/debian/argon2/-/commit/31225912349933993e49f5007e97630b20465c32

>>  Never noticed that before, but that's not a regression since buster
>>  and bullseye both have the same behavior.  (At least in my test VMs;
>>  didn't compare on real hardware.)
> 
> Based on Samuel's feedback on IRC, what used to be true in 2019 might no
> longer be today; we might try and enable pthread support, and see
> whether that makes a difference.

That seems correct!  I just rebuilt argon2 0~20190702+dfsg-2 for
bookworm without the NO_THREADS=1 and injected the udeb in the .ISO.
The PBKDF benchmark result is as one would expect on a system with 4G
RAM:

~ # echo test | cryptsetup luksFormat --debug --batch-mode /dev/vda
[…]
# Running argon2id() benchmark.
# PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 2 
(took 209 ms)
# PBKDF benchmark: memory cost = 78392, iterations = 4, threads = 2 
(took 156 ms)
# PBKDF benchmark: memory cost = 125628, iterations = 4, threads = 2 
(took 242 ms)
# PBKDF benchmark: memory cost = 129780, iterations = 4, threads = 2 
(took 242 ms)
# PBKDF benchmark: memory cost = 134070, iterations = 4, threads = 2 
(took 243 ms)
# PBKDF benchmark: memory cost = 137932, iterations = 4, threads = 2 
(took 253 ms)
# Benchmark returns argon2id() 4 iterations, 1048576 memory, 2 threads 
(for 512-bits key).
[…]

> As noted in https://mjg59.dreamwidth.org/66429.html what happens during
> installation might not get changed at all during the life of a given
> system, so if we can somewhat easily improve what happens during
> installation… :)

Ack, and given the diff that's probably a worthwhile thing to have in
bookworm :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Guilhem Moulin  (2023-04-21):
> Just uploaded 2:2.6.1-4 to sid, and locally prepared a rebuild for
> bookworm (2:2.6.1-4~deb12u1).

\o/

> Bottom line:
> 
>  * The upstream patches in the patch-queue (the 2 backported earlier
>from upstream MR !490 plus the new other two from upstream MR !498)
>only affect systems with <2G RAM (i.e., those where half the amount
>of physical memory is lower than DEFAULT_LUKS2_MEMORY_KB).  And only
>those without swap.  On such systems the memory cost is set to a
>lower value at the expense of a higher time cost, which is the
>intended behavior; it appear to leave enough head-room for the
>graphical installer to succeed with 1G RAM, so I believe the errata
>can be removed if the changes makes it to bookworm.

Looks very good to me, thanks.

>  * I was surprised to see the memory cost settle at ~550-600M on systems
>with a decent amount of RAM in d-i.  Would have expected to see 1G
>here just like after running `cryptsetup luksConvertKey` in the
>normal system.  I get a similarily low memory cost after dropping to
>a rescue shell early in d-i and running `luksFormat` manually:

[…]

>I think what happens here is that compared to the final system d-i is
>a bit crippled so the 2s threshold is reached earlier in the
>benchmark.

Summing up some out-of-band brainstorming about what “a bit crippled”
means, it might just be libargon2-1-udeb's being built without pthread
support:
  
https://salsa.debian.org/debian/argon2/-/commit/31225912349933993e49f5007e97630b20465c32

>Never noticed that before, but that's not a regression since buster
>and bullseye both have the same behavior.  (At least in my test VMs;
>didn't compare on real hardware.)

Based on Samuel's feedback on IRC, what used to be true in 2019 might no
longer be today; we might try and enable pthread support, and see
whether that makes a difference.

As noted in https://mjg59.dreamwidth.org/66429.html what happens during
installation might not get changed at all during the life of a given
system, so if we can somewhat easily improve what happens during
installation… :)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

Hi,

On Thu, 20 Apr 2023 at 20:02:27 +0200, Cyril Brulebois wrote:
>> * Backport upstream MR !498, let it mature in sid for a few
>> weeks then upload 2:2.6.1-4~deb12u1 via t-p-u.  There are only 2
>> upstream commits to cherry-pick and neither is large nor intrusive;
>> moreover like the commits previously cherry-picked they are no-op on
>> “normal” systems (only systems without swap are affected).  For
>> convenience I attach a debdiff for 2:2.6.1-3~deb12u2 and you'll also
>> find binary packages for amd64 at
>> https://people.debian.org/~guilhem/tmp/cryptsetup_2.6.1-3~deb12u2/
>> Tested: autopkgtests (incl. full upstream test suite), d-i in both
>> graphical and text install on VMs with 1024M RAM (now memory cost
>> won't exceed ~250M resp. ~300M thus leaving plenty of headroom for
>> the rest).
>
> Since you're happy with that approach, let's go for an upload to
> unstable for the time being, I'll conduct some tests shortly, and once
> it's indeed confirmed to work fine, go via t-p-u (because of the same
> fun as before with some library) so that it can be used for rc3 (if it's
> ready by then — we haven't really defined when it's going to happen
> besides “somewhen before end of April”).

Just uploaded 2:2.6.1-4 to sid, and locally prepared a rebuild for
bookworm (2:2.6.1-4~deb12u1).

Comparing PBKDF benchmark results obtained using default settings
(guided “encrypted LVM” partitioning scheme) between the last 3 releases
and 1, 2, or 4G RAM (the first luksDump is what I got out of d-i, the
second shows benchmark results on the final system — with swap), I get
the following parameters (summary at the bottom).

Buster (debian-10.12.0-amd64-netinst.iso, text install), 1024M RAM:

root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF
PBKDF:  argon2i
Time cost:  4
Memory: 504962
Threads:2
root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<

signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Hi,

(Letting Paul and the bug report know about our little chat.)

Guilhem Moulin  (2023-04-20):
> AFAICT the issue is now fully fixed upstream: on systems without swap
> the memory cost won't exceed half the amount of free memory during
> PBKDF benchmarking.

As a reminder: the “no swap” case happens in d-i when using guided
partitioning, as the swap will only be added/activated after formatting
the disk.

>  * Don't do anything: ship 2:2.6.1-3~deb12u1 in bookworm and leave the
>DI errata in place.  The downside is that the PBKDF needs roughly
>half of the physical memory, so the OOM killer might trigger if the
>rest of the system uses close to the other half.  Moreover this not
>future proof, as memory requirement increases along releases.  (That
>said the issue has been present since 2 releases and there is nothing
>we can do about existing volumes.  Concretely, that means low-memory
>Bookworm rescue systems will likely OOM when trying to luksOpen an
>existing LUKS2 volume in graphical mode.)

I'd rather avoid that one for the reasons you mention.

>  * Wait for upstream to release 2.6.2 with fixes for #-1 as well as
>other bugfixes and upload it, either via t-p-u during the hard freeze
>or later via s-p-u.  In upstream's own words “[the minor release]
>will take few week because of translation loop etc.”  The downside
>being of course more review work for the release team, as the diff is
>already rather large:
>
> https://gitlab.com/cryptsetup/cryptsetup/-/compare/v2.6.1...main?from_project_id=195655&straight=false

Waiting is definitely not needed from my point of view.

>  * Backport upstream MR !498, let it mature in sid for a few
>weeks then upload 2:2.6.1-4~deb12u1 via t-p-u.  There are only 2
>upstream commits to cherry-pick and neither is large nor intrusive;
>moreover like the commits previously cherry-picked they are no-op on
>“normal” systems (only systems without swap are affected).  For
>convenience I attach a debdiff for 2:2.6.1-3~deb12u2 and you'll also
>find binary packages for amd64 at
>https://people.debian.org/~guilhem/tmp/cryptsetup_2.6.1-3~deb12u2/
>Tested: autopkgtests (incl. full upstream test suite), d-i in both
>graphical and text install on VMs with 1024M RAM (now memory cost
>won't exceed ~250M resp. ~300M thus leaving plenty of headroom for
>the rest).

Since you're happy with that approach, let's go for an upload to
unstable for the time being, I'll conduct some tests shortly, and once
it's indeed confirmed to work fine, go via t-p-u (because of the same
fun as before with some library) so that it can be used for rc3 (if it's
ready by then — we haven't really defined when it's going to happen
besides “somewhen before end of April”).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

Hi kibi,

On Sat, 01 Apr 2023 at 01:34:54 +0200, Guilhem Moulin wrote:
> Ah right, reopened the upstream issue but forgot to follow-up here :-(
> https://gitlab.com/cryptsetup/cryptsetup/-/issues/802#note_1328592911

AFAICT the issue is now fully fixed upstream: on systems without swap
the memory cost won't exceed half the amount of free memory during PBKDF
benchmarking.  I can think of the following next steps (list probably
not exhaustive):

 * Don't do anything: ship 2:2.6.1-3~deb12u1 in bookworm and leave the
   DI errata in place.  The downside is that the PBKDF needs roughly
   half of the physical memory, so the OOM killer might trigger if the
   rest of the system uses close to the other half.  Moreover this not
   future proof, as memory requirement increases along releases.  (That
   said the issue has been present since 2 releases and there is nothing
   we can do about existing volumes.  Concretely, that means low-memory
   Bookworm rescue systems will likely OOM when trying to luksOpen an
   existing LUKS2 volume in graphical mode.)

 * Wait for upstream to release 2.6.2 with fixes for #-1 as well as
   other bugfixes and upload it, either via t-p-u during the hard freeze
   or later via s-p-u.  In upstream's own words “[the minor release]
   will take few week because of translation loop etc.”  The downside
   being of course more review work for the release team, as the diff is
   already rather large:
   
https://gitlab.com/cryptsetup/cryptsetup/-/compare/v2.6.1...main?from_project_id=195655&straight=false

 * Backport upstream MR !498, let it mature in sid for a few
   weeks then upload 2:2.6.1-4~deb12u1 via t-p-u.  There are only 2
   upstream commits to cherry-pick and neither is large nor intrusive;
   moreover like the commits previously cherry-picked they are no-op on
   “normal” systems (only systems without swap are affected).  For
   convenience I attach a debdiff for 2:2.6.1-3~deb12u2 and you'll also
   find binary packages for amd64 at
   https://people.debian.org/~guilhem/tmp/cryptsetup_2.6.1-3~deb12u2/
   Tested: autopkgtests (incl. full upstream test suite), d-i in both
   graphical and text install on VMs with 1024M RAM (now memory cost
   won't exceed ~250M resp. ~300M thus leaving plenty of headroom for
   the rest).

Your call :-)
-- 
Guilhem.
diffstat for cryptsetup-2.6.1 cryptsetup-2.6.1

 changelog   |8 
+
 patches/Check-for-physical-memory-available-also-in-PBKDF-benchma.patch |   74 
++
 patches/Use-only-half-of-detected-free-memory-on-systems-without-.patch |   43 
+
 patches/series  |2 
 4 files changed, 127 insertions(+)

diff -Nru cryptsetup-2.6.1/debian/changelog cryptsetup-2.6.1/debian/changelog
--- cryptsetup-2.6.1/debian/changelog   2023-03-26 19:18:59.0 +0200
+++ cryptsetup-2.6.1/debian/changelog   2023-04-20 13:19:06.0 +0200
@@ -1,3 +1,11 @@
+cryptsetup (2:2.6.1-3~deb12u2) UNRELEASED; urgency=medium
+
+  * Backport upstream MR !498, see #1028250:
++ 7893c33d: Check for physical memory available also in PBKDF benchmark.
++ 6721d3a8: Use only half of detected free memory on systems without swap.
+
+ -- Guilhem Moulin   Thu, 20 Apr 2023 13:19:06 +0200
+
 cryptsetup (2:2.6.1-3~deb12u1) bookworm; urgency=medium
 
   * Rebuild for Bookworm.
diff -Nru 
cryptsetup-2.6.1/debian/patches/Check-for-physical-memory-available-also-in-PBKDF-benchma.patch
 
cryptsetup-2.6.1/debian/patches/Check-for-physical-memory-available-also-in-PBKDF-benchma.patch
--- 
cryptsetup-2.6.1/debian/patches/Check-for-physical-memory-available-also-in-PBKDF-benchma.patch
 1970-01-01 01:00:00.0 +0100
+++ 
cryptsetup-2.6.1/debian/patches/Check-for-physical-memory-available-also-in-PBKDF-benchma.patch
 2023-04-20 13:19:06.0 +0200
@@ -0,0 +1,74 @@
+From: Milan Broz 
+Date: Mon, 3 Apr 2023 13:31:16 +0200
+Subject: Check for physical memory available also in PBKDF benchmark.
+
+Origin: 
https://gitlab.com/cryptsetup/cryptsetup/-/commit/7893c33d71cde09e240234c484c6c468f22c2fe7
+Bug: https://gitlab.com/cryptsetup/cryptsetup/-/issues/802#note_1328592911
+Bug-Debian: https://bugs.debian.org/1028250
+---
+ lib/internal.h| 1 +
+ lib/utils_benchmark.c | 9 +
+ lib/utils_pbkdf.c | 4 ++--
+ 3 files changed, 12 insertions(+), 2 deletions(-)
+
+diff --git a/lib/internal.h b/lib/internal.h
+index 98095fa..f261cae 100644
+--- a/lib/internal.h
 b/lib/internal.h
+@@ -89,6 +89,7 @@ int crypt_benchmark_pbkdf_internal(struct crypt_device *cd,
+  struct crypt_pbkdf_type *pbkdf,
+  size_t volume_key_size);
+ const char *crypt_get_cipher_spec(struct crypt_device *cd);
++uint32_t pbkdf_adjusted_phys_memory_kb(void);
+ 
+ /* Device backend */
+ struct device;
+diff --git a/lib/utils_benchmark.c b/lib/utils_benchmark.c
+index 728e4df..a0326ce 100644
+--- a/l

Bug#1033913: marked as done (partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode)

[Debian Bug Tracking System]

Your message dated Thu, 13 Apr 2023 13:03:56 +
with message-id 
and subject line Bug#1033913: fixed in partman-efi 99
has caused the Debian Bug report #1033913,
regarding partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in 
UEFI mode
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
1033913: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033913
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: partman-auto-lvm
Version: 87
Severity: serious
Justification: Maintainer says so

TL;DR: Answering “Yes” to the “Force UEFI installation?” makes sure the
installer pulls the right bootloader packages, despite misreading the
situation.

I've discovered this while testing D-I Bookworm RC 1 but also confirmed
it already existed in D-I Bookworm Alpha 2, and I'm therefore filing it
against the version found in the previous release (and deciding not to
block the Bookworm RC 1 release on it).



For baremetal tests on laptops requiring various firmware packages, I've
been using guided partitioning since forever, with one of these:
 - Guided - use entire disk
 - Guided - use entire disk and set up encrypted LVM

The former is used most of the time since it's slightly faster (fewer
prompts), while the latter is only used once in a while, to make sure a
“real” laptop-oriented install works fine (since every laptop should be
encrypted in my opinion).

Since I had just tested “Guided - use entire disk” in a virtual machine,
I decided to pick this instead when switching to the first laptop
(Asus Vivobook S14/S15 but that's very likely not a factor):
 - Guided - use entire disk and set up LVM

And… *WOW!*

The first surprise is this prompt:

Force UEFI installation?

This machine's firmware has started the installer in UEFI mode but
it looks like there may be existing operating systems already
installed using "BIOS compatibility mode". If you continue to
install Debian in UEFI mode, it might be difficult to reboot the
machine into any BIOS-mode operating systems later.

If you wish to install in UEFI mode and don't care about keeping the
ability to boot one of the existing systems, you have the option to
force that here. If you wish to keep the option to boot an existing
operating system, you should choose NOT to force UEFI installation
here.

which defaults to No.

That's very surprising since the only operating system prior to the
installation was a Debian system, which was getting entirely erased (due
to using the full disk), and was installed in UEFI mode anyway.

I went for the default choice, since we expect the installer to make
smart suggestions, and unsuspecting users shouldn't have to know better.

That means we end up with installing grub-pc instead of grub-efi-amd64
and shim, being prompted where to install GRUB, and of course when it's
time to reboot, the UEFI firmware rightfully refuses to boot anything
since there's absolutely no signature whatsoever, which isn't a great
idea under Secure Boot:

Secure Boot Violation

Invalid signature detected. Check Secure Boot Policy in Setup.


Some additional info:
 - As mentioned in TL;DR, this can be worked around by answering Yes to
   “Force UEFI installation?”.
 - It doesn't seem to be dependent on possible traces of an existing
   system prior to the installation: Debian installed on the entire disk
   or with encrypted LVM on the entire disk doesn't seem to make a
   difference. Starting with a wiped disk (writing ~ 2 GB worth of
   zeros at the beginning of the disk) doesn't make a difference either.
 - It very much looks like the intermediary states are slightly
   different when setting up LVM and when setting up encrypted LVM, and
   the LVM case case leads to some confusion in partman-efi's
   /lib/partman/init.d/50efi (which logs to /var/log/partman rather than
   to /var/log/syslog): “Found 0 ESPs, 3 non-ESPs”.
 - I'm filing this issue against partman-auto-lvm though, for
   discoverability purposes.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
--- End Message ---
--- Begin Message ---
Source: partman-efi
Source-Version: 99
Done: Steve McIntyre <93...@debian.org>

We believe that the bug you reported is fixed in the latest version of
partman-efi, which is due to be installed in the Debian FTP archive.

A summary of the changes between th

Bug#1033913: partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode

[Steve McIntyre]

I've just pushed an update to the code here...

On Mon, Apr 10, 2023 at 05:45:15PM +0200, Pascal Hambourg wrote:
>On 10/04/2023 at 15:13, Steve McIntyre wrote:
>> 
>> Overall comment: I'm not trying to make the heuristics 100% reliable
>> here, as I don't think that's actually possible. Instead, I'm trying
>> to tread the fine line of:
>> 
>>   * minimising false negatives - let's try to pick up on the most
>> common cases where people are dual-booting with other systems and
>> might not understand the issues here. That's 99%+ going to be
>> people with Windows installed
>> 
>>   * minimising false positives - the issue that angered Cyril in
>> particular, with an incomplete LVM setup triggering the "bios
>> bootable OS" warning
>
>IMO it is more important to avoid false positives, because switching to a
>BIOS installation on systems which are not BIOS-boot capable would create a
>non bootable system. In case oft is easier to install GRUB for BIOS boot on
>an running EFI system than the other way around.

No. The reason I added this check and warning in the first place is to
avoid breaking existing (all-too-common) systems where Windows users
have a BIOS-booting installation but their BIOS is set to boot both
UEFI and BIOS. That's a stupid combination, but again all too
common. :-( New users who are just trying to install Debian dual-boot
are much less likely to be able to diagnose this kind of problem.

>> > - Other BIOS boot loaders such as syslinux/extlinux do not need or use a 
>> > BIOS
>> > boot partition.
>> 
>> Also not a use case I'm particularly caring about, I'll be
>> honest. They're also *really* not likely to work well without another
>> filesystem in use, which I expect we'll detect anyway.
>
>Indeed other partitions are needed and will be detected, but they will not
>increment $NUM_NOT_ESP if the disk is GPT and has no BIOS boot partition (so
>$DISK_BIOS_BOOT=no), so it might cause a false negative. So why not just
>treat MSDOS and GPT disk labels equally and treat BIOS boot partitions like
>any other non-ESP ?

It's a false negative that I really don't believe or care about very
much, I'll be honest. This is getting to be an edge case on an edge
case.

>> > 1b) IIUC the patch fixes #1033913 because the disk selected for 
>> > installation
>> > has received a new GPT disklabel without a BIOS boot partition, so further
>> > checking is skipped. But IMO the root cause of #1033913 is that changes are
>> > not committed to disk after setting the 'boot' and 'esp' flags to the newly
>> > created ESP partition before stopping parted_server.
>
>I originally thought about fixing partman-auto-lvm but it appears that other
>transient states can also trigger the "force UEFI installation" dialog during
>partitioning, for example after setting up LVM in manual partitioning if
>there is no ESP partition yet. As discussed in #debian-boot, a more general
>fix might be to run the check only once because only existing partitions
>before partitioning are relevant. Are there any use cases for which this
>might cause a false negative ?

So I've now modded the code to add a flag file - it'll only run the
check and (maybe) raise the warning on the first entry into
partman. Thanks for the suggection, this is clearly the correct
answer.

>> > 4) It appears that partman fails to detect the specially crafted partition
>> > table on the installation media created with a debian image. Is it intended
>> > or fortunately unintentional ? If partman could see the EFI partition on 
>> > the
>> > installation media, the detection of BIOS-bootable systems would fail.
>> 
>> That's not a worry for today... :-)
>
>Sure, but the issue can also happen if another removable media is present.
>For instance the USB drive I use to provide missing firmware has an ESP
>partition (and a regular partition table) thus can cause a false negative.

Again, we're hitting edge cases. We can't know for sure what the user
wants here, so we can't just ignore removable media (for example).

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Since phone messaging became popular, the young generation has lost the
 ability to read or write anything that is longer than one hundred and sixty
 characters."  -- Ignatios Souvatzis

Bug#1033913: partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode

[Pascal Hambourg]

On 10/04/2023 at 15:13, Steve McIntyre wrote:


Overall comment: I'm not trying to make the heuristics 100% reliable
here, as I don't think that's actually possible. Instead, I'm trying
to tread the fine line of:

  * minimising false negatives - let's try to pick up on the most
common cases where people are dual-booting with other systems and
might not understand the issues here. That's 99%+ going to be
people with Windows installed

  * minimising false positives - the issue that angered Cyril in
particular, with an incomplete LVM setup triggering the "bios
bootable OS" warning


IMO it is more important to avoid false positives, because switching to 
a BIOS installation on systems which are not BIOS-boot capable would 
create a non bootable system. In case oft is easier to install GRUB for 
BIOS boot on an running EFI system than the other way around.



- Other BIOS boot loaders such as syslinux/extlinux do not need or use a BIOS
boot partition.


Also not a use case I'm particularly caring about, I'll be
honest. They're also *really* not likely to work well without another
filesystem in use, which I expect we'll detect anyway.


Indeed other partitions are needed and will be detected, but they will 
not increment $NUM_NOT_ESP if the disk is GPT and has no BIOS boot 
partition (so $DISK_BIOS_BOOT=no), so it might cause a false negative. 
So why not just treat MSDOS and GPT disk labels equally and treat BIOS 
boot partitions like any other non-ESP ?



1b) IIUC the patch fixes #1033913 because the disk selected for installation
has received a new GPT disklabel without a BIOS boot partition, so further
checking is skipped. But IMO the root cause of #1033913 is that changes are
not committed to disk after setting the 'boot' and 'esp' flags to the newly
created ESP partition before stopping parted_server.


I originally thought about fixing partman-auto-lvm but it appears that 
other transient states can also trigger the "force UEFI installation" 
dialog during partitioning, for example after setting up LVM in manual 
partitioning if there is no ESP partition yet. As discussed in 
#debian-boot, a more general fix might be to run the check only once 
because only existing partitions before partitioning are relevant. Are 
there any use cases for which this might cause a false negative ?



4) It appears that partman fails to detect the specially crafted partition
table on the installation media created with a debian image. Is it intended
or fortunately unintentional ? If partman could see the EFI partition on the
installation media, the detection of BIOS-bootable systems would fail.


That's not a worry for today... :-)


Sure, but the issue can also happen if another removable media is 
present. For instance the USB drive I use to provide missing firmware 
has an ESP partition (and a regular partition table) thus can cause a 
false negative.

Bug#1033913: partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode

[Steve McIntyre]

Hey Pascal, and thanks for the review!

Overall comment: I'm not trying to make the heuristics 100% reliable
here, as I don't think that's actually possible. Instead, I'm trying
to tread the fine line of:

 * minimising false negatives - let's try to pick up on the most
   common cases where people are dual-booting with other systems and
   might not understand the issues here. That's 99%+ going to be
   people with Windows installed

 * minimising false positives - the issue that angered Cyril in
   particular, with an incomplete LVM setup triggering the "bios
   bootable OS" warning

On Mon, Apr 10, 2023 at 01:01:01PM +0200, Pascal Hambourg wrote:
>partman-efi "Fix detection of BIOS-bootable systems" provides a significant
>improvement over previous behaviour. However I have a few comments.
>
>1a) The patch assumes that a GPT disk may be BIOS-bootable only if it has a
>BIOS boot partition. But a GPT disk can be BIOS-bootable even without a BIOS
>boot partition:
>- GRUB may be installed without a BIOS boot partition if /boot is a plain
>partition (using blocklists), even though it is less reliable so a BIOS boot
>partition is strongly recommended.

Yeah, GRUB installed using blocklists is so much *not* a thing anybody
should be doing these days.

>- Other BIOS boot loaders such as syslinux/extlinux do not need or use a BIOS
>boot partition.

Also not a use case I'm particularly caring about, I'll be
honest. They're also *really* not likely to work well without another
filesystem in use, which I expect we'll detect anyway.

>1b) IIUC the patch fixes #1033913 because the disk selected for installation
>has received a new GPT disklabel without a BIOS boot partition, so further
>checking is skipped. But IMO the root cause of #1033913 is that changes are
>not committed to disk after setting the 'boot' and 'esp' flags to the newly
>created ESP partition before stopping parted_server.
>This can be seen in /var/log/partman:
>
>/bin/autopartition-lvm
>NEW_LABEL sda gpt
>NEW_PARTITION 1 sda ext2 538MB (future ESP)
>NEW_PARTITION 2 sda ext2 512MB (future /boot)
>NEW_PARTITION 3 sda ext3 159GB (future LVM)
>SET_FLAGS sda3 lvm
>(user prompt to write changes to the disk)
>COMMIT sda
>...
>/lib/partman/update.d/21efi_sync_flag
>SET_FLAGS sda1 boot esp
>...
>/bin/perform_recipe_by_lvm
>QUIT <- esp and boot flags have not been committed yet so are lost
>...
>/lib/partman/init.d/50efi
>GET_FLAGS sda1 -> none
>
>2) The patch considers the 'esp' and 'boot' flags to be equal. But this is
>true only with GPT. With MSDOS, they have totally different meanings:
>- 'esp' means that the partition has the ESP type identifier.
>- 'boot' means that the partition has the active/boot indicator set. The UEFI
>specification says that this indicator is ignored by EFI boot.

ACK, I think you're correct here. Yay parted and its inconsistent
"flags" concept. :-(

>3) The patch considers LVM and RAID partitions not bootable. But both LVM and
>RAID superblocks can have a boot loader reserved area. Also, GRUB may boot
>them directly without a /boot partition.

Hmmm, maybe.

>4) It appears that partman fails to detect the specially crafted partition
>table on the installation media created with a debian image. Is it intended
>or fortunately unintentional ? If partman could see the EFI partition on the
>installation media, the detection of BIOS-bootable systems would fail.

That's not a worry for today... :-)

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
You lock the door
And throw away the key
There's someone in my head but it's not me 

Bug#1033913: partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode

[Pascal Hambourg]

partman-efi "Fix detection of BIOS-bootable systems" provides a 
significant improvement over previous behaviour. However I have a few 
comments.


1a) The patch assumes that a GPT disk may be BIOS-bootable only if it 
has a BIOS boot partition. But a GPT disk can be BIOS-bootable even 
without a BIOS boot partition:
- GRUB may be installed without a BIOS boot partition if /boot is a 
plain partition (using blocklists), even though it is less reliable so a 
BIOS boot partition is strongly recommended.
- Other BIOS boot loaders such as syslinux/extlinux do not need or use a 
BIOS boot partition.


1b) IIUC the patch fixes #1033913 because the disk selected for 
installation has received a new GPT disklabel without a BIOS boot 
partition, so further checking is skipped. But IMO the root cause of 
#1033913 is that changes are not committed to disk after setting the 
'boot' and 'esp' flags to the newly created ESP partition before 
stopping parted_server.

This can be seen in /var/log/partman:

/bin/autopartition-lvm
NEW_LABEL sda gpt
NEW_PARTITION 1 sda ext2 538MB (future ESP)
NEW_PARTITION 2 sda ext2 512MB (future /boot)
NEW_PARTITION 3 sda ext3 159GB (future LVM)
SET_FLAGS sda3 lvm
(user prompt to write changes to the disk)
COMMIT sda
...
/lib/partman/update.d/21efi_sync_flag
SET_FLAGS sda1 boot esp
...
/bin/perform_recipe_by_lvm
QUIT <- esp and boot flags have not been committed yet so are lost
...
/lib/partman/init.d/50efi
GET_FLAGS sda1 -> none

2) The patch considers the 'esp' and 'boot' flags to be equal. But this 
is true only with GPT. With MSDOS, they have totally different meanings:

- 'esp' means that the partition has the ESP type identifier.
- 'boot' means that the partition has the active/boot indicator set. The 
UEFI specification says that this indicator is ignored by EFI boot.


3) The patch considers LVM and RAID partitions not bootable. But both 
LVM and RAID superblocks can have a boot loader reserved area. Also, 
GRUB may boot them directly without a /boot partition.


4) It appears that partman fails to detect the specially crafted 
partition table on the installation media created with a debian image. 
Is it intended or fortunately unintentional ? If partman could see the 
EFI partition on the installation media, the detection of BIOS-bootable 
systems would fail.

Bug#1033913: partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode

[Pascal Hambourg]

Hello,

On 03/04/2023 at 21:50, Cyril Brulebois wrote:


  - It very much looks like the intermediary states are slightly
different when setting up LVM and when setting up encrypted LVM, and
the LVM case case leads to some confusion in partman-efi's
/lib/partman/init.d/50efi (which logs to /var/log/partman rather than
to /var/log/syslog): “Found 0 ESPs, 3 non-ESPs”.


Not sure if it is relevant, but the non-EFI partition calculation is 
wrong. Patch attached.diff --git a/init.d/efi b/init.d/efi
index 7eb8d2e..092ba80 100755
--- a/init.d/efi
+++ b/init.d/efi
@@ -71,23 +71,22 @@ for dev in /var/lib/partman/devices/*; do
 	close_dialog
 
 	for id in $partitions; do
-	efi=no
-	open_dialog GET_FLAGS $id
-	while { read_line flag; [ "$flag" ]; }; do
-		if [ "$flag" = boot ]; then
-			efi=yes
+		efi=no
+		open_dialog GET_FLAGS $id
+		while { read_line flag; [ "$flag" ]; }; do
+			if [ "$flag" = boot ]; then
+efi=yes
+			fi
+		done
+		close_dialog
+		if [ "$efi" = yes ]; then
+			mkdir -p $id
+			echo efi >$id/method
 			NUM_ESP=$(($NUM_ESP + 1))
-			# cannot break here
 		else
 			NUM_NO=$(($NUM_NO + 1))
 		fi
 	done
-	close_dialog
-	if [ "$efi" = yes ]; then
-		mkdir -p $id
-		echo efi >$id/method
-	fi
-	done
 done
 
 log "Found $NUM_ESP ESPs, $NUM_NO non-ESPs"

Bug#1033913: partman-auto-lvm: Broken "Guided - use entire disk and set up LVM" in UEFI mode

[Cyril Brulebois]

Package: partman-auto-lvm
Version: 87
Severity: serious
Justification: Maintainer says so

TL;DR: Answering “Yes” to the “Force UEFI installation?” makes sure the
installer pulls the right bootloader packages, despite misreading the
situation.

I've discovered this while testing D-I Bookworm RC 1 but also confirmed
it already existed in D-I Bookworm Alpha 2, and I'm therefore filing it
against the version found in the previous release (and deciding not to
block the Bookworm RC 1 release on it).



For baremetal tests on laptops requiring various firmware packages, I've
been using guided partitioning since forever, with one of these:
 - Guided - use entire disk
 - Guided - use entire disk and set up encrypted LVM

The former is used most of the time since it's slightly faster (fewer
prompts), while the latter is only used once in a while, to make sure a
“real” laptop-oriented install works fine (since every laptop should be
encrypted in my opinion).

Since I had just tested “Guided - use entire disk” in a virtual machine,
I decided to pick this instead when switching to the first laptop
(Asus Vivobook S14/S15 but that's very likely not a factor):
 - Guided - use entire disk and set up LVM

And… *WOW!*

The first surprise is this prompt:

Force UEFI installation?

This machine's firmware has started the installer in UEFI mode but
it looks like there may be existing operating systems already
installed using "BIOS compatibility mode". If you continue to
install Debian in UEFI mode, it might be difficult to reboot the
machine into any BIOS-mode operating systems later.

If you wish to install in UEFI mode and don't care about keeping the
ability to boot one of the existing systems, you have the option to
force that here. If you wish to keep the option to boot an existing
operating system, you should choose NOT to force UEFI installation
here.

which defaults to No.

That's very surprising since the only operating system prior to the
installation was a Debian system, which was getting entirely erased (due
to using the full disk), and was installed in UEFI mode anyway.

I went for the default choice, since we expect the installer to make
smart suggestions, and unsuspecting users shouldn't have to know better.

That means we end up with installing grub-pc instead of grub-efi-amd64
and shim, being prompted where to install GRUB, and of course when it's
time to reboot, the UEFI firmware rightfully refuses to boot anything
since there's absolutely no signature whatsoever, which isn't a great
idea under Secure Boot:

Secure Boot Violation

Invalid signature detected. Check Secure Boot Policy in Setup.


Some additional info:
 - As mentioned in TL;DR, this can be worked around by answering Yes to
   “Force UEFI installation?”.
 - It doesn't seem to be dependent on possible traces of an existing
   system prior to the installation: Debian installed on the entire disk
   or with encrypted LVM on the entire disk doesn't seem to make a
   difference. Starting with a wiped disk (writing ~ 2 GB worth of
   zeros at the beginning of the disk) doesn't make a difference either.
 - It very much looks like the intermediary states are slightly
   different when setting up LVM and when setting up encrypted LVM, and
   the LVM case case leads to some confusion in partman-efi's
   /lib/partman/init.d/50efi (which logs to /var/log/partman rather than
   to /var/log/syslog): “Found 0 ESPs, 3 non-ESPs”.
 - I'm filing this issue against partman-auto-lvm though, for
   discoverability purposes.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

Hi kibi,

On Sat, 01 Apr 2023 at 00:36:35 +0200, Cyril Brulebois wrote:
> Cyril Brulebois  (2023-03-26):
>> I'm happy to have the patches included, and I can definitely live with
>> possible temporary regressions (should that happen) that might arise
>> from having them.
>
> Pre-upload testing shows that the situation seems unchanged with
> 2:2.6.1-3~deb12u1: encrypted LVM still OOMK's with otherwise default
> options in the installer, when the VM is started with `kvm -m 1G`;
> that's fine with `kvm -m 1.2G` so at least it didn't seem to regress
> from the previous d-i release, and I've decided to continue d-i preps
> accordingly.

Ah right, reopened the upstream issue but forgot to follow-up here :-(
https://gitlab.com/cryptsetup/cryptsetup/-/issues/802#note_1328592911

As I wrote in the upstream BTS the patch appears to be incomplete,
AFAICT it helps in the PBKDF benchmark but just like you I also noticed
it still sometimes fails while running the keyslot key derivation.
Unfortunately I only found that out *after* uploading ~deb12u1…  It was
premature to think one could remove the errata from the bookworm d-i,
but at least I'm quite confident that does not make the OOMK issue
worse: it solves it at early stage but but it sometimes still trigger
later.

From a user perspective, I guess it's best to stick to the errata for
now (at least for the graphical installer; in text mode 1G RAM appear to
be enough according to my tests).  Depending on what upstream comes up
with we might suggest to fix that via s-p-u later.

Thanks to you, elbrus, an other Release Team members for everything!
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Hi again,

Cyril Brulebois  (2023-03-26):
> I'm happy to have the patches included, and I can definitely live with
> possible temporary regressions (should that happen) that might arise
> from having them.

Pre-upload testing shows that the situation seems unchanged with
2:2.6.1-3~deb12u1: encrypted LVM still OOMK's with otherwise default
options in the installer, when the VM is started with `kvm -m 1G`;
that's fine with `kvm -m 1.2G` so at least it didn't seem to regress
from the previous d-i release, and I've decided to continue d-i preps
accordingly.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Hi Guilhem,

Guilhem Moulin  (2023-03-26):
> In https://bugs.debian.org/1032235#107 elbrus (CC'ed) asked for a t-p-u
> upload of cryptsetup to fix a potential major regression should
> bookworm's src:argon2 ever be rebuilt with the bookworm toolchain.  The
> version currently in sid, 2:2.6.1-3, also includes 2 upstream patches to
> mitigate #1028250.  (“Mitigate”, because it only reduces the memory cost
> of the PBKDF on memory-constrained systems without swap.  This only buys
> time, and Milan argued that such systems are better off using a
> non-memory hard PBKDF.  I might propose a partman-crypto patch to that
> effect, but I guess it's too late for bookworm at this point.)
> 
> 2:2.6.1-3 (sid) and 2:2.6.1-1 (testing) differs as such:
> https://salsa.debian.org/cryptsetup-team/cryptsetup/-/compare/debian%2F2%252.6.1-1...debian%2F2%252.6.1-3
> 
> Would you rather have us exclude these backported upstream patches from
> the t-p-u upload or should we leave them in?  Concretely these patches
> set the maximum memory cost at ~256M on a system with 1G RAM, so in
> practice the memory pressure never exceeds 75% during installation
> (tested with d-i bookworm alpha 2 with updated src:cryptsetup udebs,
> graphical install).

Sorry, I haven't been able to follow upstream/downstream discussions too
closely, but I do appreciate everything that's been happening on that
front.

I'm happy to have the patches included, and I can definitely live with
possible temporary regressions (should that happen) that might arise
from having them.

Thanks for your help, as always.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

Hi kibi,

In https://bugs.debian.org/1032235#107 elbrus (CC'ed) asked for a t-p-u
upload of cryptsetup to fix a potential major regression should
bookworm's src:argon2 ever be rebuilt with the bookworm toolchain.  The
version currently in sid, 2:2.6.1-3, also includes 2 upstream patches to
mitigate #1028250.  (“Mitigate”, because it only reduces the memory cost
of the PBKDF on memory-constrained systems without swap.  This only buys
time, and Milan argued that such systems are better off using a
non-memory hard PBKDF.  I might propose a partman-crypto patch to that
effect, but I guess it's too late for bookworm at this point.)

2:2.6.1-3 (sid) and 2:2.6.1-1 (testing) differs as such:
https://salsa.debian.org/cryptsetup-team/cryptsetup/-/compare/debian%2F2%252.6.1-1...debian%2F2%252.6.1-3

Would you rather have us exclude these backported upstream patches from
the t-p-u upload or should we leave them in?  Concretely these patches
set the maximum memory cost at ~256M on a system with 1G RAM, so in
practice the memory pressure never exceeds 75% during installation
(tested with d-i bookworm alpha 2 with updated src:cryptsetup udebs,
graphical install).

Cheers
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#1032831: broken loginscreen when switching virtual console forth and back to tty7 out of an active kde session

[digitalwop]

Package: installation-reports


(Please provide enough information to help the Debian
maintainers evaluate the report efficiently - e.g., by filling
in the sections below.)

Boot method: USB
Image version: 
https://cdimage.debian.org/cdimage/bookworm_di_alpha2/amd64/iso-cd/
debian-bookworm-DI-alpha2-amd64-netinst.iso
Date: <06.03.2023>

Machine: its an acer travel mate 7750G laptop but i have this problem also on a 
getac s410
laptop
Partitions: 


Base System Installation Checklist:
[O] = OK, [E] = Error (please elaborate below), [ ] = didn't try it

Initial boot:   [O]
Detect network card:[O]
Configure network:  [O]
Detect media:   [O]
Load installer modules: [O]
Clock/timezone setup:   [O]
User/password setup:[O]
Detect hard drives: [O]
Partition hard drives:  [O]
Install base system:[O]
Install tasks:  [O]
Install boot loader:[O]
Overall install:[E]

Comments/Problems:

I did two new installations of debian 12 on two different laptops(listed 
above), this
problem i have on both on them.
If i start my machine i get the normal login screen of sddm on tty7.
Then i can login and the kde loading screen start and after that i have a 
normal kde
session.
But if i then change witch strg alt f2(for example) the virtual console and 
back to tty7 i am
logged out of my active kde session and get a sddm login screen with a 
"pre-typed greyed"
password.Its the same image just before the KDE loading screen appears after 
you type
your pw and hit enter like on a normal login.

I can only move my mouse around and pressing any key does nothing,no 
interaction with
the login screen is possible, but i can still switch tty back to like tty2.

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Control: retitle -1 cryptsetup might OOMK on low memory systems

Hi Guilhem,

Guilhem Moulin  (2023-02-18):
> By default the PBKDF benchmark caps the memory cost at 1GiB or half of
> the physical memory, whichever is smaller.  So indeed with 1G RAM and
> ~50% free one might trigger the OOM killer.  A workaround is to pass
> `--pbkdf-memory` with a suitable value (256M should be more than enough
> in that case) on memory-constrained systems, but cryptsetup should
> arguably adjust the memory cost on its own.  Reported the issue upstream
> at https://gitlab.com/cryptsetup/cryptsetup/-/issues/802 .

Thanks!

> But that's only for the first point.  Do you have a reproducer for the
> second point?  Tried 
> https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
> in a VM with 2G RAM [0] and chose the “encrypted LVM” scheme (both on
> the graphical and text install); the system refused to boot because the
> initramfs image contained an older e2fsck (“/dev/mapper/debian--vg-root
> has unsupported feature(s): FEATURE_C12”), however there was no problem
> after upgrading to sid at finish-install stage (and in both cases I
> could map the device, so neither bookworm's cryptsetup nor the kernel is
> at fault AFAICT).

I'm so sorry, I forgot I was hitting two separate issues at that time.
The one I can reproduce right now is the first one (OOMK), the second
one might have been some side effect of my mixing packages together…

I can definitely install with -m 1.1G (or higher), and also log into
the installed system.

Retitling to make all of this clearer.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Processed: Re: Bug#1028250: debian-installer: broken cryptsetup support

[Debian Bug Tracking System]

Processing control commands:

> retitle -1 cryptsetup might OOMK on low memory systems
Bug #1028250 [debian-installer] debian-installer: broken cryptsetup support
Changed Bug title to 'cryptsetup might OOMK on low memory systems' from 
'debian-installer: broken cryptsetup support'.

-- 
1028250: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1028250
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1028250: debian-installer: broken cryptsetup support

[Guilhem Moulin]

X-Debbugs-Cc: pkg-cryptsetup-de...@alioth-lists.debian.net

Hi kibi!

On Thu, 16 Feb 2023 at 20:14:20 +0100, Cyril Brulebois wrote:
> Cyril Brulebois  (2023-01-09):
>> Cyril Brulebois  (2023-01-08):
>>> I'm seeing at least two problems with cryptsetup while testing daily
>>> builds:
>>> - with 6.1.0-1 (currently getting into the archive), my very usual 1G
>>>  RAM / 5G storage setup can no longer get an automated encrypted LVM
>>>  setup created: cryptsetup triggers the OOMK while creating the
>>>  encrypted storage; that doesn't happen with 6.0.0-6. Not sure
>>>  cryptsetup itself is the culprit, it might just be more components or
>>>  heavier components on the kernel side, pushing memory to the limit.
>>> - with either kernel (and 1G RAM for 6.0.0-6, 2G RAM for 6.1.0-1 due
>>>  to the first point), I cannot boot into the installed system: I'm not
>>>  getting the LVM passphrase prompt, and the root device is therefore
>>>  missing.
>>>
>>> I haven't investigated either issue, and I'm not sure when I'll be able
>>> to. Help welcome.
>>>
>>> The first point could be waved aside with an errata entry; the second
>>> point is going to be a blocker for the next release.
>>
>> Trying to investigate the second one, I cannot replicate my earlier
>> results, with either a clean unstable daily build using 6.1.0-1 or with
>> D-I Bookworm Alpha 1; and besides cryptsetup uploads in early December,
>> I must admit a quick look around didn't suggest anything obvious that
>> could explain what I were getting… Bad luck, maybe; lowering severity
>> accordingly for the time being.
>
> Testing d-i built against testing udebs again, I can replicate this
> issue now. I suppose this might be some component getting bigger over
> time, and pushing the limit somehow. And the various builds I tried back
> in January might have been tiptoeing around that limit…
>
> Looking at `free` with this netboot-gtk mini.iso build, inside kvm, with
> 1G RAM, `used` is ~100M, `free` is 500+ M, and yet, cryptsetup gets
> OOMK'd.
>
> Is cryptsetup being stupid and miscomputing RAM requirements for that
> setup? (ISTR LUKS2 means heavier computation, tweaked depending on
> hardware, but I haven't followed that closely.)
>
> The memory pressure at this particular point of the installation process
> seems quite low, so crashing with free at 50% feels very wrong to me…

By default the PBKDF benchmark caps the memory cost at 1GiB or half of
the physical memory, whichever is smaller.  So indeed with 1G RAM and
~50% free one might trigger the OOM killer.  A workaround is to pass
`--pbkdf-memory` with a suitable value (256M should be more than enough
in that case) on memory-constrained systems, but cryptsetup should
arguably adjust the memory cost on its own.  Reported the issue upstream
at https://gitlab.com/cryptsetup/cryptsetup/-/issues/802 .

But that's only for the first point.  Do you have a reproducer for the
second point?  Tried 
https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
in a VM with 2G RAM [0] and chose the “encrypted LVM” scheme (both on
the graphical and text install); the system refused to boot because the
initramfs image contained an older e2fsck (“/dev/mapper/debian--vg-root
has unsupported feature(s): FEATURE_C12”), however there was no problem
after upgrading to sid at finish-install stage (and in both cases I
could map the device, so neither bookworm's cryptsetup nor the kernel is
at fault AFAICT).

Cheers
-- 
Guilhem.

[0] kvm -smp cpus=2 -cpu host -m 2G -object 
rng-random,id=rng0,filename=/dev/urandom \
  -device virtio-rng-pci,rng=rng0 -drive file=/tmp/disk.img,format=raw \
  -cdrom /tmp/debian-testing-amd64-netinst.iso -boot once=d


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Nem Jojic]

I haven’t look at cryptsetup.  What is the purpose?

Get Outlook for iOS<https://aka.ms/o0ukef>


From: Cyril Brulebois 
Sent: Thursday, February 16, 2023 14:21
To: 1028...@bugs.debian.org <1028...@bugs.debian.org>
Cc: cryptse...@packages.debian.org 
Subject: Bug#1028250: debian-installer: broken cryptsetup support

cc += cryptsetup maintainers (hi, long time no see!)

Cyril Brulebois  (2023-01-09):
> Cyril Brulebois  (2023-01-08):
> > I'm seeing at least two problems with cryptsetup while testing daily
> > builds:
> > - with 6.1.0-1 (currently getting into the archive), my very usual 1G
> > RAM / 5G storage setup can no longer get an automated encrypted LVM
> > setup created: cryptsetup triggers the OOMK while creating the
> > encrypted storage; that doesn't happen with 6.0.0-6. Not sure
> > cryptsetup itself is the culprit, it might just be more components or
> > heavier components on the kernel side, pushing memory to the limit.
> > - with either kernel (and 1G RAM for 6.0.0-6, 2G RAM for 6.1.0-1 due
> > to the first point), I cannot boot into the installed system: I'm not
> > getting the LVM passphrase prompt, and the root device is therefore
> > missing.
> >
> > I haven't investigated either issue, and I'm not sure when I'll be able
> > to. Help welcome.
> >
> > The first point could be waved aside with an errata entry; the second
> > point is going to be a blocker for the next release.
>
> Trying to investigate the second one, I cannot replicate my earlier
> results, with either a clean unstable daily build using 6.1.0-1 or with
> D-I Bookworm Alpha 1; and besides cryptsetup uploads in early December,
> I must admit a quick look around didn't suggest anything obvious that
> could explain what I were getting… Bad luck, maybe; lowering severity
> accordingly for the time being.

Testing d-i built against testing udebs again, I can replicate this
issue now. I suppose this might be some component getting bigger over
time, and pushing the limit somehow. And the various builds I tried back
in January might have been tiptoeing around that limit…

Looking at `free` with this netboot-gtk mini.iso build, inside kvm, with
1G RAM, `used` is ~100M, `free` is 500+ M, and yet, cryptsetup gets
OOMK'd.

Is cryptsetup being stupid and miscomputing RAM requirements for that
setup? (ISTR LUKS2 means heavier computation, tweaked depending on
hardware, but I haven't followed that closely.)

The memory pressure at this particular point of the installation process
seems quite low, so crashing with free at 50% feels very wrong to me…


Cheers,
--
Cyril Brulebois (k...@debian.org) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

cc += cryptsetup maintainers (hi, long time no see!)

Cyril Brulebois  (2023-01-09):
> Cyril Brulebois  (2023-01-08):
> > I'm seeing at least two problems with cryptsetup while testing daily
> > builds:
> >  - with 6.1.0-1 (currently getting into the archive), my very usual 1G
> >RAM / 5G storage setup can no longer get an automated encrypted LVM
> >setup created: cryptsetup triggers the OOMK while creating the
> >encrypted storage; that doesn't happen with 6.0.0-6. Not sure
> >cryptsetup itself is the culprit, it might just be more components or
> >heavier components on the kernel side, pushing memory to the limit.
> >  - with either kernel (and 1G RAM for 6.0.0-6, 2G RAM for 6.1.0-1 due
> >to the first point), I cannot boot into the installed system: I'm not
> >getting the LVM passphrase prompt, and the root device is therefore
> >missing.
> > 
> > I haven't investigated either issue, and I'm not sure when I'll be able
> > to. Help welcome.
> > 
> > The first point could be waved aside with an errata entry; the second
> > point is going to be a blocker for the next release.
> 
> Trying to investigate the second one, I cannot replicate my earlier
> results, with either a clean unstable daily build using 6.1.0-1 or with
> D-I Bookworm Alpha 1; and besides cryptsetup uploads in early December,
> I must admit a quick look around didn't suggest anything obvious that
> could explain what I were getting… Bad luck, maybe; lowering severity
> accordingly for the time being.

Testing d-i built against testing udebs again, I can replicate this
issue now. I suppose this might be some component getting bigger over
time, and pushing the limit somehow. And the various builds I tried back
in January might have been tiptoeing around that limit…

Looking at `free` with this netboot-gtk mini.iso build, inside kvm, with
1G RAM, `used` is ~100M, `free` is 500+ M, and yet, cryptsetup gets
OOMK'd.

Is cryptsetup being stupid and miscomputing RAM requirements for that
setup? (ISTR LUKS2 means heavier computation, tweaked depending on
hardware, but I haven't followed that closely.)

The memory pressure at this particular point of the installation process
seems quite low, so crashing with free at 50% feels very wrong to me…


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1029352: marked as done (netcfg: broken ifupdown support for wireless interfaces)

[Debian Bug Tracking System]

Your message dated Tue, 24 Jan 2023 18:21:00 +
with message-id 
and subject line Bug#1029352: fixed in netcfg 1.182
has caused the Debian Bug report #1029352,
regarding netcfg: broken ifupdown support for wireless interfaces
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
1029352: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1029352
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: netcfg
Severity: serious
Tags: patch
Justification: no networking after installation
X-Debbugs-Cc: ifupd...@packages.debian.org, w...@packages.debian.org

Hi,

I'm putting both ifupdown and wpa maintainers in the loop since I'd like
to get some feedback of the proposed resolution for this major issue
that's been annoying us for several releases:
 - wpa maintainers, you can jump to the second issue.
 - ifupdown maintainers, you can jump to the second and third issues.

I know we have a few open bug reports about this already, but I thought
I'd start a fresh bug, cutting to the chase.

I've pushed three commits in the pu/ifupdown+wireless branch:
  https://salsa.debian.org/installer-team/netcfg/-/commits/pu/ifupdown+wireless

Commit links:
 1. 
https://salsa.debian.org/installer-team/netcfg/-/commit/9494d7ec02b32538db842d88c105db1ab2a6201b
 2. 
https://salsa.debian.org/installer-team/netcfg/-/commit/247056dbb22e6eacbea6348c5c9a6951eab948bd
 3. 
https://salsa.debian.org/installer-team/netcfg/-/commit/5ca665c6c26346e3c9c37c2df6366e8e5d718238


# First issue

The first one explains why this could never work: we have a
netcfg/target_network_config template that's preseedable, without a
default value. The finish-install.d/55netcfg-copy-config script checks
whether a value is preseeded, and sets it if it isn't:
 - if NM is installed, we use NM;
 - otherwise, if the connection is wired, we use ifupdown;
 - otherwise, the connection is wireless, and we use… loopback?!

That means that at this point, /etc/network/interfaces contains entries
for the wireless interface, but it's getting emptied to only contain the
lo entry.

The proposed fix is to stick to ifupdown for wired and wireless
connections, with a small variation: if the connection is wireless and
secured (WPA or WEP with a non-empty passphrase), we chmod 400 /e/n/i
(in /target = the installed system); that used to be done a long while
ago, in the early days of the patch series but that got scratched along
the way.


# Second issue

Once that's fixed, if one gets both DHCP+SLAAC, the generated /e/n/i
looks like this (± comments/newlines):

allow-hotplug wlXXX

iface wlXXX inet dhcp

iface wlXXX inet6 auto
  wpa-ssid my-home-network
  wpa-psk  my-very-secret-passphrase

At best, we get SLAAC to work (IPv6 via RAs) but not DHCP: the first
iface stanza is missing wpa-* parameters, and we get a huge delay at
boot-up until dhclient finally times out.

My first instinct was to replicate wpa-* settings in both stanzas:

allow-hotplug wlXXX

iface wlXXX inet dhcp
  wpa-ssid my-home-network
  wpa-psk  my-very-secret-passphrase

iface wlXXX inet6 auto
  wpa-ssid my-home-network
  wpa-psk  my-very-secret-passphrase

But that doesn't work either: when the DHCP is brought up, wpa-* are
used by wpa_supplicant to do the authentication/association dance, DHCP
works. But right afterwards, when the SLAAC entry is processed, the
kernel and wpa_supplicant complain about being already associated, many
errors are logged and deassociation ultimately happens. This leaves us
with: an IPv4 address, no IPv6 address, and a DOWN interface.

What seems to work in the DHCP+SLAAC case is writing just the DHCP
entry, and let RAs bring up IPv6 connectivity on their own. I'm tempted
to go with this solution for bookworm, as that cannot be worse than the
current situation, but I'd be happy to get feedback from ifupdown and
wpa maintainers about the best way to write /e/n/i for DHCP + SLAAC.
As far as I can tell, ifupdown's interfaces manpage points at the many
packages that can deal with extra options, and wpasupplicant's
README.Debian is only about inet entries, inet6 is never mentioned.

→ ifupdown & wpa maintainers, comments welcome!


# Third issue

netcfg has some hotplug detection, which was last touched in 2005; it
tries to identify interfaces that are hotpluggable, and lists them under
/etc/network/devhotplug (in the installer's context), which is then used
to determine whether interfaces should be declared “auto” or
“allow-hotplug”. My rtl819

Bug#1029352: netcfg: broken ifupdown support for wireless interfaces

[Cyril Brulebois]

Control: tag -1 pending

Cyril Brulebois  (2023-01-21):
> I've pushed three commits in the pu/ifupdown+wireless branch:
>   
> https://salsa.debian.org/installer-team/netcfg/-/commits/pu/ifupdown+wireless

Updated branch:
  
https://salsa.debian.org/installer-team/netcfg/-/commits/pu/ifupdown+wireless-v2

> Commit links:
>  1. 
> https://salsa.debian.org/installer-team/netcfg/-/commit/9494d7ec02b32538db842d88c105db1ab2a6201b
>  2. 
> https://salsa.debian.org/installer-team/netcfg/-/commit/247056dbb22e6eacbea6348c5c9a6951eab948bd
>  3. 
> https://salsa.debian.org/installer-team/netcfg/-/commit/5ca665c6c26346e3c9c37c2df6366e8e5d718238

Updated commits, with changelog entries this time (no code change):
 1. 
https://salsa.debian.org/installer-team/netcfg/-/commit/aa62245f2960363f8b16f1f30d666936cc88bc83
 2. 
https://salsa.debian.org/installer-team/netcfg/-/commit/815cdfccaa5567fdf53594d47545d97c235de68e

Please note the third commit got moved to another branch:
  
https://salsa.debian.org/installer-team/netcfg/-/tree/pu/disable-hotplug-detection
  
https://salsa.debian.org/installer-team/netcfg/-/commit/9000be355b5e35134958c2655ec35bc75ba1b7e7

I suppose we can postpone wondering what to do about hotplug support
(netcfg currently believes everything is hotpluggable…) to a later time
(after bookworkm) given the broken “allow-hotplug” support at boot-up
(third issue) was an ifupdown regression in the end: #1022843.


My current plan includes more work on hw-detect; I'll upload netcfg with
commits from the ifupdown+wireless-v2 branch once I'm done (probably in
a few hours). Feedback regarding the netcfg commits is still very much
welcome (even after the upload, we can still tweak things before the
release).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Processed: Re: Bug#1029352: netcfg: broken ifupdown support for wireless interfaces

[Debian Bug Tracking System]

Processing control commands:

> tag -1 pending
Bug #1029352 [netcfg] netcfg: broken ifupdown support for wireless interfaces
Added tag(s) pending.

-- 
1029352: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1029352
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1029352: netcfg: broken ifupdown support for wireless interfaces

[Pascal Hambourg]

Hello,

On 21/01/2023 at 17:14, Cyril Brulebois wrote:


# Second issue

Once that's fixed, if one gets both DHCP+SLAAC, the generated /e/n/i
looks like this (± comments/newlines):

 allow-hotplug wlXXX

 iface wlXXX inet dhcp

 iface wlXXX inet6 auto
   wpa-ssid my-home-network
   wpa-psk  my-very-secret-passphrase

At best, we get SLAAC to work (IPv6 via RAs) but not DHCP: the first
iface stanza is missing wpa-* parameters, and we get a huge delay at
boot-up until dhclient finally times out.

(...)

→ ifupdown & wpa maintainers, comments welcome!


Comment from a simple user: I hate so say, but IMO ifupdown and/or 
/etc/network/interfaces format is broken by design. Stanzas mix link 
layer and network layer parameters and this causes various issues when 
you have several stanzas for the same interface. Link layer parameters 
should be applied once per interface regardless of the address family.



# Third issue

netcfg has some hotplug detection, which was last touched in 2005; it
tries to identify interfaces that are hotpluggable, and lists them under
/etc/network/devhotplug (in the installer's context), which is then used
to determine whether interfaces should be declared “auto” or
“allow-hotplug”. My rtl8192cu-based Wi-Fi USB dongle (shared from the
host via libvirt) ends up as “allow-hotplug”, which is problematic
because the module gets loaded after networking.service has returned,
failing to raise the wireless interface. Using “auto” makes ifupdown
wait a little more, and I'm getting my wireless interface configured.


That should not happen. After reading Alf's reply, I suspect a 
regression introduced in the latest bookwork ifupdown release:


   * networking.service: Add ExecStart=/sbin/ifup -a --allow=hotplug.
 Patch by "Oleg A. Arkhangelsky" 
 (Closes: #1022843)

According to the bug report, the patch also adds '--ignore-errors' so 
that ifup and networking.service return success even if hotplug 
interfaces are missing. I'm afraid it has a side effect: it also marks 
the interface as brought up, and when invoked by udev after the 
interface is created, ifup will not do anything.

Bug#1029352: netcfg: broken ifupdown support for ethernet interface

[Alf]

Similar Trouble here:

A smoothly running Debian-Bookworm system here with ethernet eno1 managed by 
ifupdown, wlp1s0 is managed by NetworkManager.

After today's bigger Update consisting of:

qttranslations5-l10n:amd64 (5.15.7-2, 5.15.8-2), qtbase5-dev-tools:amd64 
(5.15.7+dfsg-2, 5.15.8+dfsg-2), libqt5quickcontrols2-5:amd64 (5.15.7+dfsg-2, 
5.15.8+dfsg-2), qml-module-qtquick-shapes:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qml-module-qtquick-layouts:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5core5a:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qml-module-qtquick-window2:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qt5-gtk-platformtheme:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), solaar:amd64 
(1.1.8+dfsg-1, 1.1.8+dfsg-2), qml-module-qt-labs-settings:amd64 (5.15.7+dfsg-2, 
5.15.8+dfsg-2), python3-pyqt5.qtopengl:amd64 (5.15.7+dfsg-3+b1, 
5.15.7+dfsg-3+b3), libqt5charts5:amd64 (5.15.7-2, 5.15.8-2), libqt5qml5:amd64 
(5.15.7+dfsg-2, 5.15.8+dfsg-2), pyqt5-dev-tools:amd64 (5.15.7+dfsg-3+b1, 
5.15.7+dfsg-3+b3), libqt5x11extras5:amd64 (5.15.7-2, 5.15.8-2), 
libmediastreamer12:amd64 (1:5.1.64+dfsg-3+b2, 1:5.1.64+dfsg-3+b3), 
qml-module-qtquick-controls2:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5quickshapes5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qml-module-qt-labs-platform:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5network5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qml-module-qtquick-localstorage:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5dbus5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5multimedia5-plugins:amd64 (5.15.7-2, 5.15.8-2), python3-pyqt5:amd64 
(5.15.7+dfsg-3+b1, 5.15.7+dfsg-3+b3), libqt5quick5:amd64 (5.15.7+dfsg-2, 
5.15.8+dfsg-2), libqt5test5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qtwayland5:amd64 (5.15.7-2, 5.15.8-2), libqt5waylandcompositor5:amd64 
(5.15.7-2, 5.15.8-2), libqt5widgets5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
python3-scipy:amd64 (1.8.1-20, 1.8.1-21), qml-module-qtcharts:amd64 (5.15.7-2, 
5.15.8-2), libqt5gui5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5multimediagsttools5:amd64 (5.15.7-2, 5.15.8-2), python3-jsonschema:amd64 
(4.9.1-3, 4.10.3-1), libqt5multimedia5:amd64 (5.15.7-2, 5.15.8-2), 
libqt5printsupport5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), ifupdown:amd64 
(0.8.39+b1, 0.8.40), libqt5xml5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5opengl5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), linphone-desktop:amd64 
(4.4.10-2+b1, 4.4.10-2+b2), libqt5texttospeech5:amd64 (5.15.7-2, 5.15.8-2), 
xfonts-encodings:amd64 (1:1.0.4-2.1, 1:1.0.4-2.2), libqt5sql5:amd64 
(5.15.7+dfsg-2, 5.15.8+dfsg-2), qml-module-qtgraphicaleffects:amd64 (5.15.7-2, 
5.15.8-2), qml-module-qt-labs-folderlistmodel:amd64 (5.15.7+dfsg-2, 
5.15.8+dfsg-2), libqt5svg5:amd64 (5.15.7-2, 5.15.8-2), 
libqt5multimediawidgets5:amd64 (5.15.7-2, 5.15.8-2), 
libqt5quicktemplates2-5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qml-module-qtqml-models2:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qml-module-qtquick-privatewidgets:amd64 (5.15.7-2, 5.15.8-2), 
qtspeech5-speechd-plugin:amd64 (5.15.7-2, 5.15.8-2), 
qml-module-qtquick-controls:amd64 (5.15.7-2, 5.15.8-2), 
qml-module-qtquick-templates2:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
libqt5sql5-sqlite:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), libqt5qmlmodels5:amd64 
(5.15.7+dfsg-2, 5.15.8+dfsg-2), xfonts-scalable:amd64 (1:1.0.3-1.2, 
1:1.0.3-1.3), xfce4-screenshooter:amd64 (1.10.2-1, 1.10.3-1), 
libqt5designer5:amd64 (5.15.7-2, 5.15.8-2), qml-module-qtquick2:amd64 
(5.15.7+dfsg-2, 5.15.8+dfsg-2), qml-module-qtquick-dialogs:amd64 (5.15.7-2, 
5.15.8-2), libqt5qmlworkerscript5:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
python3-dbus.mainloop.pyqt5:amd64 (5.15.7+dfsg-3+b1, 5.15.7+dfsg-3+b3), 
libqt5waylandclient5:amd64 (5.15.7-2, 5.15.8-2), libqt5help5:amd64 (5.15.7-2, 
5.15.8-2), qml-module-qtqml:amd64 (5.15.7+dfsg-2, 5.15.8+dfsg-2), 
qt5-gtk2-platformtheme:amd64 (5.0.0+git23.g335dbec-4+b6, 
5.0.0+git23.g335dbec-4+b7)

and subsequent reboot I did not get any network connection.
systemctl restart networking
brought my ethernet interface up again and all was fine.

Applying Cyril's trick to substitute "allow-hotplug" by "auto" in 
/etc/network/interfaces resolved it for the moment.

Bug#1029352: netcfg: broken ifupdown support for wireless interfaces

[Cyril Brulebois]

Package: netcfg
Severity: serious
Tags: patch
Justification: no networking after installation
X-Debbugs-Cc: ifupd...@packages.debian.org, w...@packages.debian.org

Hi,

I'm putting both ifupdown and wpa maintainers in the loop since I'd like
to get some feedback of the proposed resolution for this major issue
that's been annoying us for several releases:
 - wpa maintainers, you can jump to the second issue.
 - ifupdown maintainers, you can jump to the second and third issues.

I know we have a few open bug reports about this already, but I thought
I'd start a fresh bug, cutting to the chase.

I've pushed three commits in the pu/ifupdown+wireless branch:
  https://salsa.debian.org/installer-team/netcfg/-/commits/pu/ifupdown+wireless

Commit links:
 1. 
https://salsa.debian.org/installer-team/netcfg/-/commit/9494d7ec02b32538db842d88c105db1ab2a6201b
 2. 
https://salsa.debian.org/installer-team/netcfg/-/commit/247056dbb22e6eacbea6348c5c9a6951eab948bd
 3. 
https://salsa.debian.org/installer-team/netcfg/-/commit/5ca665c6c26346e3c9c37c2df6366e8e5d718238


# First issue

The first one explains why this could never work: we have a
netcfg/target_network_config template that's preseedable, without a
default value. The finish-install.d/55netcfg-copy-config script checks
whether a value is preseeded, and sets it if it isn't:
 - if NM is installed, we use NM;
 - otherwise, if the connection is wired, we use ifupdown;
 - otherwise, the connection is wireless, and we use… loopback?!

That means that at this point, /etc/network/interfaces contains entries
for the wireless interface, but it's getting emptied to only contain the
lo entry.

The proposed fix is to stick to ifupdown for wired and wireless
connections, with a small variation: if the connection is wireless and
secured (WPA or WEP with a non-empty passphrase), we chmod 400 /e/n/i
(in /target = the installed system); that used to be done a long while
ago, in the early days of the patch series but that got scratched along
the way.


# Second issue

Once that's fixed, if one gets both DHCP+SLAAC, the generated /e/n/i
looks like this (± comments/newlines):

allow-hotplug wlXXX

iface wlXXX inet dhcp

iface wlXXX inet6 auto
  wpa-ssid my-home-network
  wpa-psk  my-very-secret-passphrase

At best, we get SLAAC to work (IPv6 via RAs) but not DHCP: the first
iface stanza is missing wpa-* parameters, and we get a huge delay at
boot-up until dhclient finally times out.

My first instinct was to replicate wpa-* settings in both stanzas:

allow-hotplug wlXXX

iface wlXXX inet dhcp
  wpa-ssid my-home-network
  wpa-psk  my-very-secret-passphrase

iface wlXXX inet6 auto
  wpa-ssid my-home-network
  wpa-psk  my-very-secret-passphrase

But that doesn't work either: when the DHCP is brought up, wpa-* are
used by wpa_supplicant to do the authentication/association dance, DHCP
works. But right afterwards, when the SLAAC entry is processed, the
kernel and wpa_supplicant complain about being already associated, many
errors are logged and deassociation ultimately happens. This leaves us
with: an IPv4 address, no IPv6 address, and a DOWN interface.

What seems to work in the DHCP+SLAAC case is writing just the DHCP
entry, and let RAs bring up IPv6 connectivity on their own. I'm tempted
to go with this solution for bookworm, as that cannot be worse than the
current situation, but I'd be happy to get feedback from ifupdown and
wpa maintainers about the best way to write /e/n/i for DHCP + SLAAC.
As far as I can tell, ifupdown's interfaces manpage points at the many
packages that can deal with extra options, and wpasupplicant's
README.Debian is only about inet entries, inet6 is never mentioned.

→ ifupdown & wpa maintainers, comments welcome!


# Third issue

netcfg has some hotplug detection, which was last touched in 2005; it
tries to identify interfaces that are hotpluggable, and lists them under
/etc/network/devhotplug (in the installer's context), which is then used
to determine whether interfaces should be declared “auto” or
“allow-hotplug”. My rtl8192cu-based Wi-Fi USB dongle (shared from the
host via libvirt) ends up as “allow-hotplug”, which is problematic
because the module gets loaded after networking.service has returned,
failing to raise the wireless interface. Using “auto” makes ifupdown
wait a little more, and I'm getting my wireless interface configured.

Looking at the contents of that /etc/network/devhotplug file, both lo
and ens3 (which are not “hotpluggable”?!) appear several times, so I'd
think the hotplug detection logic (via a script run when udev events
happen in the net subsystem) no longer works as it did in the early
2000s.

It's not clear to me what the downside of having “auto” everywhere would
be; adding some “auto” statements about interfaces that don't exist seem
to only lead to a few seconds being wasted at bootup (and the
networking.service unit being marked as failed), meaning that some
Ethernet or Wi-Fi dongl

Processed: Re: Bug#1028250: debian-installer: broken cryptsetup support

[Debian Bug Tracking System]

Processing control commands:

> severity -1 important
Bug #1028250 [debian-installer] debian-installer: broken cryptsetup support
Severity set to 'important' from 'serious'

-- 
1028250: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1028250
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Control: severity -1 important

Cyril Brulebois  (2023-01-08):
> I'm seeing at least two problems with cryptsetup while testing daily
> builds:
>  - with 6.1.0-1 (currently getting into the archive), my very usual 1G
>RAM / 5G storage setup can no longer get an automated encrypted LVM
>setup created: cryptsetup triggers the OOMK while creating the
>encrypted storage; that doesn't happen with 6.0.0-6. Not sure
>cryptsetup itself is the culprit, it might just be more components or
>heavier components on the kernel side, pushing memory to the limit.
>  - with either kernel (and 1G RAM for 6.0.0-6, 2G RAM for 6.1.0-1 due
>to the first point), I cannot boot into the installed system: I'm not
>getting the LVM passphrase prompt, and the root device is therefore
>missing.
> 
> I haven't investigated either issue, and I'm not sure when I'll be able
> to. Help welcome.
> 
> The first point could be waved aside with an errata entry; the second
> point is going to be a blocker for the next release.

Trying to investigate the second one, I cannot replicate my earlier
results, with either a clean unstable daily build using 6.1.0-1 or with
D-I Bookworm Alpha 1; and besides cryptsetup uploads in early December,
I must admit a quick look around didn't suggest anything obvious that
could explain what I were getting… Bad luck, maybe; lowering severity
accordingly for the time being.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1028250: debian-installer: broken cryptsetup support

[Cyril Brulebois]

Package: debian-installer
Severity: serious
Justification: not releasable

Hi,

I'm seeing at least two problems with cryptsetup while testing daily
builds:
 - with 6.1.0-1 (currently getting into the archive), my very usual 1G
   RAM / 5G storage setup can no longer get an automated encrypted LVM
   setup created: cryptsetup triggers the OOMK while creating the
   encrypted storage; that doesn't happen with 6.0.0-6. Not sure
   cryptsetup itself is the culprit, it might just be more components or
   heavier components on the kernel side, pushing memory to the limit.
 - with either kernel (and 1G RAM for 6.0.0-6, 2G RAM for 6.1.0-1 due
   to the first point), I cannot boot into the installed system: I'm not
   getting the LVM passphrase prompt, and the root device is therefore
   missing.

I haven't investigated either issue, and I'm not sure when I'll be able
to. Help welcome.

The first point could be waved aside with an errata entry; the second
point is going to be a blocker for the next release.


Cheers,
-- 
Cyril Brulebois -- Debian Consultant @ DEBAMAX -- https://debamax.com/

Bug#1012601: wireless-regdb: alternative broken on debian-installer install

[Dominique Martinet]

Package: wireless-regdb
Version: 2021.08.28-1
Severity: important
Tags: d-i
X-Debbugs-Cc: debian-boot@lists.debian.org

Dear Maintainer,

I've noticed after installing wireless-regdb on a fresh install the
package-provided file is not actually used (older version from
wireless-regdb-udeb is used), and update-alternative to select the
upstream version of the regdb also fails


(debian-boot@l.d.o: sorry for the explicit cc, I'm not really sure what
the d-i tag implies)


The problem is that the installer copies /lib/firmware/regulatory.db and
/lib/firmware/regulatory.db.p7s from the installer, and wireless-regdb
postinstall script does not overwrite these if they exist.

This can be reproduced in a minimal container:
root@00e7025e1eeb:/# mkdir /lib/firmware
root@00e7025e1eeb:/# touch /lib/firmware/regulatory.db
root@00e7025e1eeb:/# apt install -y wireless-regdb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  crda
The following NEW packages will be installed:
  wireless-regdb
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.9 kB of archives.
After this operation, 42.0 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 wireless-regdb all 
2020.04.29-2 [13.9 kB]
Fetched 13.9 kB in 0s (222 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package wireless-regdb.
(Reading database ... 6662 files and directories currently installed.)
Preparing to unpack .../wireless-regdb_2020.04.29-2_all.deb ...
Unpacking wireless-regdb (2020.04.29-2) ...
Setting up wireless-regdb (2020.04.29-2) ...
update-alternatives: using /lib/firmware/regulatory.db-debian to provide 
/lib/firmware/regulatory.db (regulatory.db) in auto mode
update-alternatives: warning: not replacing /lib/firmware/regulatory.db with a 
link
update-alternatives: warning: forcing reinstallation of alternative 
/lib/firmware/regulatory.db-debian because link group regulatory.db is broken
update-alternatives: warning: not replacing /lib/firmware/regulatory.db with a 
link
root@00e7025e1eeb:/# ls -l /lib/firmware/regulatory.db*
-rw-r--r-- 1 root root0 Jun 10 00:21 /lib/firmware/regulatory.db
-rw-r--r-- 1 root root 3764 Jun 30  2020 /lib/firmware/regulatory.db-debian
-rw-r--r-- 1 root root 3764 Jun 30  2020 /lib/firmware/regulatory.db-upstream
lrwxrwxrwx 1 root root   35 Jun 10 00:21 /lib/firmware/regulatory.db.p7s -> 
/etc/alternatives/regulatory.db.p7s
-rw-r--r-- 1 root root 1249 Jun 30  2020 /lib/firmware/regulatory.db.p7s-debian
-rw-r--r-- 1 root root 1182 Jun 30  2020 
/lib/firmware/regulatory.db.p7s-upstream
root@00e7025e1eeb:/# update-alternatives --config regulatory.db
There are 2 choices for the alternative regulatory.db (providing 
/lib/firmware/regulatory.db).

  SelectionPath  Priority   Status

  0/lib/firmware/regulatory.db-debian 100   auto mode
* 1/lib/firmware/regulatory.db-debian 100   manual mode
  2/lib/firmware/regulatory.db-upstream   50manual mode

Press  to keep the current choice[*], or type selection number:
update-alternatives: warning: forcing reinstallation of alternative 
/lib/firmware/regulatory.db-debian because link group regulatory.db is broken
update-alternatives: warning: not replacing /lib/firmware/regulatory.db with a 
link
root@00e7025e1eeb:/# ls -l /lib/firmware/regulatory.db
-rw-r--r-- 1 root root0 Jun 10 00:21 /lib/firmware/regulatory.db


Running with --force removes the original file with a warning and works:
root@00e7025e1eeb:/# update-alternatives --force --config regulatory.db
There are 2 choices for the alternative regulatory.db (providing 
/lib/firmware/regulatory.db).

  SelectionPath  Priority   Status

  0/lib/firmware/regulatory.db-debian 100   auto mode
* 1/lib/firmware/regulatory.db-debian 100   manual mode
  2/lib/firmware/regulatory.db-upstream   50manual mode

Press  to keep the current choice[*], or type selection number:
update-alternatives: warning: forcing reinstallation of alternative 
/lib/firmware/regulatory.db-debian because link group regulatory.db is broken
root@00e7025e1eeb:/# ls -l /lib/firmware/regulatory.db
lrwxrwxrwx 1 root root 31 Jun 10 00:23 /lib/firmware/regulatory.db -> 
/etc/alternatives/regulatory.db




The original cause for this is that deian-installer copies the files
because of the combinaison of these two:
https://salsa.debian.org/installer-team/debian-installer/-/blob/master/build/pkg-lists/base#L34
 - wireless-regdb-udeb contains the firmwares as regular files
https://salsa.debian.org/installer-team/hw-detect/-/blob/master/hw-detect.post-

Re: Bug#1010161: x11-xkb-utils-udeb: broken setxkbmap within the installer

[Cyril Brulebois]

Julien Cristau  (2022-04-25):
> Control: tag -1 upstream patch
> Control: forwarded -1 
> https://gitlab.freedesktop.org/xorg/app/setxkbmap/-/merge_requests/4
> 
> On Mon, Apr 25, 2022 at 03:44:59PM +0200, Cyril Brulebois wrote:
> > And thanks again for the quick turnaround for the libxrandr2 udeb
> > addition. The next issue is is_xwayland() erroring out at runtime, with
> > BadRROutput on the RRGetOutputInfo call. This breaks setxkbmap in the
> > installer, and prevents the graphical installer from going further than
> > 2 steps.
> > 
> Fixed with the above MR.

Thanks; for those following at home: verified in a d-i context.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1010161: x11-xkb-utils-udeb: broken setxkbmap within the installer

[Cyril Brulebois]

Package: x11-xkb-utils-udeb
Version: 7.7+6+b1
Severity: serious
Tags: d-i
Justification: breaks graphical installer
X-Debbugs-Cc: debian-boot@lists.debian.org

Hi X people,

And thanks again for the quick turnaround for the libxrandr2 udeb
addition. The next issue is is_xwayland() erroring out at runtime, with
BadRROutput on the RRGetOutputInfo call. This breaks setxkbmap in the
installer, and prevents the graphical installer from going further than
2 steps.

I think we would be happy to set a specific environment variable for you
to notice and exit early if desired; or we can check if such a variable
exists already for you to use.

Thanks for your help!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant

Re: simple-cdd: Firmware inclusion broken for Bullseye?

[Cyril Brulebois]

Hi Scott,

Scott  (2021-11-19):
> I have a feeling that non-free firmware inclusion is not working in
> simple-cdd on Bullseye.

kibi@tokyo:~$ apt-cache show simple-cdd
…
Maintainer: Simple-CDD Developers 
…

Cc-ing accordingly.

> Can anyone confirm that they have a simple-cdd project which has been
> observed to include non-free firmware in the image when run on Bullseye?
> 
> I have a simple-cdd project which includes non-free firmware when run on
> Buster, but fails to include non-free firmware when run on Bullseye.
> 
> It's been suggested to me to include `export FORCE_FIRMWARE=1` in the conf
> file, but this still doesn't result in the firmware being included in the
> image.
> 
> What I have observed, if this is useful information, is that:
> 
> 1. When I run my project without FORCE_FIRMWARE, I see a file called
> `/tmp/cd-build/bullseye/firmware-packages` which includes the
> name of the firmware I want to add to the image.
> 
> 2. When I run my project with FORCE_FIRMWARE, I see a file called
> `/tmp/cd-build/bullseye/firmware-packages` AND a file called
> `/tmp/cd-build/bullseye/tasks/firmware` and both these files
> include the name of the firmware I want to add to the image.
> 
> But the firmware never finds its way into the image.
> 
> If it's any use, here's the project:
> https://github.com/countermeasure/basic-box/
> 
> Thanks,
> 
> Scott


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

simple-cdd: Firmware inclusion broken for Bullseye?

[Scott]

Hi,

I have a feeling that non-free firmware inclusion is not working in 
simple-cdd on Bullseye.


Can anyone confirm that they have a simple-cdd project which has been 
observed to include non-free firmware in the image when run on Bullseye?


I have a simple-cdd project which includes non-free firmware when run on 
Buster, but fails to include non-free firmware when run on Bullseye.


It's been suggested to me to include `export FORCE_FIRMWARE=1` in the 
conf file, but this still doesn't result in the firmware being included 
in the image.


What I have observed, if this is useful information, is that:

1. When I run my project without FORCE_FIRMWARE, I see a file called 
`/tmp/cd-build/bullseye/firmware-packages` which includes 
the name of the firmware I want to add to the image.


2. When I run my project with FORCE_FIRMWARE, I see a file called 
`/tmp/cd-build/bullseye/firmware-packages` AND a file called 
`/tmp/cd-build/bullseye/tasks/firmware` and both these files 
include the name of the firmware I want to add to the image.


But the firmware never finds its way into the image.

If it's any use, here's the project: 
https://github.com/countermeasure/basic-box/


Thanks,

Scott

Bug#985481: marked as done (debootstrap: Detection of docker container is broken with cgroup v2)

[Debian Bug Tracking System]

Your message dated Tue, 02 Nov 2021 11:33:43 +
with message-id 
and subject line Bug#985481: fixed in debootstrap 1.0.125
has caused the Debian Bug report #985481,
regarding debootstrap: Detection of docker container is broken with cgroup v2
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
985481: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=985481
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: debootstrap
Version: 1.0.123
Severity: normal
Tags: patch
User: de...@kali.org
Usertags: origin-kali

Dear Maintainer,

The code that is meant to detect if debootstrap is running from within a
docker container is broken with cgroup v2. Talking about this particular
function and line in the file `functions`:

detect_container () {
[...]
elif grep -qs '[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo; 
then
CONTAINER="docker"

This code only works for cgroup v1.

After some research, and also after looking into the code of
systemd-detect-virt, it seems that the right way to detect a docker
container these days is to check for the file '/.dockerenv'.

Hence I'm proposing this patch:
https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/52

Thanks!


-- More debug logs:

Here's what I get on current Debian sid:

$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.10.0-4-amd64 root=<> tro quiet

$ mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 
(rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)

$ sudo docker run --rm -it debian:testing grep 
'[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo
 no ouput, the detection code is broken!

$ sudo docker run --rm -it debian:testing ls -l /.dockerenv
-rwxr-xr-x 1 root root 0 Mar 19 02:37 /.dockerenv

Just out of curiosity, I tried to get the current detection code to
work, by booting my system with cgroup v1 only. This is done by setting
the two boot parameters `systemd.unified_cgroup_hierarchy=0` and
`systemd.legacy_systemd_cgroup_controller=1`.

Here are the logs:

$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.10.0-4-amd64 root=<> ro quiet 
systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=1

$ mount | grep cgroup
tmpfs on /sys/fs/cgroup type tmpfs 
(ro,nosuid,nodev,noexec,size=4096k,nr_inodes=1024,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup 
(rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/memory type cgroup 
(rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/rdma type cgroup 
(rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup 
(rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup 
(rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup 
(rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup 
(rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/pids type cgroup 
(rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/cpuset type cgroup 
(rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/freezer type cgroup 
(rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/perf_event type cgroup 
(rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup 
(rw,nosuid,nodev,noexec,relatime,hugetlb)

$ sudo docker run --rm -it debian:testing grep 
'[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo
795 794 0:29 /docker/<> /sys/fs/cgroup/systemd 
ro,nosuid,nodev,noexec,relatime master:10 - cgroup cgroup 
rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
797 794 0:33 /docker/<> /sys/fs/cgroup/memory 
ro,nosuid,nodev,noexec,relatime master:15 - cgroup cgroup rw,memory
818 794 0:35 /docker/<> /sys/fs/cgroup/net_cls,net_prio 
ro,nosuid,nodev,noexec,relatime master:17 - cgroup cgroup rw,net_cls,net_prio
819 794 0:36 /docker/<> /sys/fs/cgroup/cpu,cpuacct 
ro,nosuid,nodev,noexec,relatime master:18 - cgroup cgroup rw,cpu,cpuacct
853 794 0:37 /docker/<> /sys/fs/cgroup/blkio 
ro,nosuid,nodev,noexec,relatime master:19 - cgroup cgroup rw,blkio
854 794 0:38 /docker/<> /sys/fs/cgroup/devices 
ro,nosuid,nodev,noexec,relatime master:20 - cgroup cgroup rw,devices
872 794 0:39 /do

Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[Joseph Carter]

Control: retitle -1 FTBFS: uses tempfile in build, appears to use deprecated 
which(1)

A closer look indicates that tempfile is only actually used to build the 
package. The apparent use of which(1) is actually a shell function, however…

On Wed, Aug 18, 2021, at 16:18, Cyril Brulebois wrote:
> > Presumably /installer-team/console-setup would be a better package to
> > patch, unless cdebconf uses tempfile somehow. 😉 I'll see what I can
> > do this evening. 🙂
> 
> Sure thing, miscompleted in my browser history plus slightly distracted,
> sorry.

As suggested before, I'm fluent in IRC, so I read what you meant. 😁 

MY goof was breaking my initramfs, being _completely_ unable to read the tiny 
font, reverting to backup, futzing with it, seeing tempfile and which being 
called in setupcon, and having to go to work before I could dig much deeper.

As noted, tempfile is only actually used to build the package. Fixed that. Left 
the function in setupcon alone.

The apparent use of which all over the place is also a shell function, but the 
shell function just emulates command -v, so I subbed that out and removed the 
functions while I was at it. Tested the result on amd64 with amdgpu added to 
initramfs for early KMS so that I can take advantage of Plymouth for status 
messages I have some chance of being able to use, and a crypttab prompt that 
doesn't get lost in the impossibly small console text.

I realized while composing the email that my patch has some trailing whitespace 
removal—nvim is configured to do that to source files as it's usually desirable 
for git sanity, but I should've trimmed those hunks out of the patch and 
retested. I don't have time to do that tonight because I can't conveniently 
just reboot right now, so I'll send you the patch as tested. Feel free to omit 
purely whitespace change hunks.

You can maybe guess the major thrust of my attempts to fiddle with 
console-setup around the time this all happened: You don't even try to set the 
font in the initramfs anymore, and I've never actually figured out why Ubuntu's 
combination of console-setup and plymouth manages to just work and Debian's 
just doesn't—I've spent a little time prodding at packages from both.

I mostly know my way around a Debian system I think—when I reinstalled this 
machine about two weeks ago, I found that neither old nor new installer was 
really set up to preserve my LUKS devices with LVM on them, keep some 
partitions and format others, etc … so I just grabbed a live USB and installed 
the system with debootstrap. It … was faster than trying ti figure out if or 
how Calamares could do that, and I know the shell of the old debian installer 
didn't provide shell UI I'd need to open the LUKS partitions so the partitioner 
could use them without obliterating anything "helpfully" for me. *shrug*

I've gotten way off topic now, but I think I just need to write a hook to throw 
in setfont and the font to set along with some quick and dirty initramfs 
scripts to make sure it dances around plymouth. I had to do something similar 
with a MBP to work around proprietary Apple crap.

Anyway, hope this is useful. Salsa didn't exist when last I might've had an 
account on it to be able to actually finish this up as a PR. I've considered 
changing that a time or two. Perhaps when the Covidium passes.

Joseph

diff -Nru console-setup.orig/debian/console-setup.config console-setup/debian/console-setup.config
--- console-setup.orig/debian/console-setup.config	2021-08-18 22:17:59.892444554 -0700
+++ console-setup/debian/console-setup.config	2021-08-18 22:42:21.524213193 -0700
@@ -64,7 +64,7 @@
 # fontsets='Arabic-Fixed15
 # Arabic-Fixed16
 # Arabic-VGA14
-# ... 
+# ...
 # Vietnamese-Fixed18
 # '
 
@@ -104,18 +104,6 @@
 
 db_capb backup
 
-which () {
-local IFS
-IFS=:
-for i in $PATH; do
-	if [ -f "$i/$1" -a -x "$i/$1" ]; then
-	echo "$i/$1"
-	return 0
-	fi
-done
-return 1
-}
-
 available_fontfaces () {
 local prefix
 case "$CODESET" in
@@ -195,14 +183,14 @@
 }
 
 kernel=unknown
-if which uname >/dev/null; then
+if command -v uname >/dev/null; then
 case "`uname`" in
 *Linux*) kernel=linux ;;
 *FreeBSD*) kernel=freebsd ;;
 esac
 fi
 
-if which locale 2>/dev/null >/dev/null; then
+if command -v locale 2>/dev/null >/dev/null; then
 eval `locale`
 fi
 
@@ -217,7 +205,7 @@
 if [ "$locale" = C ]; then
 CHARMAP=ISO-8859-15
 charmap_priority=high
-elif which locale 2>/dev/null >/dev/null; then
+elif command -v locale 2>/dev/null >/dev/null; then
 CHARMAP=`locale charmap`
 else
 CHARMAP=unknown
diff -Nru console-setup.orig/debian/console-setup-udeb.postinst console-setup/debian/console-setup-udeb.postinst
--- console-setup.orig/debian/console-setup-udeb.postinst	2021-08-18 22:17:59.892444554 -0700
+++ console-setup/debian/console-setup-udeb.postinst	2021-08-18 22:43:48.226081884 -0700
@@ -5,20 +5,6 @@
 # Source debconf library.
 . /usr/share/debconf/confmodule

Processed: Re: Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[Debian Bug Tracking System]

Processing control commands:

> retitle -1 FTBFS: uses tempfile in build, appears to use deprecated which(1)
Bug #992457 [console-setup] Broken by removal of tempfile in debianutils
Changed Bug title to 'FTBFS: uses tempfile in build, appears to use deprecated 
which(1)' from 'Broken by removal of tempfile in debianutils'.

-- 
992457: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=992457
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[Cyril Brulebois]

Joseph Carter  (2021-08-18):
> Presumably /installer-team/console-setup would be a better package to
> patch, unless cdebconf uses tempfile somehow. 😉 I'll see what I can
> do this evening. 🙂

Sure thing, miscompleted in my browser history plus slightly distracted,
sorry.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[Joseph Carter]

On Wed, Aug 18, 2021, at 14:01, Cyril Brulebois wrote:
> T. Joseph Carter  (2021-08-18):
> > It's debianutils' bug, really, and the bugs keep getting filed (and
> > resolved), but there's a half a dozen packages on my system that are
> > broken by it. Yours happens to be used at boot time and for general
> > system operation.
> 
> It's an ongoing conversation on IRC apparently, and yes, some kind of
> advance warning would have been appreciated.

O yes, I'm sure there is. I've missed those over the years. 🍿
 

> That being said, it's not entirely crazy to attempt such changes very
> early in the release cycle, and if we ought to move away from those
> tools, I don't mind much.

Yes, but you "try" to do that by marking the packages deprecated and filing 
bugs that version 5, due out X weeks, and ask them to make changes or allow 
NMU.  Ideally, you then keep it around for a release or so AFTER you make 
Debian no longer dependent upon the tools.

Dunno who else would miss tempfile, but I'm kinda partial to which since 
command -v will NOT give you the path to a file if you typically alias that 
file, and type -P is not POSIX and does not work with dash.


> > If you're busy and debianutils' change doesn't get reverted, I can
> > prepare a patch. It's literally replacing tempfile and which with
> > their more generic equivalents, after all.
> 
> I think we'd be happy to have a patch or a merge request to review, even
> more so if you've tested it on a real system.
> 
> The git repo is at:
>   https://salsa.debian.org/installer-team/cdebconf/
> 
> but a patch against the source package would be fine too.
> 
> Thanks for the heads-up!

Presumably /installer-team/console-setup would be a better package to patch, 
unless cdebconf uses tempfile somehow. 😉 I'll see what I can do this evening. 🙂

Joseph

Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[Cyril Brulebois]

Control: retitle -1 Broken by removal of tempfile in debianutils

Hi,

T. Joseph Carter  (2021-08-18):
> Debianutils >= 5 removes tempname and puts a deprecation notice on the
> which command. The setupcon script (at least) uses both of these,
> causing people's initramfs's to be subtly broken and leaving them
> without a keymap in the event of a boot error., Since the maintainer
> of Debianutils seems to be content to put out fires as they come up
> with the excuse that the stable version of Debian declares these to be
> deprecated (y'know, the one that was released a week ago at time of
> writing), it is apparently incumbent upon others to fix this in their
> packages.
> 
> It's debianutils' bug, really, and the bugs keep getting filed (and
> resolved), but there's a half a dozen packages on my system that are
> broken by it. Yours happens to be used at boot time and for general
> system operation.

It's an ongoing conversation on IRC apparently, and yes, some kind of
advance warning would have been appreciated.

That being said, it's not entirely crazy to attempt such changes very
early in the release cycle, and if we ought to move away from those
tools, I don't mind much.

> If you're busy and debianutils' change doesn't get reverted, I can
> prepare a patch. It's literally replacing tempfile and which with
> their more generic equivalents, after all.

I think we'd be happy to have a patch or a merge request to review, even
more so if you've tested it on a real system.

The git repo is at:
  https://salsa.debian.org/installer-team/cdebconf/

but a patch against the source package would be fine too.

Thanks for the heads-up!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Processed: Re: Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[Debian Bug Tracking System]

Processing control commands:

> retitle -1 Broken by removal of tempfile in debianutils
Bug #992457 [console-setup] Broken by irresponsible removal of tempfile in 
debianutils
Changed Bug title to 'Broken by removal of tempfile in debianutils' from 
'Broken by irresponsible removal of tempfile in debianutils'.

-- 
992457: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=992457
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#992457: Broken by irresponsible removal of tempfile in debianutils

[T. Joseph Carter]

Package: console-setup
Version: 1.205
Severity: important
X-Debbugs-Cc: 

Debianutils >= 5 removes tempname and puts a deprecation notice on the
which command. The setupcon script (at least) uses both of these,
causing people's initramfs's to be subtly broken and leaving them
without a keymap in the event of a boot error., Since the maintainer of
Debianutils seems to be content to put out fires as they come up with
the excuse that the stable version of Debian declares these to be
deprecated (y'know, the one that was released a week ago at time of
writing), it is apparently incumbent upon others to fix this in their
packages.

It's debianutils' bug, really, and the bugs keep getting filed (and
resolved), but there's a half a dozen packages on my system that are
broken by it. Yours happens to be used at boot time and for general
system operation.

If you're busy and debianutils' change doesn't get reverted, I can
prepare a patch. It's literally replacing tempfile and which with their
more generic equivalents, after all.

-- System Information:
Debian Release: 11.0
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-8-amd64 (SMP w/16 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, 
TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages console-setup depends on:
ii  console-setup-linux 1.205
ii  debconf 1.5.77
ii  keyboard-configuration  1.205
ii  xkb-data2.33-1

console-setup recommends no packages.

Versions of packages console-setup suggests:
ii  locales   2.31-16
ii  lsb-base  11.1.0

Versions of packages keyboard-configuration depends on:
ii  debconf 1.5.77
ii  liblocale-gettext-perl  1.07-4+b1

Versions of packages console-setup-linux depends on:
ii  init-system-helpers 1.60
ii  kbd 2.3.0-3
ii  keyboard-configuration  1.205

console-setup-linux suggests no packages.

Versions of packages console-setup is related to:
pn  console-common
pn  console-data  
pn  console-tools 
ii  gnome-control-center  1:3.38.4-1
ii  kbd   2.3.0-3
ii  systemd   247.9-1

-- debconf information:
  keyboard-configuration/layout:
  keyboard-configuration/unsupported_config_options: true
  keyboard-configuration/ctrl_alt_bksp: false
* console-setup/codeset47: # Latin1 and Latin5 - western Europe and Turkic 
languages
* console-setup/fontsize-fb47: 16x32 (framebuffer only)
  keyboard-configuration/store_defaults_in_debconf_db: true
  debian-installer/console-setup-udeb/title:
  keyboard-configuration/xkb-keymap: us
  console-setup/fontsize: 16x32
  keyboard-configuration/unsupported_layout: true
  keyboard-configuration/switch: No temporary switch
  keyboard-configuration/model: Generic 105-key PC (intl.)
  keyboard-configuration/toggle: No toggling
  console-setup/framebuffer_only:
  keyboard-configuration/layoutcode: us
  keyboard-configuration/optionscode:
  keyboard-configuration/compose: No compose key
  keyboard-configuration/modelcode: pc105
  keyboard-configuration/variantcode:
  keyboard-configuration/unsupported_config_layout: true
  console-setup/fontsize-text47: 16x32 (framebuffer only)
  console-setup/store_defaults_in_debconf_db: true
  keyboard-configuration/altgr: The default for the keyboard layout
  keyboard-configuration/unsupported_options: true
  keyboard-configuration/other:
* console-setup/fontface47: TerminusBold
  console-setup/guess_font:
* keyboard-configuration/variant: English (US)
* console-setup/charmap47: UTF-8
  console-setup/codesetcode: Lat15
  console-setup/use_system_font:

Bug#991625: marked as done (debootstrap: extra-suites= broken; attempts Package.* fetches from primary suite)

[Debian Bug Tracking System]

Your message dated Thu, 29 Jul 2021 20:00:46 +0200
with message-id <8df29295-1dd8-991a-257d-b285ed927...@molgen.mpg.de>
and subject line Re: Bug#991625: debootstrap: extra-suites= broken; attempts 
Package.* fetches from primary suite
has caused the Debian Bug report #991625,
regarding debootstrap: extra-suites= broken; attempts Package.* fetches from 
primary suite
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
991625: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=991625
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: debootstrap
Version: 1.0.123
Severity: important
X-Debbugs-Cc: deb...@iam.tj

Dear Maintainer,

   * What led up to the situation?

Attempting to reproduce another bug reported on IRC about Ubuntu and
debootstrap

   * What exactly did you do (or not do) that was effective (or
 ineffective)?

/usr/sbin/debootstrap --verbose --arch=amd64
--extra-suites=focal-updates focal ./focal-chroot
http://archive.ubuntu.com/ubuntu

   * What was the outcome of this action?

I: Checking Release signature
I: Valid Release signature (key id
F6ECB3762474EDA9D21B7022871920D1991BC93C)
I: Validating Packages I: Retrieving Packages I: Validating Packages W:
Retrying failed download of
http://archive.ubuntu.com/ubuntu/dists/focal/main/binary-amd64/Packages.xz
I: Retrieving Packages

   * What outcome did you expect instead?

File to be fetched correctly

I inserted debug code and used set -x to narrow down the bug and found
that when EXTRA_SUITES is set, in function download_release_indices(),
in the loop "for s in $SUITE $EXTRA_SUITES; do" that the value of 's' is
being changed in the call to validate_suite() where it does "for s in
$SUITE $EXTRA_SUITES; do" and the first test is successful "if [ "$s" =
"$suite" ] || [ "$s" = "$CODENAME" ]; then return 0".

In the example case this resets the suite being processed from
'focal-updates' to 'focal' and so files are downloaded from the focal
suite but the hashes tested against are from the focal-updates
Releases file, causing the validation to fail.

Patch to fix the issue at the end of this email.


-- System Information:
Debian Release: 11.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 5.13.0-soggy-02736-g7f1d227e86a3 (SMP w/4 CPU threads;
PREEMPT)
Kernel taint flags: TAINT_WARN
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE
not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages debootstrap depends on:
ii  wget  1.21-1+b1

Versions of packages debootstrap recommends:
ii  arch-test   0.17-1
ii  debian-archive-keyring  2021.1.1
ii  gnupg   2.2.27-2

Versions of packages debootstrap suggests:
pn  squid-deb-proxy-client  
pn  ubuntu-archive-keyring  

-- no debconf information

--
commit 23bb0a9f3c39b1f23f6c27bd7e8e85f43d7a0316 (HEAD -> master)
Author: Tj 
Date:   Wed Jul 28 21:27:01 2021 +0100

fix: download correct extra-suites Packages files

diff --git a/functions b/functions
index 09d93f4..ec34d84 100644
--- a/functions
+++ b/functions
@@ -551,7 +551,7 @@ extract_release_components () {
  CODENAME=""
 validate_suite () {
-   local reldest suite
+   local reldest suite s
reldest="$1"
 CODENAME=$(sed -n "s/^Codename: *//p" "$reldest")
--- End Message ---
--- Begin Message ---

Version: 1.0.124


Dear TJ,


Am 28.07.21 um 22:33 schrieb TJ:

[…]


Patch to fix the issue at the end of this email.


Thank you for debugging. The same fix was applied in commit ff63b19a 
(functions: Turn for loops variables into locals), which is in release 
1.0.124, which is in the experimental suite. (Hopefully it’s already 
that I am closing the report.)



Kind regards,

Paul


[1]: 
https://salsa.debian.org/installer-team/debootstrap/-/commit/ff63b19a7a677de4dea484d9b1e59f446afa2a08--- End Message ---

Bug#991625: debootstrap: extra-suites= broken; attempts Package.* fetches from primary suite

[TJ]

Package: debootstrap
Version: 1.0.123
Severity: important
X-Debbugs-Cc: deb...@iam.tj

Dear Maintainer,

   * What led up to the situation?

Attempting to reproduce another bug reported on IRC about Ubuntu and
debootstrap

   * What exactly did you do (or not do) that was effective (or
 ineffective)?

/usr/sbin/debootstrap --verbose --arch=amd64
--extra-suites=focal-updates focal ./focal-chroot
http://archive.ubuntu.com/ubuntu

   * What was the outcome of this action?

I: Checking Release signature
I: Valid Release signature (key id
F6ECB3762474EDA9D21B7022871920D1991BC93C)
I: Validating Packages I: Retrieving Packages I: Validating Packages W:
Retrying failed download of
http://archive.ubuntu.com/ubuntu/dists/focal/main/binary-amd64/Packages.xz
I: Retrieving Packages

   * What outcome did you expect instead?

File to be fetched correctly

I inserted debug code and used set -x to narrow down the bug and found
that when EXTRA_SUITES is set, in function download_release_indices(),
in the loop "for s in $SUITE $EXTRA_SUITES; do" that the value of 's' is
being changed in the call to validate_suite() where it does "for s in
$SUITE $EXTRA_SUITES; do" and the first test is successful "if [ "$s" =
"$suite" ] || [ "$s" = "$CODENAME" ]; then return 0".

In the example case this resets the suite being processed from
'focal-updates' to 'focal' and so files are downloaded from the focal
suite but the hashes tested against are from the focal-updates
Releases file, causing the validation to fail.

Patch to fix the issue at the end of this email.


-- System Information:
Debian Release: 11.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 5.13.0-soggy-02736-g7f1d227e86a3 (SMP w/4 CPU threads;
PREEMPT)
Kernel taint flags: TAINT_WARN
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE
not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages debootstrap depends on:
ii  wget  1.21-1+b1

Versions of packages debootstrap recommends:
ii  arch-test   0.17-1
ii  debian-archive-keyring  2021.1.1
ii  gnupg   2.2.27-2

Versions of packages debootstrap suggests:
pn  squid-deb-proxy-client  
pn  ubuntu-archive-keyring  

-- no debconf information

--
commit 23bb0a9f3c39b1f23f6c27bd7e8e85f43d7a0316 (HEAD -> master)
Author: Tj 
Date:   Wed Jul 28 21:27:01 2021 +0100

fix: download correct extra-suites Packages files

diff --git a/functions b/functions
index 09d93f4..ec34d84 100644
--- a/functions
+++ b/functions
@@ -551,7 +551,7 @@ extract_release_components () {
  CODENAME=""
 validate_suite () {
-   local reldest suite
+   local reldest suite s
reldest="$1"
 CODENAME=$(sed -n "s/^Codename: *//p" "$reldest")

partman-auto early_command preseeding requiring backslash? (was: Bug#988140: Probably broken code example in installation-guide Appendix B.5 (Running custom commands))

[Samuel Thibault]

Hello,

Boyuan Yang, le jeu. 06 mai 2021 11:10:41 -0400, a ecrit:
> I received a user report that
> https://www.debian.org/releases/bullseye/amd64/apbs05.en.html#preseed-hooks
> provides an incorrect code example:
> 
> # This command is run immediately before the partitioner starts. It may be
> # useful to apply dynamic partitioner preseeding that depends on the state
> # of the disks (which may not be visible when preseed/early_command runs).
> #d-i partman/early_command \
> #   string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"
> 
> The reporter claims that the $ sign should be escaped, like:
> 
> #d-i partman/early_command \
> #string debconf-set partman-auto/disk "\$(list-devices disk | head -n1)"

I guess the reporter actually tested that the backslash is needed. But
do we really want to have to put a backslash here, and thus it's the
documentation that needs to be fixed, or is it partman-auto which is
bogus and should rather be fixed?

Samuel

Bug#988140: Probably broken code example in installation-guide Appendix B.5 (Running custom commands)

[Boyuan Yang]

Source: installation-guide
Severity: normal
X-Debbugs-CC: sthiba...@debian.org

I received a user report that
https://www.debian.org/releases/bullseye/amd64/apbs05.en.html#preseed-hooks
provides an incorrect code example:

# This command is run immediately before the partitioner starts. It may be
# useful to apply dynamic partitioner preseeding that depends on the state
# of the disks (which may not be visible when preseed/early_command runs).
#d-i partman/early_command \
#   string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"

The reporter claims that the $ sign should be escaped, like:

#d-i partman/early_command \
#string debconf-set partman-auto/disk "\$(list-devices disk | head -n1)"

Since I am not familiar with d-i and preseed at all, please verify whether
this makes sense. If this is a problem, it exists in the installation-guide of
both Debian Buster and Debian Bullseye. If it's not, please feel free to close
this bug report.

-- 
Thanks,
Boyuan Yang


signature.asc
Description: This is a digitally signed message part

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Arnaud Rebillout]

ng_variant fakechroot || [ "$CONTAINER" = "docker" ]; then
    setup_proc_symlink
    fi


2) setup_proc() doesn't try to mount /sys

    debootstrap/functions:
    
    if [ -n "$(ls -A "$TARGET/sys")" ] && \
    grep -qs '[[:space:]]sysfs' "$TARGET/proc/filesystems" || \
    [ "$CONTAINER" = "docker" ]; then
    umount_on_exit /sys
    umount "$TARGET/sys" 2>/dev/null || true
else
    # second-stage in docker, we cannot detect it is
    # inside docker... just ignore warning
    in_target mount -t sysfs sysfs /sys || true
    umount_on_exit /sys
fi

At this point, it seems to me that applying this patch would not change 
things dramatically.


There would be a cosmetics improvements, by removing the warnings for 
those who run debootstrap in an unprivileged container.


Also the function detect_container() for docker is clearly broken, so 
it's always nice to try to fix something broken, but due to the caveat 
that Tianon mentions, it could also lead to false positives (debootstrap 
would wrongly assume that it runs in docker, while in fact it does not).


As for the other bugs reported against debootstrap handling of /proc 
(#921815 #968927 #953637), I can't reproduce any, so I really can't say 
about those.


I wonder if the debootstrap code doesn't need more serious maintenance 
regarding how it tries to handle /proc and /sys, but that's out of scope 
of this bug.


Cheers,

--
Arnaud Rebillout

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Tianon Gravi]

Hey Arnaud, thanks for the CC (and sorry for the delay).

On Mon, 12 Apr 2021 at 20:48, Arnaud Rebillout  wrote:
>  > Originally, ".dockerenv" was for transmitting the environment
>  > variables of the container across the container boundary -- I would
>  > not recommend relying on its existence either (IIRC, that code
>  > you've linked to is the only reason it still exists). There's
>  > likely something incriminatory inside /sys/fs/cgroup, but I haven't
>  > checked recently.
>  > https://github.com/moby/moby/issues/18355#issuecomment-220484748
>  >
>  > This makes it sounds like ".dockerenv" may be deprecated and later
>  > removed.
>
> That's a good point, but it's also a 5 years old comment, and the
> .dockerenv file is still present these days.
>
> I would think that if Docker plans to remove it, they will issue a more
> formal deprecation warning that will give us enough time to fix things
> on our side. Also the fact that systemd checks for this file gives me
> more confidence that it's not just me doing something fancy here: it
> seems that this is the "de facto" solution to detect docker containers.
>
> FWIW, it's also the most common solution on Q&A sites like
> stackoverflow. Other people do that, because there is no better solution
> provided apparently. Unless I missed it.
>
>  > Cgroup v2 is also mounted at /sys/fs/cgroup, so I wonder if the original
>  > check should be rewritten to check for something under this path instead
>  > of mountinfo?  Also, using this /sys/fs/cgroup method, I'm not sure if
>  > it's better debootstrap style to express the OR logical operator in the
>  > regex or a shell "||" (ie: seems to be needed because the tree under
>  > /sys/fs/cgroup is different between v1 and v2).
>
> I just had a quick look in /sys/fs/cgroup from within a container.
> Nothing obvious stands out, there's no file named docker, and nothing in
> the content of those files mentions docker. I'll attach the output below.
>
> I will CC Tianon, as he was the author of the comment mentioned above,
> and he might know better, 5 years after :)
>
> In short, Tianon, if you're reading those lines, our question is: what
> would be the right way to detect that we're running from within a docker
> container, apart from checking for the existence of the file
> `/.dockerenv` ???

Heh, I looked into the Docker code, and the only two places it's
currently used by Docker itself are in the line that creates it (as a
simple empty file in the container's init r/w layer) and some code in
libnetwork that uses it to detect that it's running inside a
container, of all things, which is both hilarious and depressing.  A
container can also happily remove it during runtime without any ill
effects (as far as I can see), but the systemd code does include a
caveat that it's not a fully surefire detection method given that it
will often end up in non-Docker rootfses or filesystem images due to
them being created in/by/from Docker environments. :D

I'd definitely love to remove it completely in Docker itself, although
I believe other maintainers would rather use it for something more
useful (like some small amount of container metadata).

So the TLDR is that it's probably fine for now (definitely for
bullseye), but the final status is still very much TBD.

All that being said, I'm honestly a bit confused by debootstrap having
any kind of "detect Docker" code -- I run debootstrap inside Docker
containers a lot, and frankly expect it to run there exactly like it
does on my host (and I supply it with SYS_ADMIN appropriately as a
result, so it can properly mount things like /proc, /sys, and /dev).

♥,
- Tianon
  4096R / B42F 6819 007F 00F8 8E36  4FD4 036A 9C25 BF35 7DD4

Processed: Re: Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Debian Bug Tracking System]

Processing control commands:

> affects -1 release-notes
Bug #985481 [debootstrap] debootstrap: Detection of docker container is broken 
with cgroup v2
Added indication that 985481 affects release-notes

-- 
985481: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=985481
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Nicholas D Steeves]

Control: affects -1 release-notes

Hi Arnaud!

Adding src:docker.io maintainers and Shengjing Zhu (recent uploader) to
CC list.

Arnaud Rebillout  writes:

> Hello Nicholas! Thanks for your feedback here, see replies below.
>

You're welcome :-)

> On Sun, 11 Apr 2021 11:51:20 -0400 Nicholas D Steeves 
>  wrote:
>
>  > I'm not sure that systemd-detect-virt and your patch are
>  > forward-compatible in light of
[snip]
>  > This makes it sounds like ".dockerenv" may be deprecated and later
>  > removed.
>
> That's a good point, but it's also a 5 years old comment, and the 
> .dockerenv file is still present these days.
>
> I would think that if Docker plans to remove it, they will issue a more 
> formal deprecation warning that will give us enough time to fix things 
> on our side. Also the fact that systemd checks for this file gives me 
> more confidence that it's not just me doing something fancy here: it 
> seems that this is the "de facto" solution to detect docker containers.
>
> FWIW, it's also the most common solution on Q&A sites like 
> stackoverflow. Other people do that, because there is no better solution 
> provided apparently. Unless I missed it.
>

Yes, I agree; It appears to be the defacto solution, and might very well
be the only solution for Bullseye in the sense that "perfect is the
enemy of the good", ie: that it's better to solve this issue in a
non-future compatible way to solve a bonafide issue in Bullseye; Later,
a future alternative to /.dockerenv can be documented in Debian.NEWS
and/or release-notes for Bookworm.

>  > Cgroup v2 is also mounted at /sys/fs/cgroup, so I wonder if the original
>  > check should be rewritten to check for something under this path instead
>  > of mountinfo?  Also, using this /sys/fs/cgroup method, I'm not sure if
>  > it's better debootstrap style to express the OR logical operator in the
>  > regex or a shell "||" (ie: seems to be needed because the tree under
>  > /sys/fs/cgroup is different between v1 and v2).
>
> I just had a quick look in /sys/fs/cgroup from within a container. 
> Nothing obvious stands out, there's no file named docker, and nothing in 
> the content of those files mentions docker. I'll attach the output below.
>
> I will CC Tianon, as he was the author of the comment mentioned above, 
> and he might know better, 5 years after :)
>
> In short, Tianon, if you're reading those lines, our question is: what 
> would be the right way to detect that we're running from within a docker 
> container, apart from checking for the existence of the file 
> `/.dockerenv` ???
>

Thank you for this investigation!  I was also unable to find an
alternative is_running_in_docker cgroupv2 check using /sys/fs/cgroup.
Hopefully one of the src:docker.io maintainers knows!  I've also added
"affects release-notes" (and filed separate release-notes bugs) to
defend against a worst-case scenario where this bug isn't resolved in
time.


Regards,
Nicholas


signature.asc
Description: PGP signature

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Arnaud Rebillout]

Hello Nicholas! Thanks for your feedback here, see replies below.


On Sun, 11 Apr 2021 11:51:20 -0400 Nicholas D Steeves 
 wrote:


> I'm not sure that systemd-detect-virt and your patch are
> forward-compatible in light of
>
> Originally, ".dockerenv" was for transmitting the environment
> variables of the container across the container boundary -- I would
> not recommend relying on its existence either (IIRC, that code
> you've linked to is the only reason it still exists). There's
> likely something incriminatory inside /sys/fs/cgroup, but I haven't
> checked recently.
> https://github.com/moby/moby/issues/18355#issuecomment-220484748
>
> This makes it sounds like ".dockerenv" may be deprecated and later
> removed.

That's a good point, but it's also a 5 years old comment, and the 
.dockerenv file is still present these days.


I would think that if Docker plans to remove it, they will issue a more 
formal deprecation warning that will give us enough time to fix things 
on our side. Also the fact that systemd checks for this file gives me 
more confidence that it's not just me doing something fancy here: it 
seems that this is the "de facto" solution to detect docker containers.


FWIW, it's also the most common solution on Q&A sites like 
stackoverflow. Other people do that, because there is no better solution 
provided apparently. Unless I missed it.



> Cgroup v2 is also mounted at /sys/fs/cgroup, so I wonder if the original
> check should be rewritten to check for something under this path instead
> of mountinfo?  Also, using this /sys/fs/cgroup method, I'm not sure if
> it's better debootstrap style to express the OR logical operator in the
> regex or a shell "||" (ie: seems to be needed because the tree under
> /sys/fs/cgroup is different between v1 and v2).

I just had a quick look in /sys/fs/cgroup from within a container. 
Nothing obvious stands out, there's no file named docker, and nothing in 
the content of those files mentions docker. I'll attach the output below.


I will CC Tianon, as he was the author of the comment mentioned above, 
and he might know better, 5 years after :)


In short, Tianon, if you're reading those lines, our question is: what 
would be the right way to detect that we're running from within a docker 
container, apart from checking for the existence of the file 
`/.dockerenv` ???


Thanks !



 Logs -- checking /sys/fs/cgroup from within a docker container


# head -n 100 /sys/fs/cgroup/*
==> /sys/fs/cgroup/cgroup.controllers <==
cpuset cpu io memory hugetlb pids rdma

==> /sys/fs/cgroup/cgroup.events <==
populated 1
frozen 0

==> /sys/fs/cgroup/cgroup.freeze <==
0

==> /sys/fs/cgroup/cgroup.max.depth <==
max

==> /sys/fs/cgroup/cgroup.max.descendants <==
max

==> /sys/fs/cgroup/cgroup.procs <==
1
16

==> /sys/fs/cgroup/cgroup.stat <==
nr_descendants 0
nr_dying_descendants 0

==> /sys/fs/cgroup/cgroup.subtree_control <==

==> /sys/fs/cgroup/cgroup.threads <==
1
16

==> /sys/fs/cgroup/cgroup.type <==
domain

==> /sys/fs/cgroup/cpu.max <==
max 10

==> /sys/fs/cgroup/cpu.pressure <==
some avg10=0.00 avg60=0.00 avg300=0.00 total=6857

==> /sys/fs/cgroup/cpu.stat <==
usage_usec 145464
user_usec 57509
system_usec 87955
nr_periods 0
nr_throttled 0
throttled_usec 0

==> /sys/fs/cgroup/cpu.weight <==
100

==> /sys/fs/cgroup/cpu.weight.nice <==
0

==> /sys/fs/cgroup/cpuset.cpus <==


==> /sys/fs/cgroup/cpuset.cpus.effective <==
0-7

==> /sys/fs/cgroup/cpuset.cpus.partition <==
member

==> /sys/fs/cgroup/cpuset.mems <==


==> /sys/fs/cgroup/cpuset.mems.effective <==
0

==> /sys/fs/cgroup/hugetlb.1GB.current <==
0

==> /sys/fs/cgroup/hugetlb.1GB.events <==
max 0

==> /sys/fs/cgroup/hugetlb.1GB.events.local <==
max 0

==> /sys/fs/cgroup/hugetlb.1GB.max <==
max

==> /sys/fs/cgroup/hugetlb.1GB.rsvd.current <==
0

==> /sys/fs/cgroup/hugetlb.1GB.rsvd.max <==
max

==> /sys/fs/cgroup/hugetlb.2MB.current <==
0

==> /sys/fs/cgroup/hugetlb.2MB.events <==
max 0

==> /sys/fs/cgroup/hugetlb.2MB.events.local <==
max 0

==> /sys/fs/cgroup/hugetlb.2MB.max <==
max

==> /sys/fs/cgroup/hugetlb.2MB.rsvd.current <==
0

==> /sys/fs/cgroup/hugetlb.2MB.rsvd.max <==
max

==> /sys/fs/cgroup/io.max <==

==> /sys/fs/cgroup/io.pressure <==
some avg10=0.00 avg60=0.00 avg300=0.00 total=24710
full avg10=0.00 avg60=0.00 avg300=0.00 total=24710

==> /sys/fs/cgroup/io.stat <==
259:0 rbytes=5431296 wbytes=0 rios=196 wios=0 dbytes=0 dios=0
254:0 rbytes=5431296 wbytes=0 rios=196 wios=0 dbytes=0 dios=0
254:1 rbytes=5431296 wbytes=0 rios=196 wios=0 dbytes=0 dios=0

==> /sys/fs/cgroup/io.weight <==
default 100

==> /sys/fs/cgroup/memory.current <==
6696960

==> /sys/fs/cgroup/memory.events <==
low 0
high 0
max 0
oom 0
oom_kill 0

==> /sys/fs/cgroup/memory.events.local <==
low 0
high 0
max 0
oom 0
oom_kill 0

==> /sys/fs/cgroup/memory.high <==
max

==> /sys/fs/cgroup/memory.low <==
0

==> /sys/fs/cgroup/memory.max <==
max

==> /sys/fs/cgroup/memory.min <==
0

==> /sys/fs/cgroup/memory.numa_stat <==
anon N0=602

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Nicholas D Steeves]

Attention LXC Team: Does a functioning /sys always exist in an LXC
container, or is it absent/disabled in some configurations?

Hi Arnaud,

Reply follows inline.

Arnaud Rebillout  writes:

> Package: debootstrap
> Version: 1.0.123
> Severity: normal
> Tags: patch
> User: de...@kali.org
> Usertags: origin-kali
>
> Dear Maintainer,
>
> The code that is meant to detect if debootstrap is running from within a
> docker container is broken with cgroup v2. Talking about this particular
> function and line in the file `functions`:
>

I agree that Bullseye should have working LXC with cgroups v2, since (as
far as I know) we have new enough packages from upstream to support
it :-)  Thank you for your interest and motivation to fix this!

> detect_container () {
> [...]
> elif grep -qs '[[:space:]]/docker/.*/sys/fs/cgroup' 
> /proc/1/mountinfo; then
> CONTAINER="docker"
>
> This code only works for cgroup v1.
>
> After some research, and also after looking into the code of
> systemd-detect-virt, it seems that the right way to detect a docker
> container these days is to check for the file '/.dockerenv'.
>
> Hence I'm proposing this patch:
> https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/52
>

I'm not sure that systemd-detect-virt and your patch are
forward-compatible in light of

Originally, ".dockerenv" was for transmitting the environment
variables of the container across the container boundary -- I would
not recommend relying on its existence either (IIRC, that code
you've linked to is the only reason it still exists).  There's
likely something incriminatory inside /sys/fs/cgroup, but I haven't
checked recently.
https://github.com/moby/moby/issues/18355#issuecomment-220484748

This makes it sounds like ".dockerenv" may be deprecated and later
removed.  It seems reasonable to contact Debian's systemd maintainer[s]
about this probable future issue, because if checking for ".dockerenv"
is robust enough for Bullseye's systemd, then it might be robust enough
for debootstrap.  That said, I still wonder if this method could pose a
problem when using debootstrap, lxc, and docker from future
bullseye-backports, if ".dockerenv" support is removed sometime during
Bullseye's life-cycle.

Cgroup v2 is also mounted at /sys/fs/cgroup, so I wonder if the original
check should be rewritten to check for something under this path instead
of mountinfo?  Also, using this /sys/fs/cgroup method, I'm not sure if
it's better debootstrap style to express the OR logical operator in the
regex or a shell "||" (ie: seems to be needed because the tree under
/sys/fs/cgroup is different between v1 and v2).


Thanks again!
Regards,
Nicholas


signature.asc
Description: PGP signature

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Arnaud R]

I just noticed that there is snippet in scripts/debian-common:

    if doing_variant fakechroot || [ "$CONTAINER" = "docker" ]; then
    setup_proc_symlink
    fi

The following bugs are related to docker and /proc, and are possibly 
related:


https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921815

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968927

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953637


On Fri, 19 Mar 2021 10:02:44 +0700 Arnaud Rebillout  
wrote:


> Package: debootstrap
> Version: 1.0.123
> Severity: normal
> Tags: patch
> User: de...@kali.org
> Usertags: origin-kali
>
> Dear Maintainer,
>
> The code that is meant to detect if debootstrap is running from within a
> docker container is broken with cgroup v2. Talking about this particular
> function and line in the file `functions`:
>
> detect_container () {
> [...]
> elif grep -qs '[[:space:]]/docker/.*/sys/fs/cgroup' 
/proc/1/mountinfo; then

> CONTAINER="docker"
>
> This code only works for cgroup v1.
>
> After some research, and also after looking into the code of
> systemd-detect-virt, it seems that the right way to detect a docker
> container these days is to check for the file '/.dockerenv'.
>
> Hence I'm proposing this patch:
> https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/52
>
> Thanks!
>
>
> -- More debug logs:
>
> Here's what I get on current Debian sid:
>
> $ cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-5.10.0-4-amd64 root=<> tro quiet
>
> $ mount | grep cgroup
> cgroup2 on /sys/fs/cgroup type cgroup2 
(rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)

>
> $ sudo docker run --rm -it debian:testing grep 
'[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo

>  no ouput, the detection code is broken!
>
> $ sudo docker run --rm -it debian:testing ls -l /.dockerenv
> -rwxr-xr-x 1 root root 0 Mar 19 02:37 /.dockerenv
>
> Just out of curiosity, I tried to get the current detection code to
> work, by booting my system with cgroup v1 only. This is done by setting
> the two boot parameters `systemd.unified_cgroup_hierarchy=0` and
> `systemd.legacy_systemd_cgroup_controller=1`.
>
> Here are the logs:
>
> $ cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-5.10.0-4-amd64 root=<> ro quiet 
systemd.unified_cgroup_hierarchy=0 
systemd.legacy_systemd_cgroup_controller=1

>
> $ mount | grep cgroup
> tmpfs on /sys/fs/cgroup type tmpfs 
(ro,nosuid,nodev,noexec,size=4096k,nr_inodes=1024,mode=755)
> cgroup on /sys/fs/cgroup/systemd type cgroup 
(rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
> cgroup on /sys/fs/cgroup/memory type cgroup 
(rw,nosuid,nodev,noexec,relatime,memory)

Bug#985481: debootstrap: Detection of docker container is broken with cgroup v2

[Arnaud Rebillout]

Package: debootstrap
Version: 1.0.123
Severity: normal
Tags: patch
User: de...@kali.org
Usertags: origin-kali

Dear Maintainer,

The code that is meant to detect if debootstrap is running from within a
docker container is broken with cgroup v2. Talking about this particular
function and line in the file `functions`:

detect_container () {
[...]
elif grep -qs '[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo; 
then
CONTAINER="docker"

This code only works for cgroup v1.

After some research, and also after looking into the code of
systemd-detect-virt, it seems that the right way to detect a docker
container these days is to check for the file '/.dockerenv'.

Hence I'm proposing this patch:
https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/52

Thanks!


-- More debug logs:

Here's what I get on current Debian sid:

$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.10.0-4-amd64 root=<> tro quiet

$ mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 
(rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)

$ sudo docker run --rm -it debian:testing grep 
'[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo
 no ouput, the detection code is broken!

$ sudo docker run --rm -it debian:testing ls -l /.dockerenv
-rwxr-xr-x 1 root root 0 Mar 19 02:37 /.dockerenv

Just out of curiosity, I tried to get the current detection code to
work, by booting my system with cgroup v1 only. This is done by setting
the two boot parameters `systemd.unified_cgroup_hierarchy=0` and
`systemd.legacy_systemd_cgroup_controller=1`.

Here are the logs:

$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.10.0-4-amd64 root=<> ro quiet 
systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=1

$ mount | grep cgroup
tmpfs on /sys/fs/cgroup type tmpfs 
(ro,nosuid,nodev,noexec,size=4096k,nr_inodes=1024,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup 
(rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/memory type cgroup 
(rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/rdma type cgroup 
(rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup 
(rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup 
(rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup 
(rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup 
(rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/pids type cgroup 
(rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/cpuset type cgroup 
(rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/freezer type cgroup 
(rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/perf_event type cgroup 
(rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup 
(rw,nosuid,nodev,noexec,relatime,hugetlb)

$ sudo docker run --rm -it debian:testing grep 
'[[:space:]]/docker/.*/sys/fs/cgroup' /proc/1/mountinfo
795 794 0:29 /docker/<> /sys/fs/cgroup/systemd 
ro,nosuid,nodev,noexec,relatime master:10 - cgroup cgroup 
rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
797 794 0:33 /docker/<> /sys/fs/cgroup/memory 
ro,nosuid,nodev,noexec,relatime master:15 - cgroup cgroup rw,memory
818 794 0:35 /docker/<> /sys/fs/cgroup/net_cls,net_prio 
ro,nosuid,nodev,noexec,relatime master:17 - cgroup cgroup rw,net_cls,net_prio
819 794 0:36 /docker/<> /sys/fs/cgroup/cpu,cpuacct 
ro,nosuid,nodev,noexec,relatime master:18 - cgroup cgroup rw,cpu,cpuacct
853 794 0:37 /docker/<> /sys/fs/cgroup/blkio 
ro,nosuid,nodev,noexec,relatime master:19 - cgroup cgroup rw,blkio
854 794 0:38 /docker/<> /sys/fs/cgroup/devices 
ro,nosuid,nodev,noexec,relatime master:20 - cgroup cgroup rw,devices
872 794 0:39 /docker/<> /sys/fs/cgroup/pids 
ro,nosuid,nodev,noexec,relatime master:21 - cgroup cgroup rw,pids
873 794 0:40 /docker/<> /sys/fs/cgroup/cpuset 
ro,nosuid,nodev,noexec,relatime master:22 - cgroup cgroup rw,cpuset
891 794 0:41 /docker/<> /sys/fs/cgroup/freezer 
ro,nosuid,nodev,noexec,relatime master:23 - cgroup cgroup rw,freezer
892 794 0:42 /docker/<> /sys/fs/cgroup/perf_event 
ro,nosuid,nodev,noexec,relatime master:24 - cgroup cgroup rw,perf_event
910 794 0:43 /docker/<> /sys/fs/cgroup/hugetlb 
ro,nosuid,nodev,noexec,relatime master:25 - cgroup cgroup rw,hugetlb

Conclusion: the debootstrap code that detects a docker container used to
work for cgroup v1, but it's broken for cgroup v2.


-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (5

Processed: retitle 970678 to Network preseeding using http is broken

[Debian Bug Tracking System]

Processing commands for cont...@bugs.debian.org:

> retitle 970678 Network preseeding using http is broken
Bug #970678 [debian-installer] udev-udeb: setup /dev/fd, /dev/std{in,out,err} 
symlinks
Changed Bug title to 'Network preseeding using http is broken' from 'udev-udeb: 
setup /dev/fd, /dev/std{in,out,err} symlinks'.
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
970678: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970678
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#970678: Address change due to junk email (Was: Network preseeding using http is broken)

[Martin Samuelsson]

Spammers have harvested the email alias used to report this issue, and 
actively abuse it. As a consequence I'm suspending it. I'm subscribed to the 
bug report using another address.


Follow-up sent to the bug will reach me. For instructions on how to email me 
directly, please follow instructions at https://bugs.debian.org/199392#40 or 
find a disposable contact address on my web page.


Sorry about the noise.
--
/Martin

Re: Bug#970678: Network preseeding using http is broken

[Bjørn Mork]

Ben Hutchings  writes:
> On Mon, 2020-09-21 at 20:42 +0100, Ben Hutchings wrote:
>> On Mon, 2020-09-21 at 17:43 +0200, Martin Samuelsson wrote:
>> > Philip Hands @ 2020-09-21 (Monday), 15:30 (+0200)
>> > > Martin Samuelsson  writes:
>> > > 
>> > > Just to be clear on this point, are you saying [...]
>> > 
>> > I'm saying there is no /dev/fd/ at all on current daily debian-installer 
>> > images and hasn't been since at least 20200818 (which was the oldest one I 
>> > could try with current kernel modules).
>> [...]
>> 
>> Then we should investigate and fix that instead of removing code that
>> uses it.
>
> It's a udev regression, bug #967546.

Looks like that's deliberate, so probably Not A Bug.  Quoting from
https://github.com/systemd/systemd/commit/6b2229c6c60d0486 :

 "if people want to use udev from other init systems
  they should do this on their own,"

You might want to just fork udev while it still sort of works outside
systemd.



Bjørn

Bug#970678: Network preseeding using http is broken

[Ben Hutchings]

On Mon, 2020-09-21 at 20:42 +0100, Ben Hutchings wrote:
> On Mon, 2020-09-21 at 17:43 +0200, Martin Samuelsson wrote:
> > Philip Hands @ 2020-09-21 (Monday), 15:30 (+0200)
> > > Martin Samuelsson  writes:
> > > 
> > > Just to be clear on this point, are you saying [...]
> > 
> > I'm saying there is no /dev/fd/ at all on current daily debian-installer 
> > images and hasn't been since at least 20200818 (which was the oldest one I 
> > could try with current kernel modules).
> [...]
> 
> Then we should investigate and fix that instead of removing code that
> uses it.

It's a udev regression, bug #967546.

Ben.

-- 
Ben Hutchings
Anthony's Law of Force: Don't force it, get a larger hammer.



signature.asc
Description: This is a digitally signed message part

Bug#970678: Network preseeding using http is broken

[Martin Samuelsson]

Philip Hands @ 2020-09-21 (Monday), 22:38 (+0200)


BTW The example I pasted was just busybox running on my laptop running
full Debian, so was not supposed to be demonstrating it working under
d-i.


I could likely have been more precise from the beginning about the exact 
cause. Sorry for making you required to ask for clarification.



This mixes the stdout of wget with the %OK% %EOF% stuff, and then puts
it all into the sed, which seems flawed.


And that is why you're a Debian developer and I'm merely a user. I agree.


I'd have thought something like this would do the trick (not tested yet):

local RETVAL=$(
{
  {
wget "$@" 2>&1 >/dev/null && echo 0 >&3
  } | {
grep -q "$file_not_found_pattern" && echo 4 >&3
  } || echo 1 >&3
} 3>&1 | head -1
)


Looks good to me. As in, this code I can actually follow and understand.  
Which was a bit of a stretch for me to claim about for the sed construct.


I don't really understand why the tail would be needed, but am not objecting 
to it.


Replacing the RETVAL assignment expression as above works for me. Both from 
debian-installer and when running this simple test script in a shell:


 #!/bin/sh
 fetch-url http://pxeserver./preseed/base.txt base.txt >/dev/null
 [ $? = 0 ] || exit
 echo "Existing file downloaded"

 fetch-url http://pxeserver./non-existant.txt non-existant.txt >/dev/null
 [ $? = 4 ] || exit
 echo "Non-existing file returned 404"

 fetch-url http://invalid-host./whatever.txt invalid-host.txt >/dev/null
 [ $? = 1 ] || exit
 echo "Unresolvable host return general error"


One could of course use stderr for that (>&2 ... 2>&1) for getting the
echo-ed return codes out, rather than fd #3, but I think this is clearer
(since one really isn't trying to produce stderr output) and AFAIK it
should work fine even if /dev/fd/ is missing.


I agree. Nicely done!


I suspect there may be a way of getting this to work without the need
for a local variable and the echoing for the result codes, so I will
ponder on that...


Maybe it is, and if so it might include early return statements and thus 
save spawning grep on successful fetches. I reckon you could be happy and 
proud with the code you've posted already though.



BTW I just saw Ben's comment that we should just fix the missing /dev/fd
which strikes me as entirely sensible.


It's entirely sensible to also attempt fixing the root cause, but my opinion 
is that it wouldn't hurt to treat missing fd-nodes and an overcomplex 
wget404() as two separate issues. Please feel free to fork the bug report.


One could also envision adding a test case for detecting missing fd:s within 
some test suite for d-i. Are there any tests run on nightly builds, which 
could help catching regressions like these?

--
/Martin

Bug#970678: Network preseeding using http is broken

[Philip Hands]

Martin Samuelsson  writes:

> I'm saying there is no /dev/fd/ at all

Oh, fair enough.  That's odd.

BTW The example I pasted was just busybox running on my laptop running
full Debian, so was not supposed to be demonstrating it working under
d-i.

...
> --- http.orig 2020-09-21 17:21:24.159480072 +0200
> +++ http.simplified   2020-09-21 17:21:03.951356698 +0200
> @@ -14,10 +14,10 @@
>  
>   local RETVAL=$( {
>   echo 1
> - wget "$@" 2>&1 >&3 && echo %OK%
> + wget "$@" 2>&1 && echo %OK%
>   echo %EOF%
> - } | ( sed -ne 
> '1{h;d};/'"$file_not_found_pattern"'/{p;s/.*/4/;h;d};/^%OK%$/{s/.*/0/;h;d};$!p;$x;$w
>  /dev/fd/4' >&2 ) 4>&1
> - ) 3>&1
> + } | ( sed -ne 
> '1{h;d};/'"$file_not_found_pattern"'/{p;s/.*/4/;h;d};/^%OK%$/{s/.*/0/;h;d};$!p;$x;$p
>  '>&2 )
> + ) 
>   return $RETVAL
>   }

This mixes the stdout of wget with the %OK% %EOF% stuff, and then puts
it all into the sed, which seems flawed.

One could replace the >&3 with >/dev/null, and keep the sed, but if one
isn't trying to preserve the wget output I don't see the point of
keeping the sed at all.

I'd have thought something like this would do the trick (not tested yet):

local RETVAL=$(
 {
   {
 wget "$@" 2>&1 >/dev/null && echo 0 >&3
   } | {
 grep -q "$file_not_found_pattern" && echo 4 >&3
   } || echo 1 >&3
 } 3>&1 | head -1
)

(the patterns will need to be tweaked to take account of sed vs. grep)

One could of course use stderr for that (>&2 ... 2>&1) for getting the
echo-ed return codes out, rather than fd #3, but I think this is clearer
(since one really isn't trying to produce stderr output) and AFAIK it
should work fine even if /dev/fd/ is missing.

I suspect there may be a way of getting this to work without the need
for a local variable and the echoing for the result codes, so I will
ponder on that...

BTW I just saw Ben's comment that we should just fix the missing /dev/fd
which strikes me as entirely sensible. Even if nothing uses /dev/fd/ in
d-i (other than here) it seems wrong to simply ignore the fact that it's
missing, since people might well try to use it in preseed scripts etc.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#970678: Network preseeding using http is broken

[Ben Hutchings]

On Mon, 2020-09-21 at 17:43 +0200, Martin Samuelsson wrote:
> Philip Hands @ 2020-09-21 (Monday), 15:30 (+0200)
> > Martin Samuelsson  writes:
> > 
> > Just to be clear on this point, are you saying [...]
> 
> I'm saying there is no /dev/fd/ at all on current daily debian-installer 
> images and hasn't been since at least 20200818 (which was the oldest one I 
> could try with current kernel modules).
[...]

Then we should investigate and fix that instead of removing code that
uses it.

Ben.

-- 
Ben Hutchings
The Peter principle: In a hierarchy, every employee tends to rise to
their level of incompetence.




signature.asc
Description: This is a digitally signed message part

Bug#970678: Network preseeding using http is broken

[Martin Samuelsson]

Geert Stappers @ 2020-09-21 (Monday), 17:18 (+0200)


Under which circumstance does the bug shows itself?


As far as I understand /dev/fd seems to be completely missing. Haven't dug 
into it. For what its worth, it seems /proc/self/fd is still available. I 
did experiment with redirecting sed there instead before realizing it wasn't 
really necessary to retain wget's output.

--
/Martin

Bug#970678: Network preseeding using http is broken

[Martin Samuelsson]

Philip Hands @ 2020-09-21 (Monday), 15:30 (+0200)

Martin Samuelsson  writes:

Just to be clear on this point, are you saying [...]


I'm saying there is no /dev/fd/ at all on current daily debian-installer 
images and hasn't been since at least 20200818 (which was the oldest one I 
could try with current kernel modules).


Missing fd:s is the root cause, and a different issue which I hope someone 
else is addressing in case it is a bug. (Nothing else in d-i failed for me 
due to it.)



To illustrate this, here's some output from busybox shell, running on my
laptop:


Yet those examples are not from the context of a recent debian-installer 
nightly image, right? 


If you noticed that it's not there when e.g. simply listing the contents
of /dev/fd then that's normal AFAIK.


The issue I reported is that wget404() is unnecessarily complex and fragile, 
for no reasonable reason. (You've stated so yourself in the README.wget404)


A suggested fix for /usr/lib/fetch-url/http looks as below:

--- http.orig   2020-09-21 17:21:24.159480072 +0200
+++ http.simplified 2020-09-21 17:21:03.951356698 +0200
@@ -14,10 +14,10 @@

local RETVAL=$( {
echo 1
-   wget "$@" 2>&1 >&3 && echo %OK%
+   wget "$@" 2>&1 && echo %OK%
echo %EOF%
-   } | ( sed -ne 
'1{h;d};/'"$file_not_found_pattern"'/{p;s/.*/4/;h;d};/^%OK%$/{s/.*/0/;h;d};$!p;$x;$w 
/dev/fd/4' >&2 ) 4>&1
-   ) 3>&1
+   } | ( sed -ne 
'1{h;d};/'"$file_not_found_pattern"'/{p;s/.*/4/;h;d};/^%OK%$/{s/.*/0/;h;d};$!p;$x;$p 
'>&2 )
+		) 
		return $RETVAL

}

I just tried patching as above at the first of the interactive debug prompts 
when setting the DEBUG bootflag high. Installation succeeded, and fetch-url 
still returns 0 on success, 1 on general failure and 4 on http not found.  
The only difference is that any output from wget is discarded already inside 
wget404() rather than by the call to wget404().


It seems me mentioning the issue on irc made you fix typos in the README.  
The preservation of wget's stdout and stderr looks fenomenal, but it fills 
no practical purpose. Please consider killing your darling. Simple code is 
beautiful and robust code.

--
/Martin

Bug#970678: Network preseeding using http is broken

[Geert Stappers]

On Mon, Sep 21, 2020 at 03:30:27PM +0200, Philip Hands wrote:
> Martin Samuelsson  writes:
> 
> > Booting the installer with DEBCONF_DEBUG=5 and debuging /bin/preseed_fetch, 
> > /bin/fetch-url and /usr/lib/fetch_url/http shows that wget404() in the 
> > latter is what's failing. It seems the pipeline fails since /dev/fd/4 does 
> > not exist.
> 
> Just to be clear on this point, are you saying that /dev/fd/4 does not
> exist when you look for it in a shell, or rather specifically when doing
> so in a context where it is in use?
> 
> If you noticed that it's not there when e.g. simply listing the contents
> of /dev/fd then that's normal AFAIK.
> 
> To illustrate this, here's some output from busybox shell, running on my
> laptop:
> 
> =-=-=-=-
> ~ $ echo /dev/fd/*
> /dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3
> ~ $ ( echo /dev/fd/* ) 4>&1
> /dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3 /dev/fd/4
> ~ $ ( echo /dev/fd/* ) 4>&1 5>&1
> /dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3 /dev/fd/4 /dev/fd/5
> ~ $  ( echo /dev/fd/* ) 6>&1
> /dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3 /dev/fd/6
> ~ $
> =-=-=-=-
> 


On Mon, Sep 21, 2020 at 02:58:36PM +0200, Philip Hands wrote:
 ...
> 
> The fact that /dev/fd/4 is missing does seem to be a separate bug, but a
> quick grep -lr suggests that this is the only place it's used in d-i, so
> perhaps a bug we need not worry about too much.
> 


Under which circumstance does the bug shows itself?



Regards
Geert Stappers
-- 
Silence is hard to parse

Bug#970678: Network preseeding using http is broken

[Philip Hands]

Martin Samuelsson  writes:

> Booting the installer with DEBCONF_DEBUG=5 and debuging /bin/preseed_fetch, 
> /bin/fetch-url and /usr/lib/fetch_url/http shows that wget404() in the 
> latter is what's failing. It seems the pipeline fails since /dev/fd/4 does 
> not exist.

Just to be clear on this point, are you saying that /dev/fd/4 does not
exist when you look for it in a shell, or rather specifically when doing
so in a context where it is in use?

If you noticed that it's not there when e.g. simply listing the contents
of /dev/fd then that's normal AFAIK.

To illustrate this, here's some output from busybox shell, running on my
laptop:

=-=-=-=-
~ $ echo /dev/fd/*
/dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3
~ $ ( echo /dev/fd/* ) 4>&1
/dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3 /dev/fd/4
~ $ ( echo /dev/fd/* ) 4>&1 5>&1
/dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3 /dev/fd/4 /dev/fd/5
~ $  ( echo /dev/fd/* ) 6>&1
/dev/fd/0 /dev/fd/1 /dev/fd/10 /dev/fd/2 /dev/fd/3 /dev/fd/6
~ $
=-=-=-=-

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#970678: Network preseeding using http is broken

[Philip Hands]

Martin Samuelsson  writes:

...
> Reading README.wget404[1] clearly states this output redirection dance is 
> never actually used, and that this convoluted expression merely exists 
> because it could possibly-maybe be useful some day. As far as I can see the 
> callers of wget404() does indeed not use its output.
>
> Given that this construct does cause real world problems, without adding any 
> actual value, how would you consider having wget404() simplified to simply 
> consume the error message and thus skip all usage of &3 and &4?

Seems fair enough to me -- if the wget output is actually needed at some
point in the future, one can always get hold of the sed incantation from
git.

That being the case, a warning could be added to the modified wget404 to
say that it now discards output, and that if that's a problem one should
look at  for a version that manages to preserve it, as
long as /dev/fd/4 is available.

The fact that /dev/fd/4 is missing does seem to be a separate bug, but a
quick grep -lr suggests that this is the only place it's used in d-i, so
perhaps a bug we need not worry about too much.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#970678: Network preseeding using http is broken

[Martin Samuelsson]

Package: debian-installer
Version: 20200920

Debian installer fails to fetch preseed files over http.

How to reproduce:

Boot the installer with url=http://pxeserver./example.txt

Where example.txt contains:
d-i preseed/include string something.txt \
   other.txt \
   more.txt

The installer will fail to fetch at least one of the required files, and 
catching pcaps will show no fetch attempts are made.


Booting the installer with DEBCONF_DEBUG=5 and debuging /bin/preseed_fetch, 
/bin/fetch-url and /usr/lib/fetch_url/http shows that wget404() in the 
latter is what's failing. It seems the pipeline fails since /dev/fd/4 does 
not exist.


Changing /dev/fd/4 to /dev/null is a workaround making the problem go away.

Reading README.wget404[1] clearly states this output redirection dance is 
never actually used, and that this convoluted expression merely exists 
because it could possibly-maybe be useful some day. As far as I can see the 
callers of wget404() does indeed not use its output.


Given that this construct does cause real world problems, without adding any 
actual value, how would you consider having wget404() simplified to simply 
consume the error message and thus skip all usage of &3 and &4?


[1]: 
https://sources.debian.org/src/debian-installer-utils/1.133/README.wget404/#L119

Bug#969965: Switching keyboard layouts broken in text installer (Was: Installation of Debian 11 mostly successfully)

[Holger Wansing]

Control: retitle -1 text installer: unable to switch keyboard layouts - 
kbd_mode broken
Control: severity -1 critical
Control: reassign -1 kbd


Bernhard  wrote:
> One small error found:
> In the command line of the boot image, the option keymap=de don't work.
> In installer, there is US keyboard layout.

This affects only the text installer, in the graphical installer everything
works fine. And it's not only occuring when you select the keymap via 
commandline
but also when choosing it normally via installer dialog.
Switching to keyboard layouts other than US is no longer possible.

I boiled that down to the latest version of kbd, which breaks console-setup run 
with mutiple error messages like this:


/bin/setupcon:
line 105:
kbd_mode: not found

The images from https://d-i.debian.org/daily-images/amd64/20200820-00:20/
work fine, and one day later it fails.
The new version of kbd was uploaded on 20200821.

Reassigning to kbd and renaming.
And raising severity, since it completely breaks keyboard layout switching
in the text installer.


Holger



-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Processed: Re: Bug#969965: Switching keyboard layouts broken in text installer (Was: Installation of Debian 11 mostly successfully)

[Debian Bug Tracking System]

Processing control commands:

> retitle -1 text installer: unable to switch keyboard layouts - kbd_mode broken
Bug #969965 [installation-reports] Installation of Debian 11 mostly successfully
Changed Bug title to 'text installer: unable to switch keyboard layouts - 
kbd_mode broken' from 'Installation of Debian 11 mostly successfully'.
> severity -1 critical
Bug #969965 [installation-reports] text installer: unable to switch keyboard 
layouts - kbd_mode broken
Severity set to 'critical' from 'normal'
> reassign -1 kbd
Bug #969965 [installation-reports] text installer: unable to switch keyboard 
layouts - kbd_mode broken
Bug reassigned from package 'installation-reports' to 'kbd'.
Ignoring request to alter found versions of bug #969965 to the same values 
previously set
Ignoring request to alter fixed versions of bug #969965 to the same values 
previously set

-- 
969965: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=969965
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Re: Missing daily builds on cdimage.debian.org (Re: Firmware testing images broken)

[Holger Wansing]

Hi,

Steve McIntyre  wrote:
> I think you've misinterpreted this. We *don't* try to keep the daily
> builds around for more than a few days. They're a moving target. The
> only reason that the 2019 images are still around at all is that
> they're the last few builds where we had a build for mips. (There's
> logic in the cleanup scripts to keep a few builds per architecture.)
> I'll clean up those old directories now.

Ah, yes. I mixed that up with 
https://d-i.debian.org/daily-images/amd64/ on dillon.
There we hold images for 4 weeks.

thanks for clarifying

Holger


-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Re: Firmware testing images broken

[Steve McIntyre]

On Thu, Sep 17, 2020 at 07:25:36PM +0100, Ben Hutchings wrote:
>On Tue, 2020-09-15 at 10:01 +0100, Steve McIntyre wrote:
>> 
>> And fixed as of today's daily builds.
>> 
>> Thanks for the report!
>
>Thanks Steve.  We probably should have thought to update that after
>changing the dependencies (which was needed because the module
>dependencies changed).

ACK - some warning might have been nice. Meh, no biggie...

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Since phone messaging became popular, the young generation has lost the
 ability to read or write anything that is longer than one hundred and sixty
 characters."  -- Ignatios Souvatzis

Re: Firmware testing images broken

[Ben Hutchings]

On Tue, 2020-09-15 at 10:01 +0100, Steve McIntyre wrote:
> On Mon, Sep 14, 2020 at 09:51:57AM +0100, Steve McIntyre wrote:
> > Hi Brandon,
> > 
> > On Sun, Sep 13, 2020 at 03:17:11AM -0400, Brandon Werner wrote:
> > > Hi,
> > > 
> > > I tried to use one of the Sid DI netinst firmware images and they
> > > seem to be broken for about a week or so now. After picking the
> > > language and keyboard, the iso is unable to mount. I am using an EFI
> > > machine with a usb flash drive and don't have a disk drive to test if
> > > the error occurs in that scenario. I tested on three different EFI
> > > machines that I have with very different processors and got the same
> > > results. One thing that might help is that I saw that linux 5.8
> > > entered Sid at the time things stopped working and wonder if a
> > > possible regression could be due to that
> > 
> > ACK, looks like a regression in config somewhere - the isofs
> > filesystem is missing compared to the last build with 5.7.0-3.
> 
> And fixed as of today's daily builds.
> 
> Thanks for the report!

Thanks Steve.  We probably should have thought to update that after
changing the dependencies (which was needed because the module
dependencies changed).

Ben.

-- 
Ben Hutchings
One of the nice things about standards is that
there are so many of them.




signature.asc
Description: This is a digitally signed message part

Re: Firmware testing images broken

[Steve McIntyre]

On Mon, Sep 14, 2020 at 09:51:57AM +0100, Steve McIntyre wrote:
>Hi Brandon,
>
>On Sun, Sep 13, 2020 at 03:17:11AM -0400, Brandon Werner wrote:
>>Hi,
>>
>>I tried to use one of the Sid DI netinst firmware images and they
>>seem to be broken for about a week or so now. After picking the
>>language and keyboard, the iso is unable to mount. I am using an EFI
>>machine with a usb flash drive and don't have a disk drive to test if
>>the error occurs in that scenario. I tested on three different EFI
>>machines that I have with very different processors and got the same
>>results. One thing that might help is that I saw that linux 5.8
>>entered Sid at the time things stopped working and wonder if a
>>possible regression could be due to that
>
>ACK, looks like a regression in config somewhere - the isofs
>filesystem is missing compared to the last build with 5.7.0-3.

And fixed as of today's daily builds.

Thanks for the report!

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"Yes, of course duct tape works in a near-vacuum. Duct tape works
 anywhere. Duct tape is magic and should be worshipped."
   -― Andy Weir, "The Martian"

Re: Missing daily builds on cdimage.debian.org (Re: Firmware testing images broken)

[Steve McIntyre]

Hey Holger,

On Mon, Sep 14, 2020 at 10:21:39PM +0200, Holger Wansing wrote:
>Steve McIntyre  wrote:
>> Hi Brandon,
>> 
>> On Sun, Sep 13, 2020 at 03:17:11AM -0400, Brandon Werner wrote:
>> >Hi,
>> >
>> >I tried to use one of the Sid DI netinst firmware images and they
>> >seem to be broken for about a week or so now. After picking the
>> >language and keyboard, the iso is unable to mount. I am using an EFI
>> >machine with a usb flash drive and don't have a disk drive to test if
>> >the error occurs in that scenario. I tested on three different EFI
>> >machines that I have with very different processors and got the same
>> >results. One thing that might help is that I saw that linux 5.8
>> >entered Sid at the time things stopped working and wonder if a
>> >possible regression could be due to that
>> 
>> ACK, looks like a regression in config somewhere - the isofs
>> filesystem is missing compared to the last build with 5.7.0-3.
>
>Another issue:
>While debugging the said installer errors, I found that at
>https://cdimage.debian.org/cdimage/daily-builds/daily/
>there are many daily builds missing apparently:
>
>There, we have daily builds for
>20190818-5/
>20190818-7/
>20190819-1/
>20190820-1/
>20190820-3/
>20190820-5/
>20200913-3/
>20200913-5/
>20200913-7/
>20200914-1/
>20200914-3/
>20200914-5/
>
>So there is more than a year missing in between.
>Not good, if you want to boil errors down to a specific package version or 
>commit...
>
>Any idea, why this happened?

I think you've misinterpreted this. We *don't* try to keep the daily
builds around for more than a few days. They're a moving target. The
only reason that the 2019 images are still around at all is that
they're the last few builds where we had a build for mips. (There's
logic in the cleanup scripts to keep a few builds per architecture.)
I'll clean up those old directories now.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"I can't ever sleep on planes ... call it irrational if you like, but I'm
 afraid I'll miss my stop" -- Vivek Das Mohapatra