Re: Replace discover by isenkram in d-i, better integration with virtualizations/clouds
Le 04/12/2017 à 02:22, Thomas Goirand a écrit : > On 12/03/2017 05:30 PM, Raphael Hertzog wrote: >> In the last years, Petter Rheinholdtsen worked on isenkram[2] with a >> similar but a bit broader goal. I noticed it has better support >> of clouds and that it will install some virtualization/cloud-related >> packages automatically whereas discover does not. It also makes it easier >> to install the appropriate firmware packages. > > Raphael, > > Could you give examples of packages that gets installed this way? Also, > how is d-i related to cloud? The images aren't generated using d-i > anyway, so I don't see how the cloud images would be affected. > > Cheers, > > Thomas Goirand (zigo) > Vagrant images for Virtual Box are still using d-i, if that counts as cloud image. However I haven't noted anything problematic using d-i on qemu for what concerns virtualization/cloud support detection in my use case. Actually I prefer the installer not too be smart, for instance I am not sure if I want the installer installing automatically a qemu-guest-agent package if it detects qemu as the underlying platform.
Bug#872948: debootstrap: Debootstrap does not explain what is calls a Debian base system
Le 23/08/2017 à 10:12, Cyril Brulebois a écrit :
> Ansgar Burchardt <ans...@debian.org> (2017-08-23):
>> Emmanuel Kasper <m...@debian.org> writes:
>>> The default base system installed by debootstrap includes all packages
>>> with Pritority essential and
>>> important, but this was not yet documented.
>>
>> There is no "essential" priority. There is only "required" (and its
>> dependencies).
>
> Well, we also have “Essential: yes” packages, which is maybe what Emmanuel
> had in mind? Those are handled by apt rather than by debootstrap though.
>
Hum this was not actually what I had in mind, so here I attach a V2 with
the right wording (s/essential/required/), also removing the TARGET
mention since the first section of the man page is clear about that.
diff --git a/debootstrap.8 b/debootstrap.8
index e802003..5cdd6a6 100644
--- a/debootstrap.8
+++ b/debootstrap.8
@@ -74,13 +74,11 @@ With this option set, this behaviour is disabled.
.IP "\fB\-\-variant=minbase|buildd|fakechroot\fP"
Name of the bootstrap script variant to use.
Currently, the variants supported are minbase, which only includes
-essential packages and apt; buildd, which installs the build-essential
-packages into
-.IR TARGET ;
-and fakechroot, which installs the packages without root privileges.
-The default, with no \fB\-\-variant=X\fP argument, is to create a base
-Debian installation in
-.IR TARGET .
+\fIrequired\fR packages and apt; buildd, which installs the build-essential
+packages and fakechroot, which installs the packages without root privileges.
+The default, with no \fB\-\-variant=X\fP argument, is to create a
+base Debian installation with all packages of priority \fIrequired\fR and
+\fIimportant\fR, including apt.
.IP
.IP "\fB\-\-merged-usr\fP"
Create /{bin,sbin,lib}/ symlinks pointing to their counterparts in /usr/.
Bug#872948: debootstrap: Debootstrap does not explain what is calls a Debian base system
Package: debootstrap
Version: 1.0.89
Severity: minor
Tags: patch
The debootstrap man page says:
The default, with no --variant=X argument, is to create a base Debian
installation in TARGET.
but does not explain what comes in the base Debian installation.
The patch included tries to improve that.
-- System Information:
Debian Release: 9.1
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 3.16.0-4-amd64 (SMP w/1 CPU core)
Locale: LANG=C.UTF-8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8), LANGUAGE=C.UTF-8
(charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages debootstrap depends on:
ii wget 1.18-5
Versions of packages debootstrap recommends:
ii debian-archive-keyring 2017.5
ii gnupg 2.1.18-6
debootstrap suggests no packages.
-- no debconf information
commit 5e18585594bf93a1bec5e9f4f2496e016084805c
Author: Emmanuel Kasper <m...@debian.org>
Date: Tue Aug 22 22:12:21 2017 +0200
Document which packages are installed by a default variant
The default base system installed by debootstrap includes all packages with
Pritority essential and
important, but this was not yet documented.
diff --git a/debootstrap.8 b/debootstrap.8
index e802003..a3afc90 100644
--- a/debootstrap.8
+++ b/debootstrap.8
@@ -74,13 +74,13 @@ With this option set, this behaviour is disabled.
.IP "\fB\-\-variant=minbase|buildd|fakechroot\fP"
Name of the bootstrap script variant to use.
Currently, the variants supported are minbase, which only includes
-essential packages and apt; buildd, which installs the build-essential
+\fIessential\fR packages and apt; buildd, which installs the build-essential
packages into
.IR TARGET ;
and fakechroot, which installs the packages without root privileges.
-The default, with no \fB\-\-variant=X\fP argument, is to create a base
-Debian installation in
-.IR TARGET .
+The default, with no \fB\-\-variant=X\fP argument, is to create a
+base Debian installation with all packages of priority \fIessential\fR and
+\fIimportant\fR, including apt.
.IP
.IP "\fB\-\-merged-usr\fP"
Create /{bin,sbin,lib}/ symlinks pointing to their counterparts in /usr/.
Bug#853855: (no subject)
Le 03/02/2017 à 23:55, Cyril Brulebois a écrit : > Ian Campbell <i...@hellion.org.uk> (2017-02-03): >> On Fri, 2017-02-03 at 15:51 +0100, Emmanuel Kasper wrote: >>> Actually on further research, net.ifnames and most dot-containing >>> parameters are not here for the kernel, but to configure on boot >>> various systemd components, >> >> d-i doesn't use systemd, does it? > > It certainly doesn't right now. From https://packages.debian.org/sid/udev-udeb d-i has udev built from systemd source. The parameters from https://www.freedesktop.org/software/systemd/man/systemd-udevd.service.html# can be be passed to the udev service of the installer (notice the abundance of dots in those parameter :) signature.asc Description: OpenPGP digital signature
Bug#853855: (no subject)
Actually on further research, net.ifnames and most dot-containing parameters are not here for the kernel, but to configure on boot various systemd components, the list of which can be found in systemd-232/man/kernel-command-line.xml or online in https://www.freedesktop.org/software/systemd/man/kernel-command-line.html
Bug#853855: di-utils: Kernel boot options containing a dot are not propagated to the installed system
Le 03/02/2017 à 12:38, Ian Campbell a écrit :
> On Fri, 2017-02-03 at 12:22 +0100, Emmanuel Kasper wrote:
>>>> A kernel boot param like net.ifnames=0 will be skipped when the
>>>> installer parses the boot option for setting the bootloader.
>>>>
>>>> Found in di-utils:
>>>>
>>>> # Skip module-specific variables
>>>> varnodot="${var##*.*}"
>>>> if [ "$varnodot" = "" ]; then
>>>> continue
>>>> fi
>>>>
>>>> So basically any option containing a dot is not propagated to the
>>>> installed system. This was introduced by
>>>> 7cf15980d714da8b958a73c93459ee09fdbb9415 ("Skip new module-
>>>> specific
>>>> parameters in user-params.")
>>>>
>>>> I found no documented or obvious reason for this behaviour.
>>>
>>> Sounds like the assumption was that any "foo.bar=baz" arguments
>>> were
>>> always to be used as the "bar=baz" option when loading the "foo"
>>> module
>>> (i.e. "modprobe foo bar=baz"), which I think the installer supports
>>> (for convenience) but perhaps not the installed system (where they
>>> should instead be in /etc/modules or /etc/modprobe.conf or similar)
>>> does not?
>>>
>>> which I think the installer supports
>>> (for convenience) but perhaps not the installed system (where they
>>> should instead be in /etc/modules or /etc/modprobe.conf or similar)
>>
>> Thanks for the analysys, it looks like a very much plausible rationale
>> behind this commit.
>>
>> If I understand you correctly, net.ifnames is understood as
>> a kernel module option, and modules options are not propagated to the
>> bootloader, because they "should" be configured in
>> /etc/modprobe.d/my_module.
>
> That's along the lines of what I think might be happening, yess
>
>> Actually I might be missing something, but if for instance I need a
>> kernel module option to make the installer boot on my hardware, like a
>> radeon.modeset=0, I probably need this on the installed system as well ?
>
> Correct, my (unchecked) assumption was that something in d-i would be
> doing so.
>
> You might find evidence of that in the fact that your net.ifnames=bar
> ended up under /etc/ somewhere, either in /etc/modprobe.d/* or
> /etc/modprobe.conf or /etc/modules.
>
good catch
virt-cat -a testing.build/testing.raw /etc/modprobe.d/local.conf
# Local module settings
# Created by the Debian installer
options net ifnames=0
Bug#853855: di-utils: Kernel boot options containing a dot are not propagated to the installed system
>> A kernel boot param like net.ifnames=0 will be skipped when the
>> installer parses the boot option for setting the bootloader.
>>
>> Found in di-utils:
>>
>> # Skip module-specific variables
>> varnodot="${var##*.*}"
>> if [ "$varnodot" = "" ]; then
>> continue
>> fi
>>
>> So basically any option containing a dot is not propagated to the
>> installed system. This was introduced by
>> 7cf15980d714da8b958a73c93459ee09fdbb9415 ("Skip new module-specific
>> parameters in user-params.")
>>
>> I found no documented or obvious reason for this behaviour.
>
> Sounds like the assumption was that any "foo.bar=baz" arguments were
> always to be used as the "bar=baz" option when loading the "foo" module
> (i.e. "modprobe foo bar=baz"), which I think the installer supports
> (for convenience) but perhaps not the installed system (where they
> should instead be in /etc/modules or /etc/modprobe.conf or similar)
> does not?
>
> which I think the installer supports
> (for convenience) but perhaps not the installed system (where they
> should instead be in /etc/modules or /etc/modprobe.conf or similar)
Thanks for the analysys, it looks like a very much plausible rationale
behind this commit.
If I understand you correctly, net.ifnames is understood as
a kernel module option, and modules options are not propagated to the
bootloader, because they "should" be configured in
/etc/modprobe.d/my_module.
Actually I might be missing something, but if for instance I need a
kernel module option to make the installer boot on my hardware, like a
radeon.modeset=0, I probably need this on the installed system as well ?
Bug#853855: di-utils: Kernel boot options containing a dot are not propagated to the installed system
Package: di-utils
Version: 1.117
Severity: minor
Tags: d-i
A kernel boot param like net.ifnames=0 will be skipped when the installer
parses the boot option for setting the bootloader.
Found in di-utils:
# Skip module-specific variables
varnodot="${var##*.*}"
if [ "$varnodot" = "" ]; then
continue
fi
So basically any option containing a dot is not propagated to the
installed system. This was introduced by
7cf15980d714da8b958a73c93459ee09fdbb9415 ("Skip new module-specific
parameters in user-params.")
I found no documented or obvious reason for this behaviour.
-- System Information:
Debian Release: 8.7
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 3.16.0-4-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Passing parameter to grub via installer boot parameter
Hi I am reading in the debian installer guide A “---” in the boot options has special meaning. Kernel parameters that appear after the last “---” may be copied into the bootloader configuration for the installed system (if supported by the installer for the bootloader). The installer will automatically filter out any options (like preconfiguration options) that it recognizes. I can't get this to pass a parameter net.ifnames=0 amd amd64 using the netinst amd64 iso. The parameter is properly handled by the kernel, but never ends up in /etc/default/grub like I expected. /proc/cmdline as seen from inside the running installer looks like this: BOOT_IMAGE=/install.amd/vmlinuz vga=788 initrd=/install.amd/initrd.gz --- quiet net.ifnames=0 preseed/url=http://10.0.2.2:8148/testing-preseed.cfg auto locale=en_US kbd-chooser/method=us netcfg/get_hostname=testing.raw netcfg/get_domain=vagrantup.com fb=false debconf/frontend=noninteractive console-setup/ask_detect=false console-keymaps-at/keymap=us keyboard-configuration/xkb-keymap=us am I missing something ? Emmanuel
Bug#806273: use grub-mount as the sole source of partition probes (disable kernel readonly mounts)
changes since v1:
* do not fallback on dangerous read only kernel mounts if grub-mount is
missing, just exit with error
>From 34a2c247fa08d4e01aa08b5b75977c66d71df4f8 Mon Sep 17 00:00:00 2001
From: Emmanuel Kasper <emman...@libera.cc>
Date: Tue, 15 Nov 2016 14:52:23 +0100
Subject: [PATCH v2] use grub-mount as the sole source of partition probes
(disable kernel readonly mounts)
the read only kernel mounts of os-probes caused various data corruption in virtual machines
and exported block devices due to the following chain of event:
1. os-prober tries to mount via grub-mount each block device as seen from /sys/block
2. in case of iscsi exported block devices or virtualization environment, such
a block device could be a whole disk image with a partition table
3. since grub-mount expects a filesystem superblock but encounters a partition table
it fails and then give hand to
4. kernel read only mounts, calling the function ro_partition
5. the ro_partition function sets the block device readonly via blockdev --setro
6. a number of kernel mounts are attempted via various kernel modules
7. the block device is set to readwrite
now when I/O happened on the iscsi initiator or virtual machines between 5-7
the blocks cannot be flushed to the block device since it has been
locked by os-prober. This causes a filesystem error and the filesystem to be
remounted read only.
since grub-mount is now available on all the platforms debian supports
we assume we can disable the risky behaviour without losing too much os-prober functionnality
grub-mount has also now support for all filesystems which the kernel knows,
the exception being QNX
---
debian/control | 2 +-
os-probes/common/50mounted-tests | 27 +++
2 files changed, 12 insertions(+), 17 deletions(-)
diff --git a/debian/control b/debian/control
index 10459bd..ac307f5 100644
--- a/debian/control
+++ b/debian/control
@@ -22,7 +22,7 @@ Package: os-prober
Architecture: any
Section: utils
Priority: extra
-Depends: ${shlibs:Depends}, ${misc:Depends}
+Depends: ${shlibs:Depends}, ${misc:Depends}, grub-common
Description: utility to detect other OSes on a set of drives
This package detects other OSes available on a system and outputs the
results in a generic machine-readable format.
diff --git a/os-probes/common/50mounted-tests b/os-probes/common/50mounted-tests
index 561163b..8e1c87f 100755
--- a/os-probes/common/50mounted-tests
+++ b/os-probes/common/50mounted-tests
@@ -47,25 +47,20 @@ fi
mounted=
if type grub-mount >/dev/null 2>&1 && \
- type grub-probe >/dev/null 2>&1 && \
- grub-mount "$partition" "$tmpmnt" 2>/dev/null; then
- mounted=1
- type="$(grub-probe -d "$partition" -t fs)" || true
- if [ "$type" ]; then
- debug "mounted using GRUB $type filesystem driver"
- else
- debug "mounted using GRUB, but unknown filesystem?"
+ type grub-probe >/dev/null 2>&1; then
+ if grub-mount "$partition" "$tmpmnt" 2>/dev/null; then
+ mounted=1
+ type="$(grub-probe -d "$partition" -t fs)" || true
+ if [ "$type" ]; then
+ debug "mounted using GRUB $type filesystem driver"
+ else
+ debug "mounted using GRUB, but unknown filesystem?"
type=fuseblk
+ fi
fi
else
- ro_partition "$partition"
- for type in $types $delaytypes; do
- if mount -o ro -t "$type" "$partition" "$tmpmnt" 2>/dev/null; then
- debug "mounted as $type filesystem"
- mounted=1
- break
- fi
- done
+ echo "Cannot find grub-mount (Try installing grub-common)" >&2
+ exit 1
fi
if [ "$mounted" ]; then
--
2.1.4
Bug#648208: os-prober: blockdev --setro affects running kvm instances
>> Since version 1.45, os-prober instead uses grub-mount when it's available >> -- and if grub is installed to use os-prober, it will pull it in. > >> So unless another bootloader is also using os-prober, or someone >> installs and uses it by hand, this won't happen in unstable/testing. > It's not really the case. I agree here with Jean Francois. Here the problematic code in os-probes/common/50mounted-tests line 49, simplified for better understanding if type grub-mount && type grub-probe && grub-mount "$partition" ; etc ... else blockdev --setro done fi the problem here is that when using LVM what 50mounted-tests treats as a $partition might very well be a disk image (512 first bytes will contain a MBR) In that case grub-mount will fail and pass the hand to ro_partition which will be happily set the whole device as read only, causing the file system errors in the guest poking on the idea of Michael I was wondering it would be possible to test before calling blockdev --setro if the block device we target has a file descriptor pointing to it. This would indicate that something/someone is actually doing something with the block device and we should rather leave it alone. in pseudo shell it would look like: if ls -l /proc/*/fd | grep $(realpath $partition); then debug "active file descriptor on $partition) return else blockdev --setro fi I find this a bit hacky however. What we also do would be simply to change the depency on os-prober from "recommends" to "suggests" in os-prober, and add os-prober as a dependency of the desktop task of the debian-installer (which is selected by default) so that non-server users get it installed.
Solving os-prober related problems by adding it to the desktop task during installation
Hello I have noticed during fresh debian installs the package os-prober is automatically installed when you select grub as a bootloader, as a 'Recommends' of grub-common. Unfortunately the way os-prober detects other OSes on local media, might prove cumbersome or even dangerous if you use do virtualization. For instance if using partitions to store Xen DomU, os prober will create a grub entry for each of your VM in the grub menu list. There is also this bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701814 where iscsi-exported block devices would get corrupted by os-prober. To get around this problems, I was wondering if it would a good idea to downgrad the os-prober 'Recommends' to 'Suggests', and add os-prober as a package in the desktop Taskel. I have yet to see a dualboot server install, but on a workstation os-prober makes sense. The problem that I see with this reasoning, is that it assumes a debian text-only installation does not require dual boot. Comments ? Emmanuel -- To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55658b26.5090...@proxmox.com
Bug#368119: installation-report: parted fails to read partition table on targetted hard drive
Package: installation-report Severity: normal Package: installation-reports Boot method: CD Image version: debian-testing-alpha-netinst downloaded the 19/05/2006 Date: 20/05/2006 Machine: Compaq Alphaserver DS10 with SRM console Processor:EV6 Memory:512M Partitions: As seen by GNU parted parted /dev/hda print ( targeted install disk ) Error: Unable to satisfy all constraints on the partition. Information: Don't forget to update /etc/fstab, if necessary. parted /dev/sda print ( sarge disk ) Disk geometry for /dev/sda: 0.000-8678.478 megabytes Disk label type: bsd MinorStart End Filesystem Flags 1 0.000 0.954 2 0.955 77.249 ext2 3 77.249 5990.030 ext3 4 5990.030 6234.171 linux-swap 5 6234.171 8678.478 ext3 As seen by fdisk fdisk -l /dev/hda Disk /dev/hda: 120.0 GB, 120034123776 bytes 255 heads, 63 sectors/track, 14593 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes 8 partitions: # start end size fstype [fsize bsize cpg] a:1*1*0* ext2 b:1 1217 1217 ext2 c: 1217 7782 6566 ext2 d: 7782 14593 6812 ext2 fdisk -l /dev/sda Disk /dev/sda: 9100 MB, 9100044288 bytes 255 heads, 63 sectors/track, 1106 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes 6 partitions: # start end size fstype [fsize bsize # cpg] a:1*1*0* ext2 b:1* 10*9* ext2 c: 10* 764* 753* ext2 d: 764* 795* 31* swap e: 795* 1107* 311* ext2 Output of lspci and lspci -n: :00:01.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) :00:07.0 ISA bridge: ALi Corporation M1533 PCI to ISA Bridge [Aladdin IV] (rev c3) :00:09.0 Ethernet controller: Digital Equipment Corporation DECchip 21142/43 (rev 41) :00:0b.0 Ethernet controller: Digital Equipment Corporation DECchip 21142/43 (rev 41) :00:0d.0 IDE interface: ALi Corporation M5229 IDE (rev c1) :00:0e.0 PCI bridge: Hint Corp HB6 Universal PCI-PCI bridge (non-transparent mode) (rev 15) :00:0f.0 Multimedia audio controller: Creative Labs SB Live! EMU10k1 (rev 07) :00:0f.1 Input device controller: Creative Labs SB Live! MIDI/Game Port (rev 07) :00:10.0 USB Controller: NEC Corporation USB (rev 43) :00:10.1 USB Controller: NEC Corporation USB (rev 43) :00:10.2 USB Controller: NEC Corporation USB 2.0 (rev 04) :00:11.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05) :01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 85 :00:01.0 0c03: 10b9:5237 (rev 03) :00:07.0 0601: 10b9:1533 (rev c3) :00:09.0 0200: 1011:0019 (rev 41) :00:0b.0 0200: 1011:0019 (rev 41) :00:0d.0 0101: 10b9:5229 (rev c1) :00:0e.0 0604: 3388:0021 (rev 15) :00:0f.0 0401: 1102:0002 (rev 07) :00:0f.1 0980: 1102:7002 (rev 07) :00:10.0 0c03: 1033:0035 (rev 43) :00:10.1 0c03: 1033:0035 (rev 43) :00:10.2 0c03: 1033:00e0 (rev 04) :00:11.0 0100: 1077:1020 (rev 05) :01:00.0 0300: 102b:0525 (rev 85) Base System Installation Checklist: [O] = OK, [E] = Error (please elaborate below), [ ] = didn't try it Initial boot worked:[O] Configure network HW: [O] Config network: [O] Detect CD: [O] Load installer modules: [O] Detect hard drives: [O] Partition hard drives: [E] Create file systems:[ ] Mount partitions: [ ] Install base system:[ ] Install boot loader:[ ] Reboot:
