Bug#881969: making bootable SD cards
Vagrant Cascadian wrote: > u-boot-install --board=Cubietruck --device=/dev/mmcblk0 > > u-boot is where the information such as > supported boot media and device offsets generally comes from, as it > sometimes changes changes between different versions of u-boot Verison specificity is another big reason to need this in one place and not scattered around. > Doing this would also make me want to split the flash-kernel database > out into a separate package from the boot script/kernel+initrd copying > parts of flash-kernel It would also be useful to be able to query the flash-kernel database for particular fields to avoid other duplication of info. If there were a way to query for the kernel image variant to use for a board, I could use that in propellor. -- see shy jo signature.asc Description: PGP signature
Bug#881969: making bootable SD cards
On 2017-11-17, Karsten Merker wrote: > On Thu, Nov 16, 2017 at 07:54:42PM -0400, Joey Hess wrote: >> A utility in flash-kernel would be much nicer than needing to puzzle out dd >> commands from README.Debian files and hope you got it right. I'm currently >> having to embed those dd commands inside propellor; they're also embedded >> inside debian-installer (build/boot/arm/u-boot-image-config). I've definitely thought about providing such a utility in u-boot-tools that at least handles a limited set of boards, and replace part of the image generation code in debian-installer. This could also be used to create independently bootable images. > Among the points of this discussion were: > > - On some devices u-boot isn't stored on an SD card but on an > onboard SPI flash chip with a rather limited number of write > cycles (in the area of ~1000) and no defects management. > Unconditionally writing u-boot on every invocation of > flash-kernel (which includes everything that modifies the > initrd) would have the potential to kill these devices in > comparatively short time. We could leave those devices as unsupported... > - Knowing the device type one is running on isn't necessarily > enough. Several supported devices are available in different > hardware configuration variants that influence where the u-boot > image can get written to (SD card, onboard eMMC, onboard raw > NAND, SPI flash, and combinations thereof). Once Linux is > running, there is no way to determine where the u-boot that > brought the system up was loaded from. Flash-kernel pulls the > system type from a /proc entry, but that doesn't provide the > information whether the current device e.g. has the SPI flash > for u-boot populated or not, and if yes, whether it has > actually been used for booting the system, so flash-kernel > cannot decide without user-interaction where to write the > u-boot image. Maybe not in an automated fashion, but specifying something like: u-boot-install --board=Cubietruck --device=/dev/mmcblk0 Or even with limited autodetection: u-boot-install --board=auto --device=foo.img u-boot-install --board=auto --device=/dev/mmcblk2 This could at least simplify the process of looking up the exact offsets and so on, and fail if it can't safely determine what to do. And --force might be an option as well. This could also be called by debian-installer, instead of maintaining it's own list of offsets... What's been hard is figuring out if this should be in u-boot-tools or part of flash-kernel. Obviously, flash-kernel has the database of boards based on /proc/device-tree/model, but u-boot is where the information such as supported boot media and device offsets generally comes from, as it sometimes changes changes between different versions of u-boot, and I hate updating things in multiple places. After having thought about it more while of writing this email, I think it should be in u-boot-tools, and *optionally* use the flash-kernel database to autodetect the appropriate device. Autodetection might require another field or two in the flash-kernel database... Doing this would also make me want to split the flash-kernel database out into a separate package from the boot script/kernel+initrd copying parts of flash-kernel; there is now the u-boot-menu package which can be used to generate an extlinux style menu instead of flash-kernel's boot scripts, and with u-boot EFI emulation, grub-efi can be used as well in some cases. live well, vagrant signature.asc Description: PGP signature
Bug#881969: making bootable SD cards
Karsten Merker wrote: > to use d-i/flash-kernel on the target device, one obviously needs > to already have put a u-boot onto the device in some form (such > as preinstalled in the d-i SD card images), otherwise one > couldn't have booted it Not necessarily, see for example /target in d-i when the machine has been booted from other media than the target disk. As noted in my initial message, d-i does not handle this in all cases, requiring clumsy warnings on wiki pages to warn the user about its deficiencies. If flash-kernel provided a way to do it, d-i could easily to it for at least cases where u-boot is installed on a safe media like a SD card. > As a result of these issues, it was deemed unsuitable for > flash-kernel to attempt installing u-boot. A separate program included in flash-kernel that looks at the machine database to determine how to install u-boot and installs it to a specified device would avoid all of those issues. That is what I am suggesting. -- see shy jo signature.asc Description: PGP signature
Bug#881969: making bootable SD cards
Control: severity 881969 wishlist [CCing Vagrant Cascadian, the Debian u-boot maintainer] On Thu, Nov 16, 2017 at 07:54:42PM -0400, Joey Hess wrote: > Package: flash-kernel > Version: 3.87 > Severity: normal > > Therefore you usually have to setup an SD card with the appropriate u-boot > version for your particular device (see below) as a prerequisite for > installing Debian. If you use the pre-made SD card images with the > installer, this step is not necessary, as these images already contain > u-boot. > -- https://wiki.debian.org/InstallingDebianOn/Allwinner > > This seems to fall squarely in flash-kernel's wheelhouse, since it > already handles the other parts of u-boot installation, and it knows > the name of the board being installed. > > The d-i SD card images avoid the problem, but they are only one way to > install; there are even other ways to use d-i to install that need this > manual step first. > > The main difficulty in putting it in flash-kernel is that it might be > installed in a chroot in a situation where it should not be altering > the boot sector of the host's disk. So, something like grub-installer > seems to be called for, so the user specifies the device to install to. > > A utility in flash-kernel would be much nicer than needing to puzzle out dd > commands from README.Debian files and hope you got it right. I'm currently > having to embed those dd commands inside propellor; they're also embedded > inside debian-installer (build/boot/arm/u-boot-image-config). Hello, to use d-i/flash-kernel on the target device, one obviously needs to already have put a u-boot onto the device in some form (such as preinstalled in the d-i SD card images), otherwise one couldn't have booted it, i.e. flash-kernel could only cover the topic of updating u-boot from within a running system. There has been a discussion about the latter in the past and the consensus reached at that time was that it is practically impossible to safely determine the version of an already installed u-boot image in a platform-independant way, and installing u-boot unconditionally on every invocation of flash-kernel is considered too riscy as there are a number of unsolved problems with such an approach. Among the points of this discussion were: - On some devices u-boot isn't stored on an SD card but on an onboard SPI flash chip with a rather limited number of write cycles (in the area of ~1000) and no defects management. Unconditionally writing u-boot on every invocation of flash-kernel (which includes everything that modifies the initrd) would have the potential to kill these devices in comparatively short time. - Knowing the device type one is running on isn't necessarily enough. Several supported devices are available in different hardware configuration variants that influence where the u-boot image can get written to (SD card, onboard eMMC, onboard raw NAND, SPI flash, and combinations thereof). Once Linux is running, there is no way to determine where the u-boot that brought the system up was loaded from. Flash-kernel pulls the system type from a /proc entry, but that doesn't provide the information whether the current device e.g. has the SPI flash for u-boot populated or not, and if yes, whether it has actually been used for booting the system, so flash-kernel cannot decide without user-interaction where to write the u-boot image. - As u-boot is more than just a bootloader - it also provides BIOS-like functionality - there can be a major difference between messing up automatically installing GRUB and messing up automatically installing u-boot. In the GRUB case, the user can simply boot a rescue system and fix the bootloader. In case of a broken u-boot installation to an SPI flash or to an eMMC on systems where these have a higher boot priority than the SD slot, the system can be completely dead and require specific hardware tooling (such as an external SPI flasher) to revive the system again. As a result of these issues, it was deemed unsuitable for flash-kernel to attempt installing u-boot. What we might do sometime in the future is adding a u-boot-installer udeb to d-i which on a very limited subset of systems allows the user to explicitly decide to install u-boot to a user-selected device (such as eMMC or SPI flash) after being informed about the riscs of doing so. I had started designing a proof-of-concept for such a udeb, but have had to put that on hold due to having to take care of a number of higher-priority issues. I'm setting the severity of this bug down from "normal" to "wishlist" as it is about requesting the addition of a new feature and not about a bug in existing functionality. Regards, Karsten -- Gem. Par. 28 Abs. 4 Bundesdatenschutzgesetz widerspreche ich der Nutzung sowie der Weitergabe meiner personenbezogenen Daten für Zwecke der Werbung sowie der Markt- oder Meinungsforschung.
Bug#881969: making bootable SD cards
Package: flash-kernel Version: 3.87 Severity: normal Therefore you usually have to setup an SD card with the appropriate u-boot version for your particular device (see below) as a prerequisite for installing Debian. If you use the pre-made SD card images with the installer, this step is not necessary, as these images already contain u-boot. -- https://wiki.debian.org/InstallingDebianOn/Allwinner This seems to fall squarely in flash-kernel's wheelhouse, since it already handles the other parts of u-boot installation, and it knows the name of the board being installed. The d-i SD card images avoid the problem, but they are only one way to install; there are even other ways to use d-i to install that need this manual step first. The main difficulty in putting it in flash-kernel is that it might be installed in a chroot in a situation where it should not be altering the boot sector of the host's disk. So, something like grub-installer seems to be called for, so the user specifies the device to install to. A utility in flash-kernel would be much nicer than needing to puzzle out dd commands from README.Debian files and hope you got it right. I'm currently having to embed those dd commands inside propellor; they're also embedded inside debian-installer (build/boot/arm/u-boot-image-config). -- System Information: Debian Release: buster/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.13.0-1-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), LANGUAGE=en_US.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) -- see shy jo signature.asc Description: PGP signature
