Re: Identify disks for setup-storage
Thank you all - I have managed to get a hold on my disks, much in the described way. Cheers Thomas On 09/01/2023 22.22, andrew bezella wrote: On Sun, 2023-01-08 at 11:26 +0100, Thomas Lange wrote: Hi Thomas, I use a script to reorder the variable $disklist, so setup-storage can mostly use disk1, but I can define the order of the disk depending on disk types or even the serial number of a disk. The script is class/99-disklist.sh: we use a similar strategy inspired by a 2013 discussion[1]. with batches of homogeneously-built hosts we can use the information in `/dev/disk/by-path` to walk through a known disk layout in our preferred order (see attached). hth, andy 1. https://lists.uni-koeln.de/pipermail/linux-fai/2013-November/010229.html -- Thomas Roth Department: Informationstechnologie Location: SB3 2.291 GSI Helmholtzzentrum für Schwerionenforschung GmbH Planckstraße 1, 64291 Darmstadt, Germany, www.gsi.de Commercial Register / Handelsregister: Amtsgericht Darmstadt, HRB 1528 Managing Directors / Geschäftsführung: Professor Dr. Paolo Giubellino, Dr. Ulrich Breuer, Jörg Blaurock Chairman of the Supervisory Board / Vorsitzender des GSI-Aufsichtsrats: State Secretary / Staatssekretär Dr. Volkmar Dietz
Re: UEFI boot order, Re: Tip: Remote FAI install
Hi, this doesn't seem to be a trivial task as the state of the boot medium prior to the fai installation as well as the UEFI settings for the single network interface both can have all kinds of states, e.g.: (of course for production I'd disable unneeded UEFI setting, in my case all IPv6 and HTTP boot options, so just the IPv4 PXE boot option usually would be active leading to the IPv4 PXE having highest boot priority and then just installed HD/NVMe medium having the second boot priority) root@jammysrv:/srv/fai/config# cat hooks/partition.GRUB_EFI #! /bin/bash # prior to disk partioning collect the UEFI boot order efibootmgr --- root@jammysrv:/srv/fai/config# less /var/log/fai/remote-logs/l65/last/fai.log ---Output [...] Calling hook: partition.GRUB_EFI BootCurrent: 0001 Timeout: 1 seconds BootOrder: 0001,0002,0003,0004,,0005 Boot* ubuntu Boot0001* UEFI: PXE IP4 Intel(R) Ethernet Connection (2) I219-V Boot0002* UEFI: HTTP IP4 Intel(R) Ethernet Connection (2) I219-V Boot0003* UEFI: HTTP IP6 Intel(R) Ethernet Connection (2) I219-V Boot0004* UEFI: PXE IP6 Intel(R) Ethernet Connection (2) I219-V Boot0005* UEFI: PXE IP6 Intel(R) Ethernet Connection (2) I219-V partition.GRUB_EFI OK. [...] root@jammysrv:/srv/fai/config# cat scripts/GRUB_EFI/01-UEFI-Collect_BootOrder #! /bin/bash error=0; trap 'error=$(($?>$error?$?:$error))' ERR # save maximum error code efibootmgr -v exit $error ---Output = shell: GRUB_EFI/01-UEFI-Collect_BootOrder = BootCurrent: 0001 Timeout: 1 seconds BootOrder: 0001,0002,0003,0004,,0005 Boot* ubuntu HD(1,GPT,6205f216-7aa2-4531-8319-d7b77a00514d,0x800,0x10 )/File(\EFI\UBUNTU\GRUBX64.EFI) Boot0001* UEFI: PXE IP4 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci (0x1f,0x6)/MAC(b42e9987238b,0)/IPv4(0.0.0.00.0.0.0,0,0)..BO Boot0002* UEFI: HTTP IP4 Intel(R) Ethernet Connection (2) I219-V PciRoot( 0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv4(0.0.0.00.0.0.0,0,0)/Uri()..BO Boot0003* UEFI: HTTP IP6 Intel(R) Ethernet Connection (2) I219-V PciRoot( 0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv6([::]:<->[::]:,0,0)/Uri()..BO Boot0004* UEFI: PXE IP6 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv6([::]:<->[::]:,0,0)..BO Boot0005* UEFI: PXE IP6 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv6([::]:<->[::]:,0,0)..BO GRUB_EFI/01-UEFI-Collect_BootOrder OK. = shell: GRUB_EFI/10-setup = ainsl: appending to /target/etc/default/grub: GRUB_DISABLE_OS_PROBER=true Installing for x86_64-efi platform. Installation finished. No error reported. Grub installed on /dev/nvme0n1 = (hostdisk//dev/nvme0n1) Sourcing file `/etc/default/grub' Sourcing file `/etc/default/grub.d/init-select.cfg' Generating grub configuration file ... Found linux image: /boot/vmlinuz-5.15.0-58-generic Found initrd image: /boot/initrd.img-5.15.0-58-generic Memtest86+ needs a 16-bit boot, that is not available on EFI, exiting Warning: os-prober will not be executed to detect other bootable partitions. Systems on them will not be added to the GRUB boot configuration. Check GRUB_DISABLE_OS_PROBER documentation entry. Adding boot menu entry for UEFI Firmware Settings ... done GRUB_EFI/10-setupOK. root@jammysrv:/srv/fai/config# cat scripts/GRUB_EFI/90_UEFI_Adjust_BootOrder_pre #! /bin/bash error=0; trap 'error=$(($?>$error?$?:$error))' ERR # save maximum error code efibootmgr -v exit $error ---Output = shell: GRUB_EFI/90_UEFI_Adjust_BootOrder_pre = BootCurrent: 0001 Timeout: 1 seconds BootOrder: ,0001,0002,0003,0004,0005 Boot* ubuntu HD(1,GPT,ac900736-afc3-4b46-8bb5-ce1c1b71c9a8,0x800,0x10)/File(\EFI\ubuntu\grubx64.efi) Boot0001* UEFI: PXE IP4 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv4(0.0.0.00.0.0.0,0,0)..BO Boot0002* UEFI: HTTP IP4 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv4(0.0.0.00.0.0.0,0,0)/Uri()..BO Boot0003* UEFI: HTTP IP6 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv6([::]:<->[::]:,0,0)/Uri()..BO Boot0004* UEFI: PXE IP6 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv6([::]:<->[::]:,0,0)..BO Boot0005* UEFI: PXE IP6 Intel(R) Ethernet Connection (2) I219-V PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b42e9987238b,0)/IPv6([::]:<->[::]:,0,0)..BO GRUB_EFI/90_UEFI_Adjust_BootOrder_pre OK. root@jammysrv:/srv/fai/config# cat scripts/GRUB_EFI/91_UEFI_Adjust_BootOrder #! /bin/bash error=0; trap 'error=$(($?>$error?$?:$error))' ERR # save maximum error code # FAI List 19.01.2023, 21:10h (Thomas Lange) #Hi, # #I found this code that move the first boot entry (which is expected to #be the new entry after an installation) to the end of the boot list. #
Re: Create fai-cd for other architecture?
On Wed, 2023-01-25 at 14:12:56 +0100, Thomas Lange wrote: > > On Wed, 25 Jan 2023 13:43:30 +0100, Steffen Grunewald > > said: > > > Cross-platform doesn't work since "grub-mkstandalone" would throw an > > "exec format error". > > Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 > hardcoded. > What about running a qemu cross-arch VM on amd64? It would be slow, > but should be doable? Fixing the hard-coded arch is easy if you tell > me how this looks for arm64. I could boot the machine and perform some experiments; a RasPi on the other hand isn't that slow (but at home, not at the office now). I already replaced the few amd64 occurrences in fai-cd with arm64, but got stuck when it proceeded to look for grub-i386. This is pure EFI so no special tricks would be required. Do we actually need the whole grub stuff? > I'm happy for some hints how to support fai-cd on arm64, and maybe > also cross-arch available. But I'm not sure if grub-mkstandalone can > do cross-arch. Perhaps I'm thinking too complicated. All I want to do is start a kernel and initrd, assign a (fixed, as I don't trust DHCP for this one) IP address and set the kernel cmdline I'd otherwise pass to PXE. A first attempt to use a Debian metinstall image, and edit the boot config, has failed, I suspect I got the root= parameter wrong. Got to look that up for the case there had been no proper DHCP handshake before. - S
Re: Create fai-cd for other architecture?
> On Wed, 25 Jan 2023 13:43:30 +0100, Steffen Grunewald > said: > Cross-platform doesn't work since "grub-mkstandalone" would throw an > "exec format error". > Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 hardcoded. What about running a qemu cross-arch VM on amd64? It would be slow, but should be doable? Fixing the hard-coded arch is easy if you tell me how this looks for arm64. I'm happy for some hints how to support fai-cd on arm64, and maybe also cross-arch available. But I'm not sure if grub-mkstandalone can do cross-arch. -- reagrds Thomas
Re: Create fai-cd for other architecture?
On Wed, 2023-01-25 at 13:04:44 +0100, Steffen Grunewald wrote: > Hello, > > I've got an ARM64 server that doesn't do PXE properly, and (of course...) > I don't have a second ARM64 machine at hand to prepare a FAI CD. > The NFSROOT already exists. > Is it possible to run fai-cd on AMD64, with some setup/config tricks, to > create an ISO image that can be booted on the ARM machine, or do I have > to find something with the right architecture (a RasPi running Debian > probably would do)? Answering myself: Cross-platform doesn't work since "grub-mkstandalone" would throw an "exec format error". Running "fai-cd" on arm64 doesn't work since it has x86_64 and i386 hardcoded. Any alternative to create a bootable stick for this arm64 machine, to mount a remote nfsroot and access a remote config tree? (Just take an installer stick image and replace the kernel and initrd, and provide a kernel commandline?) - S
Create fai-cd for other architecture?
Hello, I've got an ARM64 server that doesn't do PXE properly, and (of course...) I don't have a second ARM64 machine at hand to prepare a FAI CD. The NFSROOT already exists. Is it possible to run fai-cd on AMD64, with some setup/config tricks, to create an ISO image that can be booted on the ARM machine, or do I have to find something with the right architecture (a RasPi running Debian probably would do)? Thanks, S -- Steffen Grunewald, Cluster Administrator Max Planck Institute for Gravitational Physics (Albert Einstein Institute) Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany ~~~ Fon: +49-331-567 7274 Mail: steffen.grunewald(at)aei.mpg.de ~~~
