Re: Define sda as the smallest disk
Hi Diego,
Nice catch for the scsi grep.
Will try that ASAP.
Thanks
Le ven. 23 févr. 2024 à 14:35, Diego Zuccato a écrit :
>
> Just a followup.
> My script had a serious bug (plus a typo), too: since it gets sourced,
> $0 is not what I expected :( After many failed atttempts finally I
> noticed it and patched.
>
> What I just tested:
> -8<--
> root@fai:/srv/fai/config/class# cat 99-disklist.sh
> #! /bin/bash
>
> mydisks() {
>
> find $* -type l -printf "%f %l\n" | grep -Pv
> '^md|-part\d|^wwn-|^nvme-eui|^nvme-nvme' | egrep '^scsi|^ata|^nvme' |
> sed -e 's#.*/##g'| tr '\n' ' '
> }
>
> # This is really important, because we use shell globbing for creating
> the list of disks
> cd /dev/disk/by-id || echo Cannot get disk information
>
> filter='nvme*'
>
> # can not use $0: script is sourced, not run explicitly !
> datadir=$FAI/class/99-disklist.d
> echo " Testing $datadir/$HOSTNAME"
> if [ -f $datadir/$HOSTNAME ] ; then
> # Source host-specific list. Can define a new list or override filter
> . $datadir/$HOSTNAME
> fi
>
> if [ -z $newlist ]; then
> echo "newlist is empty, filter=$filter"
> newlist=$(mydisks $filter )
> fi
>
> if [ -n "$newlist" ]; then
> echo New disklist in 99-disklist.sh: $newlist
> echo "disklist=\"$newlist\" # $0" >> $LOGDIR/additional.var
> fi
> -8<--
>
> And 99-disklist.d/fast00 (the host I'm installing) contains:
> -8<--
> #!/bin/bash
> #filter='scsi-*'
> #newlist='sdt '
> . /usr/lib/fai/subroutines
>
> newlist=$(smallestdisk)
> -8<--
>
> Hope it can be useful for others.
>
> Diego
>
> Il 22/02/2024 09:02, Diego Zuccato ha scritto:
> > I think there's a bug (well, a missing piece) in 99-disklist.sh: egrep
> > ignores SCSI disk, considering just ATA and NVME.
> >
> > If it can be useful, I also modified the script to look into
> > 99-disklist.d for hostname-specific configs (I prefer having separate
> > files instead of embedding in a bigger one):
> >
> > -8<-- 99-disklist -8<--
> > #! /bin/bash
> >
> > mydisks() {
> >
> > find $* -type l -printf "%f %l\n" | grep -Pv
> > '^md|-part\d|^wwn-|^nvme-eui|^nvme-nvme' | egrep '^scsi|^ata|^nvme' |
> > sed -e 's#.*/##g'| tr '\n' ' '
> > }
> >
> > # This is really important, because we use shell globbing for creating
> > the list of disks
> > cd /dev/disk/by-id || echo Cannot get disk information
> >
> > filter='nvme*'
> >
> > if [ -f $0.d/$HOSTNAME ] ; then
> > # Source host-specific list.
> > # Can define a new list or just override filter
> > . $0.d/$HOSTNAME
> > fi
> >
> > if [ -z $newlist ]; then
> > newlist=$(mydisks $filter* )
> > fi
> >
> > if [ -n "$newlist" ]; then
> > echo New disklist: $newlist
> > echo disklist=\"$newlist\" >> $LOGDIR/additional.var
> > fi
> > -8<--
> > Note the missing .sh extension.
> >
> > The 99-disklist.d/$HOSTNAME file is just:
> > -8<--
> > #!/bin/bash
> > filter='scsi-*'
> > # or
> > #newlist='sda'
> > -8<--
> >
> > HiH.
> >
> > Diego
> >
> > Il 31/01/2024 13:49, Thomas Lange ha scritto:
> >> Hi all,
> >>
> >> that is the example how to change the shell variable $disklist:
> >> https://fai-project.org/download/misc/99-disklist.sh
> >>
> >> Create the script class/99-disklist.sh in your config space
> >> (/s/rv/fai/config)
> >>
> >> These are the imprtant lines:
> >>
> >> if [ -n "$newlist" ]; then
> >> echo New disklist: $newlist
> >> echo disklist=\"$newlist\" >> $LOGDIR/additional.var
> >> fi
> >>
> >>
> >> This script writes the new valuespf disklist to
> >> $LOGDIR/additional.var. Then FAI will parse it and sets the new value
> >> for disklist before calling setup-storage.
> >>
> >> regards Thomas
> >
>
> --
> Diego Zuccato
> DIFA - Dip. di Fisica e Astronomia
> Servizi Informatici
> Alma Mater Studiorum - Università di Bologna
> V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
> tel.: +39 051 20 95786
Re: Problem with setup-storage
Hi Thomas, I will wait for the patch, because, yes, I have this bug. I do not know exactly what is going on, but I can send you, for example, some of my logs privately with privatebin. We can keep in touch on IRC about that. TYVM, Best Le mer. 7 févr. 2024 à 10:01, pa...@kish.by a écrit : > > Hi Thomas, > Also FAI 6.2 have problem with setup-storage when we use mdadm+lvm > I wrote later about it > > > > 7 февр. 2024 г., в 11:53, Thomas Lange написал(а): > > I'm preparing a patch and will upload a new FAI version if it's ready. > >
Re: Problem with setup-storage
Hi,
I don't know if this is really related, but as I also said on IRC,
`checkdisk` returns an empty string. All the functions depending on it
are also returning an empty string. I tried to modify it manually in
subroutines with the following content, but it did not help either:
```bash
checkdisk() {
# read lines with device name and size and check if device is a disk
# $1 can be a device name that will be ignored. Used for the device
# name of the USB stick if we boot from it
local igndev=('loop') #devices to ignore
igndev+=($1)
local isdisk
ls /dev | while read device blocks; do
isdisk=1
# old way of detecting disks: [ $(cat
/sys/block/$device/removable) -eq 1 ] && isdisk=0
[ $(stat -c %G /dev/$device) = "disk" ] || isdisk=0
# if set, ignore these devices
[[ "${igndev[@]}" =~ ${device}* ]] && isdisk=0
if [ $isdisk -eq 1 ]; then
echo "$device"
fi
done
}
```
here is the content of `fai-disk-info`:
```bash
#! /bin/bash
set_bootstick
all_disks_and_size | checkdisk $FAI_BOOTSTICK | once_only
```
That is really odd, because when I checked that modified version of
`checkdisk` locally, it works on my local machine. Moreover, I am able
to see disk devices in /dev when dracut spawn a shell terminal. I
don't know why it does not work in the dracut chroot mounted space
system; maybe a `sh` vs a `bash` issue although the subroutine calls
directly `/bin/bash` .. O_o ... All files in /usr/lib/fai/ are set in
755 mode, with owner and group root.
I am currently stuck with some machines unable to install because of
this issue...
Best regards,
Rémy
Le mar. 6 févr. 2024 à 15:59, Stephen Quinney via linux-fai
a écrit :
>
> Diese Nachricht wurde eingewickelt um DMARC-kompatibel zu sein. Die
> eigentliche Nachricht steht dadurch in einem Anhang.
>
> This message was wrapped to be DMARC compliant. The actual message
> text is therefore in an attachment.
>
>
> -- Forwarded message --
> From: Stephen Quinney
> To: linux-fai
> Cc:
> Bcc:
> Date: Tue, 6 Feb 2024 14:59:08 +
> Subject: Problem with setup-storage
> I'm using the packages from Debian unstable so I reported this to the
> Debian bug tracker first but then thought that it might be better done
> through the mailing list.
>
> We use the setup-storage utility in a standalone manner in our local
> installation and configuration scripts with the rest of FAI. Up until
> 6.2 this worked just fine but now it fails.
>
> This seems to be related to changes in the fai-disk-info script.
>
> With 6.0 I get:
>
> /usr/lib/fai/fai-disk-info
> sda
>
> and with 6.2 I now get:
>
> /usr/lib/fai/fai-disk-info
> bash: set_bootstick: command not found
> bash: once_only: command not found
> bash: all_disks_and_size: command not found
> bash: checkdisk: command not found
>
> At first I thought it was down to not having the fai-client package
> installed (it's not listed as a dependency for fai-setup-storage)
> since it provides the /usr/lib/fai/subroutines. However, installing
> that package hasn't helped.
>
> Any help would be appreciated.
>
> Regards,
>
> Stephen Quinney
Re: Define sda as the smallest disk
I know I could change directly the subroutine code from the
"set_disk_info" function, but if I can avoid doing this and doing
something better / cleaner...
Le mer. 31 janv. 2024 à 12:48, Rémy Dernat a écrit :
>
> Ok, Thanks Thomas.
>
> So, let's say I want to get a `$disklist` as an environment variable
> for the setup-storage step (or '-D'), and this disklist is :
> `$(newdisk_list=$(all_disks_by_size |sort -r | tr "\n" " ")`
>
> How could I change my disk_config file to use this new list ? By
> modifying `class/20-hw-detect.sh` (?); in the beginning of the
> scripts, it says that the outputs of `*.sh` scripts are ignored ...
>
> Thanks,
> Rémy
>
> Le mer. 31 janv. 2024 à 12:42, Thomas Lange a écrit :
> >
> > >>>>> On Wed, 31 Jan 2024 11:15:46 +0100, Rémy Dernat
> > >>>>> said:
> >
> > > Hi Thomas,
> > > That looks fantastic. Is there any examples to use these functions ?
> > No, the documentation is really bad. In FAI the disk list is created
> > by these two steps:
> >
> > set_bootstick
> > all_disks_and_size | checkdisk $FAI_BOOTSTICK | once_only
> >
> >
> >
> > From the sources (lib/subroutines)
> >
> > matchdisks() {
> > # matchdisks PATTERN PATTERN
> >
> > checkdisk() {
> > # read lines with device name and size and check if device is a disk
> > # $1 can be a device name that will be ignored. Used for the device
> > # name of the USB stick if we boot from it
> >
> > disks_by_id() {
> > # list all disks by ID and their link to the device name
> >
> > all_disks_and_size() {
> > # print a list of devices and their block size
> >
> >
> > # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > all_disks_by_size() {
> > all_disks_and_size | sort -nr -k2 | checkdisk $FAI_BOOTSTICK
> > }
> > # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > largestdisk() {
> > all_disks_and_size | sort -nr -k2 | checkdisk $FAI_BOOTSTICK | head -1
> > }
> > # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > smallestdisk() {
> > all_disks_and_size | sort -n -k2 | checkdisk $FAI_BOOTSTICK | head -1
> > }
> >
> > set_bootstick() {
> > # determine the device of the USB stick we boot from
> > # sets the variable FAI_BOOTSTICK
> >
> >
> >
> > --
> > regards Thomas
Re: Define sda as the smallest disk
Ok, Thanks Thomas.
So, let's say I want to get a `$disklist` as an environment variable
for the setup-storage step (or '-D'), and this disklist is :
`$(newdisk_list=$(all_disks_by_size |sort -r | tr "\n" " ")`
How could I change my disk_config file to use this new list ? By
modifying `class/20-hw-detect.sh` (?); in the beginning of the
scripts, it says that the outputs of `*.sh` scripts are ignored ...
Thanks,
Rémy
Le mer. 31 janv. 2024 à 12:42, Thomas Lange a écrit :
>
> >>>>> On Wed, 31 Jan 2024 11:15:46 +0100, Rémy Dernat
> >>>>> said:
>
> > Hi Thomas,
> > That looks fantastic. Is there any examples to use these functions ?
> No, the documentation is really bad. In FAI the disk list is created
> by these two steps:
>
> set_bootstick
> all_disks_and_size | checkdisk $FAI_BOOTSTICK | once_only
>
>
>
> From the sources (lib/subroutines)
>
> matchdisks() {
> # matchdisks PATTERN PATTERN
>
> checkdisk() {
> # read lines with device name and size and check if device is a disk
> # $1 can be a device name that will be ignored. Used for the device
> # name of the USB stick if we boot from it
>
> disks_by_id() {
> # list all disks by ID and their link to the device name
>
> all_disks_and_size() {
> # print a list of devices and their block size
>
>
> # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> all_disks_by_size() {
> all_disks_and_size | sort -nr -k2 | checkdisk $FAI_BOOTSTICK
> }
> # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> largestdisk() {
> all_disks_and_size | sort -nr -k2 | checkdisk $FAI_BOOTSTICK | head -1
> }
> # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> smallestdisk() {
> all_disks_and_size | sort -n -k2 | checkdisk $FAI_BOOTSTICK | head -1
> }
>
> set_bootstick() {
> # determine the device of the USB stick we boot from
> # sets the variable FAI_BOOTSTICK
>
>
>
> --
> regards Thomas
Re: Define sda as the smallest disk
Hi Thomas,
That looks fantastic. Is there any examples to use these functions ?
Best regards,
Rémy
Le lun. 29 janv. 2024 à 17:31, Thomas Lange a écrit :
>
> FAI 6.2 now provides some helper functions to create your own disklist:
>
> grepv_disks()
> grep_disks()
> notmatchdisks()
> matchdisks()
> smallestdisk()
> largestdisk()
> all_disks_by_size()
> all_disks_and_size()
> once_only()
> checkdisk()
> disks_by_id()
>
>
> Here's an older script how to change the disklist:
> https://fai-project.org/download/misc/99-disklist.sh
>
>
> >>>>> On Mon, 29 Jan 2024 17:13:39 +0100, Rémy Dernat
> >>>>> said:
>
>
> > So, I would like to define something like disk1 / sda is :
> > fdisk -l 2>/dev/null |awk '/Disk / {print $2,$3}' |sort -n -k2 |head -1
>
> > Then, this would be used by setup-storage.
>
> --
> regards Thomas
Define sda as the smallest disk
Hi,
I have a weird issue since Ubuntu 22.04.
It seems that disks are not in the right order; for example sda became
sdg, etc...
This is really annoying as my smallest disk should be used for system,
not the other disks. Normally, it was sda.
So, I would like to define something like disk1 / sda is :
fdisk -l 2>/dev/null |awk '/Disk / {print $2,$3}' |sort -n -k2 |head -1
Then, this would be used by setup-storage.
Any idea on how to do this ?
Best regards,
Re: Disk config with GRUB/EFI, RAID + LVM
BTW, I am wondering what would be the corresponding BTRFS config. Does something like this work as expected or not (mixing / and /tmp this time) : ``` disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid primary - 2G -- primary - 70% -- primary - 0- -- disk_config disk2 sameas:disk1 disk_config btrfs fstabkey:uuid btrfs raid1 /boot/efi disk1.1,disk2.1 auto,ssd,lazytime,rw,nosuid,nodev,exec,subvol=@boot btrfs raid1 / disk1.4,disk2.4 subvol=@root,defaults,noatime,compress=zstd btrfs raid1 /home disk1.3,disk2.3 subvol=@home,defaults,noatime,compress=zstd ``` ?? Le mar. 5 déc. 2023 à 13:15, Rémy Dernat a écrit : > > Hi, > > After digging some time around this issue, I am finally able to boot > the RAID1 partition with EFI boot enabled on it (using this example: > https://fai-project.org/download/misc/disk_config/lvm-on-luks-on-raid). > > Here is my final disk_config file: > > ```# example of new config file for setup-storage > # > # > > disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid > primary - 2G - - > primary - 0- - - > > disk_config disk2 sameas:disk1 > > disk_config raid > raid1 /boot/efi disk1.1,disk2.1 vfat rw,noatime > mdcreateopts="--verbose --metadata=0.90" > raid1 - disk1.2,disk2.2 - - > > disk_config lvm > vg vg_system md1 > vg_system-swap swap 16G swap sw > vg_system-root / 60G-200G ext4 rw,noatime > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > > ``` > > Best regards, > Rémy > > Le ven. 1 déc. 2023 à 12:11, Rémy Dernat a écrit : > > > > Hi Andrew, > > > > Thanks for suggestions. > > > > Now, my file "disk_config/GRUB_EFI_RAID1" is [*]: > > > > ``` > > # example of new config file for setup-storage > > # > > # > > > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 > > primary - 2G - - > > primary /boot/efi 1G vfat rw createopts="-F32" > > primary - 2G - - > > primary - 0- - - > > > > #disk_config disk2 sameas:disk1 > > disk_config disk2 disklabel:gpt fstabkey:uuid > > primary - 2G - - > > primary /boot/efi2 1G vfat rw createopts="-F32" > > primary - 2G - - > > primary - 0- - - > > > > disk_config raid fstabkey:uuid > > raid1 /boot disk1.1,disk2.1 ext4 rw,noatime mdcreateopts="--verbose > > --metadata=0.90" > > raid1 swap disk1.3,disk2.3 swap sw > > raid1 - disk1.4,disk2.4 - - > > > > disk_config lvm fstabkey:uuid > > vg vg_system md2 > > vg_system-root / 60G-200G ext4 rw,noatime > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > > ``` > > > > => Even if the installation succeeded, grub does not seem to work. > > > > Anyway, I am wondering how you set this in the disk_config file (syntax ?): > > > > > I like to put a 100MB partition at the end of disks in a RAID pair to > > > allow > > > for slightly differing size of a replacement disk. Although currently it > > > seems that any new disk we get is many many GB bigger than the old failed > > > one. > > > > Thanks again, > > > > Best regards, > > Rémy > > > > > > > > [*] there was an error for /boot partition which was not formatted the > > right way (vfat does not allow symbolic links). vfat should be used > > for /boot/EFI only. > > > > Le ven. 1 déc. 2023 à 05:43, Andrew Ruthven a écrit : > > > > > > Nice, good work. > > > > > > A couple of suggestions... > > > > > > I'd put swap on RAID1, if you have a disk go offline while the system is > > > running, you've lost anything that had been swapped out to the swap on the > > > failed disk. > > > > > > I like to put a 100MB partition at the end of disks in a RAID pair to > > > allow > > > for slightly differing size of a replacement disk. Although currently it > > > seems that any new disk we get is many many GB bigger than the old failed > > > one. > > > > > > I've found that some servers allow the EFI partition to be RAID1, the > > > current servers we're using support that. So no need to copy files around. > > > > > > Cheers, > > > Andrew > > > > > > On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote: > > > > > > > Ok, F
Re: Disk config with GRUB/EFI, RAID + LVM
Hi, After digging some time around this issue, I am finally able to boot the RAID1 partition with EFI boot enabled on it (using this example: https://fai-project.org/download/misc/disk_config/lvm-on-luks-on-raid). Here is my final disk_config file: ```# example of new config file for setup-storage # # disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid primary - 2G - - primary - 0- - - disk_config disk2 sameas:disk1 disk_config raid raid1 /boot/efi disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose --metadata=0.90" raid1 - disk1.2,disk2.2 - - disk_config lvm vg vg_system md1 vg_system-swap swap 16G swap sw vg_system-root / 60G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` Best regards, Rémy Le ven. 1 déc. 2023 à 12:11, Rémy Dernat a écrit : > > Hi Andrew, > > Thanks for suggestions. > > Now, my file "disk_config/GRUB_EFI_RAID1" is [*]: > > ``` > # example of new config file for setup-storage > # > # > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 > primary - 2G - - > primary /boot/efi 1G vfat rw createopts="-F32" > primary - 2G - - > primary - 0- - - > > #disk_config disk2 sameas:disk1 > disk_config disk2 disklabel:gpt fstabkey:uuid > primary - 2G - - > primary /boot/efi2 1G vfat rw createopts="-F32" > primary - 2G - - > primary - 0- - - > > disk_config raid fstabkey:uuid > raid1 /boot disk1.1,disk2.1 ext4 rw,noatime mdcreateopts="--verbose > --metadata=0.90" > raid1 swap disk1.3,disk2.3 swap sw > raid1 - disk1.4,disk2.4 - - > > disk_config lvm fstabkey:uuid > vg vg_system md2 > vg_system-root / 60G-200G ext4 rw,noatime > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > ``` > > => Even if the installation succeeded, grub does not seem to work. > > Anyway, I am wondering how you set this in the disk_config file (syntax ?): > > > I like to put a 100MB partition at the end of disks in a RAID pair to allow > > for slightly differing size of a replacement disk. Although currently it > > seems that any new disk we get is many many GB bigger than the old failed > > one. > > Thanks again, > > Best regards, > Rémy > > > > [*] there was an error for /boot partition which was not formatted the > right way (vfat does not allow symbolic links). vfat should be used > for /boot/EFI only. > > Le ven. 1 déc. 2023 à 05:43, Andrew Ruthven a écrit : > > > > Nice, good work. > > > > A couple of suggestions... > > > > I'd put swap on RAID1, if you have a disk go offline while the system is > > running, you've lost anything that had been swapped out to the swap on the > > failed disk. > > > > I like to put a 100MB partition at the end of disks in a RAID pair to allow > > for slightly differing size of a replacement disk. Although currently it > > seems that any new disk we get is many many GB bigger than the old failed > > one. > > > > I've found that some servers allow the EFI partition to be RAID1, the > > current servers we're using support that. So no need to copy files around. > > > > Cheers, > > Andrew > > > > On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote: > > > > > Ok, FYI, I think I have a working configuration with : > > > > > > ``` > > > # example of new config file for setup-storage > > > # > > > # > > options> > > > > > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 > > > primary - 2G - - > > > primary /boot/efi 1G vfat rw createopts="-F32" > > > primary swap 2G swap sw,pri=1 > > > primary - 0- - - > > > > > > #disk_config disk2 sameas:disk1 > > > disk_config disk2 disklabel:gpt fstabkey:uuid > > > primary - 2G - - > > > primary /boot/efi2 1G vfat rw createopts="-F32" > > > primary swap 2G swap sw,pri=1 > > > primary - 0- - - > > > > > > disk_config raid fstabkey:uuid > > > #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose > > > --metadata=1.0" createopts="-L boot" > > > raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose > > > --metadata=0.90" > > > raid1 - disk1.4,disk2.4 - - > > > > > > disk_config lvm fstabkey:uuid > > > vg vg_system md1 > > > vg_system-root / 60G-200G ext4 rw,noatime > >
Re: Disk config with GRUB/EFI, RAID + LVM
Hi Andrew, Thanks for suggestions. Now, my file "disk_config/GRUB_EFI_RAID1" is [*]: ``` # example of new config file for setup-storage # # disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 primary - 2G - - primary /boot/efi 1G vfat rw createopts="-F32" primary - 2G - - primary - 0- - - #disk_config disk2 sameas:disk1 disk_config disk2 disklabel:gpt fstabkey:uuid primary - 2G - - primary /boot/efi2 1G vfat rw createopts="-F32" primary - 2G - - primary - 0- - - disk_config raid fstabkey:uuid raid1 /boot disk1.1,disk2.1 ext4 rw,noatime mdcreateopts="--verbose --metadata=0.90" raid1 swap disk1.3,disk2.3 swap sw raid1 - disk1.4,disk2.4 - - disk_config lvm fstabkey:uuid vg vg_system md2 vg_system-root / 60G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev ##vg_system-scratch /scratch 30%- ext2 rw,noatime vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` => Even if the installation succeeded, grub does not seem to work. Anyway, I am wondering how you set this in the disk_config file (syntax ?): > I like to put a 100MB partition at the end of disks in a RAID pair to allow > for slightly differing size of a replacement disk. Although currently it > seems that any new disk we get is many many GB bigger than the old failed > one. Thanks again, Best regards, Rémy [*] there was an error for /boot partition which was not formatted the right way (vfat does not allow symbolic links). vfat should be used for /boot/EFI only. Le ven. 1 déc. 2023 à 05:43, Andrew Ruthven a écrit : > > Nice, good work. > > A couple of suggestions... > > I'd put swap on RAID1, if you have a disk go offline while the system is > running, you've lost anything that had been swapped out to the swap on the > failed disk. > > I like to put a 100MB partition at the end of disks in a RAID pair to allow > for slightly differing size of a replacement disk. Although currently it > seems that any new disk we get is many many GB bigger than the old failed > one. > > I've found that some servers allow the EFI partition to be RAID1, the > current servers we're using support that. So no need to copy files around. > > Cheers, > Andrew > > On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote: > > > Ok, FYI, I think I have a working configuration with : > > > > ``` > > # example of new config file for setup-storage > > # > > # > options> > > > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 > > primary - 2G - - > > primary /boot/efi 1G vfat rw createopts="-F32" > > primary swap 2G swap sw,pri=1 > > primary - 0- - - > > > > #disk_config disk2 sameas:disk1 > > disk_config disk2 disklabel:gpt fstabkey:uuid > > primary - 2G - - > > primary /boot/efi2 1G vfat rw createopts="-F32" > > primary swap 2G swap sw,pri=1 > > primary - 0- - - > > > > disk_config raid fstabkey:uuid > > #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose > > --metadata=1.0" createopts="-L boot" > > raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose > > --metadata=0.90" > > raid1 - disk1.4,disk2.4 - - > > > > disk_config lvm fstabkey:uuid > > vg vg_system md1 > > vg_system-root / 60G-200G ext4 rw,noatime > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > > ``` > > > > My previous message was crystal clear. md0 did not have any formatted > > filesystem on it. So I added `vfat` option on /boot. > > > > Best regards, > > Rémy > > > > > > Le jeu. 30 nov. 2023 à 16:53, Rémy Dernat > <[remy...@gmail.com](mailto:remy...@gmail.com)> a écrit : > > > > > > > > What is strange here, is that md1 is found, as well as md0, but > > > contrary to md0, md1 have an UUID ... > > > > > > However, UUID seems to be retrieve by blkid from /dev/disk/by-id, > > > where both md array have their own mdadm UUID, but only md1 have an > > > UUID from LVM (with pvdisplay / pvs ) ... > > > > > > > > > Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat > <[remy...@gmail.com](mailto:remy...@gmail.com)> a écrit : > > > > > > > > > > > Hi Andrew, > > > > > > > > Indeed, this removed the syntax issue. However, I have a new error > now: > > > > > > > > ``` > > > > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2 > > > > Command had
Re: Disk config with GRUB/EFI, RAID + LVM
Ok, FYI, I think I have a working configuration with : ``` # example of new config file for setup-storage # # disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 primary - 2G - - primary /boot/efi 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - #disk_config disk2 sameas:disk1 disk_config disk2 disklabel:gpt fstabkey:uuid primary - 2G - - primary /boot/efi2 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - disk_config raid fstabkey:uuid #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose --metadata=1.0" createopts="-L boot" raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose --metadata=0.90" raid1 - disk1.4,disk2.4 - - disk_config lvm fstabkey:uuid vg vg_system md1 vg_system-root / 60G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev ##vg_system-scratch /scratch 30%- ext2 rw,noatime vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` My previous message was crystal clear. md0 did not have any formatted filesystem on it. So I added `vfat` option on /boot. Best regards, Rémy Le jeu. 30 nov. 2023 à 16:53, Rémy Dernat a écrit : > > What is strange here, is that md1 is found, as well as md0, but > contrary to md0, md1 have an UUID ... > > However, UUID seems to be retrieve by blkid from /dev/disk/by-id, > where both md array have their own mdadm UUID, but only md1 have an > UUID from LVM (with pvdisplay / pvs ) ... > > > Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat a écrit : > > > > Hi Andrew, > > > > Indeed, this removed the syntax issue. However, I have a new error now: > > > > ``` > > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2 > > Command had non-zero exit code > > ``` > > Indeed, blkid is not able to found md0 UUID. > > > > I tried to modify the disk_config file to avoid this error, but I am > > still stuck with it. Here is what look like my disk_config file now: > > > > ``` > > # example of new config file for setup-storage > > # > > # > > > > disk_config disk1 > > primary - 2G - - > > primary /boot/efi 1G vfat rw createopts="-F32" > > primary swap 2G swap sw,pri=1 > > primary - 0- - - > > > > #disk_config disk2 sameas:disk1 > > disk_config disk2 disklabel:gpt fstabkey:uuid > > primary - 2G - - > > primary /boot/efi2 1G vfat rw createopts="-F32" > > primary swap 2G swap sw,pri=1 > > primary - 0- - - > > > > disk_config raid fstabkey:uuid > > raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose > > --metadata=0.90" createopts="-L boot" > > raid1 - disk1.4,disk2.4 - - > > > > disk_config lvm fstabkey:uuid > > vg vg_system md1 > > vg_system-root / 60G-200G ext4 rw,noatime > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > > ``` > > > > I also tried to change the raid metadata to 1.0, but that did not > > change anything. I wrote back ` fstabkey:uuid` behind disk_config raid > > because it does not fail with syntax error. > > > > Thanks for help > > > > Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven a écrit : > > > > > > Hey, > > > > > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote: > > > > Hi awesome Fai guys, > > > > > > > > I have a weird syntax issue with the following disk_config: > > > > > > > [snip] > > > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1 > > > > > > This line is your problem, remove everything after the word raid. So it > > > should just be: > > > > > > disk_config raid > > > > > > Cheers, > > > Andrew > > > -- > > > Andrew Ruthven, Wellington, New Zealand > > > and...@etc.gen.nz | > > > Catalyst Cloud: | This space intentionally left blank > > > https://catalystcloud.nz | > > >
Re: Disk config with GRUB/EFI, RAID + LVM
What is strange here, is that md1 is found, as well as md0, but contrary to md0, md1 have an UUID ... However, UUID seems to be retrieve by blkid from /dev/disk/by-id, where both md array have their own mdadm UUID, but only md1 have an UUID from LVM (with pvdisplay / pvs ) ... Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat a écrit : > > Hi Andrew, > > Indeed, this removed the syntax issue. However, I have a new error now: > > ``` > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2 > Command had non-zero exit code > ``` > Indeed, blkid is not able to found md0 UUID. > > I tried to modify the disk_config file to avoid this error, but I am > still stuck with it. Here is what look like my disk_config file now: > > ``` > # example of new config file for setup-storage > # > # > > disk_config disk1 > primary - 2G - - > primary /boot/efi 1G vfat rw createopts="-F32" > primary swap 2G swap sw,pri=1 > primary - 0- - - > > #disk_config disk2 sameas:disk1 > disk_config disk2 disklabel:gpt fstabkey:uuid > primary - 2G - - > primary /boot/efi2 1G vfat rw createopts="-F32" > primary swap 2G swap sw,pri=1 > primary - 0- - - > > disk_config raid fstabkey:uuid > raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose > --metadata=0.90" createopts="-L boot" > raid1 - disk1.4,disk2.4 - - > > disk_config lvm fstabkey:uuid > vg vg_system md1 > vg_system-root / 60G-200G ext4 rw,noatime > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > ``` > > I also tried to change the raid metadata to 1.0, but that did not > change anything. I wrote back ` fstabkey:uuid` behind disk_config raid > because it does not fail with syntax error. > > Thanks for help > > Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven a écrit : > > > > Hey, > > > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote: > > > Hi awesome Fai guys, > > > > > > I have a weird syntax issue with the following disk_config: > > > > > [snip] > > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1 > > > > This line is your problem, remove everything after the word raid. So it > > should just be: > > > > disk_config raid > > > > Cheers, > > Andrew > > -- > > Andrew Ruthven, Wellington, New Zealand > > and...@etc.gen.nz | > > Catalyst Cloud: | This space intentionally left blank > > https://catalystcloud.nz | > >
Re: Disk config with GRUB/EFI, RAID + LVM
Hi Andrew, Indeed, this removed the syntax issue. However, I have a new error now: ``` /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2 Command had non-zero exit code ``` Indeed, blkid is not able to found md0 UUID. I tried to modify the disk_config file to avoid this error, but I am still stuck with it. Here is what look like my disk_config file now: ``` # example of new config file for setup-storage # # disk_config disk1 primary - 2G - - primary /boot/efi 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - #disk_config disk2 sameas:disk1 disk_config disk2 disklabel:gpt fstabkey:uuid primary - 2G - - primary /boot/efi2 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - disk_config raid fstabkey:uuid raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose --metadata=0.90" createopts="-L boot" raid1 - disk1.4,disk2.4 - - disk_config lvm fstabkey:uuid vg vg_system md1 vg_system-root / 60G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev ##vg_system-scratch /scratch 30%- ext2 rw,noatime vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` I also tried to change the raid metadata to 1.0, but that did not change anything. I wrote back ` fstabkey:uuid` behind disk_config raid because it does not fail with syntax error. Thanks for help Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven a écrit : > > Hey, > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote: > > Hi awesome Fai guys, > > > > I have a weird syntax issue with the following disk_config: > > > [snip] > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1 > > This line is your problem, remove everything after the word raid. So it > should just be: > > disk_config raid > > Cheers, > Andrew > -- > Andrew Ruthven, Wellington, New Zealand > and...@etc.gen.nz | > Catalyst Cloud: | This space intentionally left blank > https://catalystcloud.nz | >
Disk config with GRUB/EFI, RAID + LVM
Hi awesome Fai guys, I have a weird syntax issue with the following disk_config: ``` # example of new config file for setup-storage # # disk_config disk1 disklabel:gpt fstabkey:uuid primary - 2G - - primary /boot/efi 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - #disk_config disk2 sameas:disk1 disk_config disk2 disklabel:gpt fstabkey:uuid primary - 2G - - primary /boot/efi2 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - disk_config raid disklabel:gpt fstabkey:uuid bootable:1 raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--metadata=0.90" raid1 - disk1.4,disk2.4 - - disk_config lvm fstabkey:uuid vg vg_system md1 vg_system-root / 60G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev ##vg_system-scratch /scratch 30%- ext2 rw,noatime vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` Resulting in the error: ``` Starting setup-storage 3.0 Using config file: GRUB_EFI_RAID1 ERROR (line 18): Invalid line: Was expecting '\n' but found "disk_config raid disklabel:gpt fstabkey:uuid bootable:1" instead Syntax error ``` Can be checked again with `setup-storage -s -f GRUB_EFI_RAID1`. I also tried to display hidden characters with 'cat -A ...' but I did not see anything strange. Any idea would be useful ! Thanks :)
Re: FAI + SaltStack anybody?
Hi, I did not read this whole threads, but yes, here we are currently managing a FAI server through SaltStack. It configures pxelinux files and my DHCP server. FAI rootfs installs the SaltStack repository with a script class, and my SaltStack server auto-accept keys from known hostnames through a SaltStack reactor or orchestrator, depending on the machine. When the key is accepted, a highstate is deployed to finish the install when the orchestrator is launched. All my machines configurations are stored on the SaltStack pillars. Those pillars contains the SaltStack minion's name, the hostname, the mac address, the IP address, the boot state and some other useful informations. When a machine is finally installed, the orchestrator change the value "boot" in my pillar corresponding to the machine to "OS" instead of "install" and the value is deployed to the tftp FAI server to changed the pxelinux file like fai-chboot would have done with states tftp and dhcp. When a machine needs to be reinstalled, orchestrator starts by changing its boot state, deploys the tftp state, reboot the machine and removes the key. Then the machine is installed; there is a big timeout in order to wait for the reinstall. Then the machine tries to reconnect to the machine "salt", key is auto-accepted, highstate is deployed, etc.. Problem with the orchestrator is that it is only one machine by one machine, contrary to a fully reactor system. Hope it helps, Best regards, Rémy Le mer. 11 oct. 2023, 13:33, Markus Köberl via linux-fai < linux-fai@uni-koeln.de> a écrit : > Diese Nachricht wurde eingewickelt um DMARC-kompatibel zu sein. Die > eigentliche Nachricht steht dadurch in einem Anhang. > > This message was wrapped to be DMARC compliant. The actual message > text is therefore in an attachment. > > > -- Forwarded message -- > From: "Markus Köberl" > To: linux-fai@uni-koeln.de > Cc: > Bcc: > Date: Wed, 11 Oct 2023 13:32:46 +0200 > Subject: Re: FAI + SaltStack anybody? > On Thursday, 5 October 2023 14:59:40 CEST Diego Zuccato wrote: > > Hello all. > > > > Does someone use FAI to install the base system that will be managed by > > Salt? > > I'm trying to integrate 'em but there's still something that doesn't > > "click"... > > > > My current idea is to use Salt to orchestrate the install, but maybe > > it's better left to FAI? How can I "pass around" minion key so I don't > > have to manually re-approve the new key every time? > > The ideal scenario would be: target generates its keypair, sends the > > pubkey to FAI that "certifies" it's from the system being installed and > > passes it to Salt. Should I write a custom fai-monitor (that would be > > needed anyway to disable netboot once system is reinstalled)? > > > > TIA. > > My solution at the moment is non-interactive. > In classes I have a script which asks for username and password for the > salt > api to save a cookie which is valid for a 30min. > Later during the fai installation a script uses the cookie to get the salt > key > via the salt api. After the first boot salt is doing the rest... > > Instead of using the non-interactive approach I guess you could also > provide > the cookie base64 encoded via boot parameter or dhcp. > > > regards > Markus > -- > Markus Koeberl > Graz University of Technology > Signal Processing and Speech Communication Laboratory > E-mail: markus.koeb...@tugraz.at
Re: Managing LVM + RAID install on a machine with EFI
Thomas, Ok, this is helpful. I found this blog : https://terrydactyl10.wordpress.com/2015/11/04/centos-7-with-mdadm-software-raid-and-uefi-boot/ So here is my resulting disk_config file now : # example of new config file for setup-storage # # disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 primary - 500 - - primary /boot/efi 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - #disk_config disk2 sameas:disk1 disk_config disk2 disklabel:gpt fstabkey:uuid primary - 500 - - primary /boot/efi2 1G vfat rw createopts="-F32" primary swap 2G swap sw,pri=1 primary - 0- - - disk_config raid disklabel:gpt fstabkey:uuid raid1 /boot disk1.1,disk2.1 ext4 rw,noatime,errors=remount-ro raid1 - disk1.4,disk2.4 - - disk_config lvm fstabkey:uuid vg vg_system md1 vg_system-root / 40G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext4 rw,noatime,nosuid,nodev vg_system-scratch /scratch 30%- ext2 rw,noatime ##vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev Maybe I can try it now. Best regards, Le mer. 10 mars 2021 à 10:24, Thomas Lange a écrit : > > > >>>>> On Wed, 10 Mar 2021 10:00:30 +0100, Rémy Dernat > >>>>> said: > > > disk config disk1 > First, you need a GPT disk label for UEFI. > > > disk_config raid fstabkey:uuid > > radi1 /boot/efi disk1.1,disk2.1 vfat rw createopts="-F32" > > I'm not sure if raid for /boot/efi will work, since the UEFI bios must > read this partition. UEFI does not have any knowledge about linux raid. > > Search the internet for UEFI /boot/efi and raid setup. IIRC people do > not use raid on /boot/efi but they have a secondary copy which must be > synced if you change files on the original /boot/efi partition. And > you need two entries in your UEFI bios for both /boot/efi copies. > > -- > viele Grüße Thomas
Re: Managing LVM + RAID install on a machine with EFI
Hi Thomas, Ok, here is the configuration I was thinking about : disk config disk1 primary - 2G - - primary swap 2G swap sw,pri=1 primary - 0- - - disk_config disk2 sameas:disk1 disk_config raid fstabkey:uuid radi1 /boot/efi disk1.1,disk2.1 vfat rw createopts="-F32" raid1 - disk1.3,disk2.3 - - disk_config lvm fstabkey:uuid vg vg_system md1 vg_system-root / 40G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext4 rw,noatime,nosuid,nodev vg_system-scratch /scratch 30%- ext2 rw,noatime ##vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev As you can see I changed my boot partition for EFI. Do you think it is Ok now ? Thanks, Le mer. 10 mars 2021 à 09:54, Thomas Lange a écrit : > > Your disk config does not have a /boot/efi partition. So this can't be a > UEFI machine. > > P.S.: Your email looks very weird in my email client. Maybe because of > the ``` in html. Can you please disable html mails? > > >>>>> On Wed, 10 Mar 2021 09:43:27 +0100, Rémy Dernat > >>>>> said: > > > Hi,I currently have that > kind of disk configuration for my UEFI machines > :```disk_config disk1 disklabel:msdos > bootable:1 fstabkey:uuidprimary / 40G-200Gext4 > rw,noatime,errors=remount-rological /tmp20G-50G ext4 > rw,noatime,nosuid,nodev createo-- > viele Grüße Thomas
Managing LVM + RAID install on a machine with EFI
Hi, I currently have that kind of disk configuration for my UEFI machines : ``` disk_config disk1 disklabel:msdos bootable:1 fstabkey:uuid primary / 40G-200Gext4 rw,noatime,errors=remount-ro logical /tmp20G-50G ext4 rw,noatime,nosuid,nodev createopts="-L tmp -m 0" tuneopts="-c 0 -i 0" logical /scratch30%-ext2 rw,noatime,nosuid,nodev createopts="-L scratch -m 1" tuneopts="-c 0 -i 0" logical swap200-1G swap sw ## logical /home 30%- ext4 rw,noatime,nosuid,nodev createopts="-L home -m 1" tuneopts="-c 0 -i 0" ``` I would like to change it to softraid + LVM. After reading setup-storage man page, I think it should seems like that : ``` disk config disk1 primary - 2G - - primary swap 2G swap sw,pri=1 primary - 0- - - disk_config disk2 sameas:disk1 disk_config raid fstabkey:uuid raid1 /boot disk1.1,disk2.1 ext4 rw,noatime,errors=remount-ro raid1 - disk1.3,disk2.3 - - disk_config lvm fstabkey:uuid vg vg_system md1 vg_system-root / 40G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext4 rw,noatime,nosuid,nodev vg_system-scratch /scratch 30%- ext2 rw,noatime ##vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` I could also say that I am not sure about what would be the first disk, or even if it is a SSD (NVMe, PCIe, SATA...) or HDD... Can anyone can take a look at this disk configuration and confirm if it is look ok to him ? Thanks, Best regards, Rémy.
Re: AW: Install Fedora CoreOS using FAI
Hi I have a couple of questions. - Do you deploy this system to operate a k8s cluster ? - If so, I imagine you disable the swap (?). But why are you deploying this stack over bare-metal, with no VM ? - Why not using kickstart, which is the redhat standard tftp system ? (I mean, I am a big of FAI and its powerful class system, but anyway that choice seems a bit odd to me), - btw, why not using k3s, rancherOS or whatever other k8s ready distro ? I have my opinion about these last questions, but I would also like to hear yours. Finally, I also had to deal with httpd/tftp issues. It depends on your booting system (I am using syslinux 6.0.4 + lighttpd for UEFI booting systems while I am using basic tftp for legacy boot clients). Best regards, Rémy Le lun. 20 avr. 2020 à 08:41, Andrew Ruthven a écrit : > Hey, > > Nice. I had considered using HTTP, but didn't want to deviate from the > "recommended" approach. Well. Up until I decided to completely the > approach. ;) > > That is also a much smaller initrd! > > The main reason I changed was to be able to configure the disk layout, > the much faster build was a very pleasant side effect. > > Cheers, > Andrew > > On Mon, 2020-04-20 at 06:33 +, Schulz, Reiner wrote: > > For our OpenShift we use http to load the PXE file: > > > > /var/lib/tftpboot/gpxelinux.0 > > > > dhcpd.conf: > > filename "gpxelinux.0"; > > > > nginx: > > $ ls -lh /usr/share/nginx/html/rhcos-4.2.18-x86_64-* > > 63M 11. Feb 17:04 /usr/share/nginx/html/rhcos-4.2.18-x86_64- > > installer-initramfs.img > > 7,8M 11. Feb 17:04 /usr/share/nginx/html/rhcos-4.2.18-x86_64- > > installer-kernel > > 703M 11. Feb 17:04 /usr/share/nginx/html/rhcos-4.2.18-x86_64-metal- > > bios.raw.gz > > > > Reiner Schulz > > -- > Andrew Ruthven, Wellington, New Zealand > and...@etc.gen.nz | linux.conf.au 2021, Canberra, AU > Catalyst Cloud:| http://lca2021.linux.org.au/ >https://catalystcloud.nz| > >
Re: Some modules not found, dropping to dracut shell
Oh I see, Ok... That explains why it worked before and then, why it does not work anymore. Thanks, Le lun. 4 févr. 2019 à 11:27, Thomas Lange a écrit : > >>>>> On Mon, 4 Feb 2019 10:26:02 +0100, Rémy Dernat > said: > > > I have some difficulties to setup a working nfsroot environement; > The initrd image does not seem to contain all the necessary modules : > > > chroot /srv/fai/nfsroot > > > find /lib/modules/4.9.0-8-amd64/ -name "*bnx2*" > > /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2i > > /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2i/bnx2i.ko > > /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc > > /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc/bnx2fc.ko > > > /lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2.ko > > > /lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x > > > > /lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko > > exit > > > lib/firmware/bnx2 > > lib/firmware/bnx2/bnx2-mips-06-6.2.3.fw > > lib/firmware/bnx2/bnx2-mips-09-6.2.1b.fw > > lib/firmware/bnx2/bnx2-rv2p-06-6.0.15.fw > > > > > # package list for creating the NFSROOT > > > # some network cards needs firmware > > PACKAGES install-norec NONFREE > > firmware-bnx2 firmware-bnx2x firmware-realtek > > firmware-misc-nonfree > > > > You could see that there are both linux-image-* and firmware-bnx2*. > Yes, but the firmware packages are not used for your nfsroot. > In FAI 5.5 there was the new option -N added to fai-make-nfsroot. You > have to use this option, in order to get the packages listed in > /etc/fai/NFSROOT in the section NONFREE added to the nfsroot. See man > fai-make-nfsroot(8). > > -- > regards Thomas >
Some modules not found, dropping to dracut shell
Hi, I have some difficulties to setup a working nfsroot environement; The initrd image does not seem to contain all the necessary modules : chroot /srv/fai/nfsroot find /lib/modules/4.9.0-8-amd64/ -name "*bnx2*" /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2i /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2i/bnx2i.ko /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc /lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc/bnx2fc.ko /lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2.ko /lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x /lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko exit With lsinitramfs, from the fai server : # not working initrd lsinitramfs /srv/tftp/fai/initrd.img-4.9.0-8-amd64 |grep -E "bnx2|rtl_nic" lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2.ko lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc/bnx2fc.ko While, with a working initrd : lsinitramfs /var/www/html/fai/initrd.img-4.9.0-8-amd64 |grep -E "bnx2|rtl_nic" lib/firmware/bnx2 lib/firmware/bnx2/bnx2-mips-06-6.2.3.fw lib/firmware/bnx2/bnx2-mips-09-6.2.1b.fw lib/firmware/bnx2/bnx2-rv2p-06-6.0.15.fw lib/firmware/bnx2/bnx2-rv2p-09-6.0.17.fw lib/firmware/bnx2/bnx2-rv2p-09ax-6.0.17.fw lib/firmware/bnx2x lib/firmware/bnx2x/bnx2x-e1-7.13.1.0.fw lib/firmware/bnx2x/bnx2x-e1h-7.13.1.0.fw lib/firmware/bnx2x/bnx2x-e2-7.13.1.0.fw lib/firmware/rtl_nic lib/firmware/rtl_nic/rtl8105e-1.fw lib/firmware/rtl_nic/rtl8106e-1.fw lib/firmware/rtl_nic/rtl8106e-2.fw lib/firmware/rtl_nic/rtl8107e-1.fw lib/firmware/rtl_nic/rtl8107e-2.fw lib/firmware/rtl_nic/rtl8168d-1.fw lib/firmware/rtl_nic/rtl8168d-2.fw lib/firmware/rtl_nic/rtl8168e-1.fw lib/firmware/rtl_nic/rtl8168e-2.fw lib/firmware/rtl_nic/rtl8168e-3.fw lib/firmware/rtl_nic/rtl8168f-1.fw lib/firmware/rtl_nic/rtl8168f-2.fw lib/firmware/rtl_nic/rtl8168g-2.fw lib/firmware/rtl_nic/rtl8168g-3.fw lib/firmware/rtl_nic/rtl8168h-1.fw lib/firmware/rtl_nic/rtl8168h-2.fw lib/firmware/rtl_nic/rtl8402-1.fw lib/firmware/rtl_nic/rtl8411-1.fw lib/firmware/rtl_nic/rtl8411-2.fw lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2.ko lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x lib/modules/4.9.0-8-amd64/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc lib/modules/4.9.0-8-amd64/kernel/drivers/scsi/bnx2fc/bnx2fc.ko I do not understand why the first one does not contain all the necessary modules. My NFSROOT is like this : # package list for creating the NFSROOT PACKAGES install-norec FULL nfs-common fai-nfsroot rpcbind rsync lftp lshw procinfo dump reiserfsprogs xfsprogs xfsdump btrfs-progs dosfstools hwinfo hdparm smartmontools subversion rdate emacs25-nox numactl dnsutils netcat-traditional nmap pxelinux syslinux-common # in jessie we need both ca-certificates # for get-config-dir-https and apt-transport-https usbutils pciutils ssh netselect mdadm #git # git consumes a lot of disk space on the FAI CD PACKAGES install-norec # dracut replaces live-boot and initramfs-tools dracut live-boot- initramfs-tools- dracut-config-generic dracut-network sysvinit-core systemd-sysv- wget- curl less moreutils ntpdate lvm2 psmisc dialog console-common kbd xz-utils # some network cards needs firmware PACKAGES install-norec NONFREE firmware-bnx2 firmware-bnx2x firmware-realtek firmware-misc-nonfree # you should not edit the lines below # architecture dependend list of packages that are installed PACKAGES install-norec I386 grub-pc linux-image-686-pae PACKAGES install-norec AMD64 grub-pc linux-image-amd64 #linux-image-amd64/stretch-backports # if you want to use a newer kernel PACKAGES install-norec ARM64 grub-efi-arm64 linux-image-arm64 PACKAGES install-norec ARMHF grub-efi-arm linux-image-armmp PACKAGES install-norec DEBIAN_9 apt-transport-https PACKAGES install-norec DEBIAN_10 fdisk PACKAGES install-norec DEBIAN_11 fdisk You could see that there are both linux-image-* and firmware-bnx2*. How should I change my [ NFSROOT / FAI config ] to include the missing firmwares ? Best regards, Rémy.
Re: Network interface names
Hi Steffen,
That is weird; this should work even for pure debian.
However, reading my message, I saw that some parts of my scripts are
useless. Maybe you can try the following instead (just replacing $NIC1 by
$iface which is not used) :
```
#! /bin/bash
iface=`ip -o -f inet addr show |awk '$2 !~ "lo|docker" {print $2;exit;}'`
mac=`ip -o -f link addr |awk -v iface=$iface '{ if ( $2 == iface":" )
{print;} }' |cut -d"/" -f2|awk '{print $2}'`
fields="ID_NET_NAME_FROM_DATABASE ID_NET_NAME_ONBOARD ID_NET_NAME_SLOT
ID_NET_NAME_PATH"
for field in $fields; do
name=$(udevadm info /sys/class/net/$iface | sed -rn "s/^E:
$field=(.+)/\1/p")
if [[ $name ]]; then
newnicname=$name
break
fi
done
fcopy /etc/netplan/01-netcfg.yaml
fcopy /etc/cron.d/dhclient
sed -ri "s|inet_to_replace|$newnicname|" $target/etc/netplan/01-netcfg.yaml
sed -ri "s|inet_to_replace|$newnicname|" $target/etc/cron.d/dhclient
sed -ri "s|inet_to_replace|$newnicname|" $target/etc/network/interfaces
sed -ri "s|mac_to_replace|$mac|" $target/etc/netplan/01-netcfg.yaml
```
What would be interesting would be to get your logs, particularly your
"variables.log" (for example, to check $NIC1) and "error.log".
Moreover, if you can also supply your network configuration files
(/etc/network/interfaces and so on...), it would be useful.
Best regards,
Rémy.
Le jeu. 22 nov. 2018 à 17:22, Steffen Grunewald <
steffen.grunew...@aei.mpg.de> a écrit :
> On Thu, 2018-11-22 at 12:18:52 +0100, Rémy Dernat wrote:
> > Ok; replying to myself. I found the solution.
> >
> > I just created a script for my BIONIC64 class (using the debian script) :
> >
> > ```
> > #! /bin/bash
> >
> > iface=`ip -o -f inet addr show |awk '$2 !~ "lo|docker" {print $2;exit;}'`
> > mac=`ip -o -f link addr |awk -v iface=$iface '{ if ( $2 == iface":" )
> > {print;} }' |cut -d"/" -f2|awk '{print $2}'`
> > fields="ID_NET_NAME_FROM_DATABASE ID_NET_NAME_ONBOARD ID_NET_NAME_SLOT
> > ID_NET_NAME_PATH"
> > for field in $fields; do
> > name=$(udevadm info /sys/class/net/$NIC1 | sed -rn "s/^E:
> > $field=(.+)/\1/p")
> > if [[ $name ]]; then
> > newnicname=$name
> > break
> > fi
> > done
> >
> >
> > fcopy /etc/netplan/01-netcfg.yaml
> > fcopy /etc/cron.d/dhclient
> >
> > sed -ri "s|inet_to_replace|$newnicname|"
> $target/etc/netplan/01-netcfg.yaml
> > sed -ri "s|inet_to_replace|$newnicname|" $target/etc/cron.d/dhclient
> > sed -ri "s|inet_to_replace|$newnicname|" $target/etc/network/interfaces
> > sed -ri "s|mac_to_replace|$mac|" $target/etc/netplan/01-netcfg.yaml
> > ```
> >
> > With `{mac,inet}_to_replace` in those files (which is replaced with my
> sed).
>
> Hi Remy,
>
> this doesn't work for pure Debian (which hasn't been converted to use
> netplan
> AFAICT), but I can imagine to use udev rules to assign any (old or new
> style)
> interface names to the devices.
>
> I'm running a class script that collects all available interface names (as
> the FAI NFSROOT creates them) and MACs, and builds a persistent-net.rules
> file.
>
> For now, I'm sticking with old-style ones, but this script could be adapted
> to use udevadm info to retrieve the "new-style name" from
> /sys/class/net/eth*
> should this become necessary.
>
> Thanks for sharing your insight,
> Steffen
>
Re: Network interface names
Ok; replying to myself. I found the solution.
I just created a script for my BIONIC64 class (using the debian script) :
```
#! /bin/bash
iface=`ip -o -f inet addr show |awk '$2 !~ "lo|docker" {print $2;exit;}'`
mac=`ip -o -f link addr |awk -v iface=$iface '{ if ( $2 == iface":" )
{print;} }' |cut -d"/" -f2|awk '{print $2}'`
fields="ID_NET_NAME_FROM_DATABASE ID_NET_NAME_ONBOARD ID_NET_NAME_SLOT
ID_NET_NAME_PATH"
for field in $fields; do
name=$(udevadm info /sys/class/net/$NIC1 | sed -rn "s/^E:
$field=(.+)/\1/p")
if [[ $name ]]; then
newnicname=$name
break
fi
done
fcopy /etc/netplan/01-netcfg.yaml
fcopy /etc/cron.d/dhclient
sed -ri "s|inet_to_replace|$newnicname|" $target/etc/netplan/01-netcfg.yaml
sed -ri "s|inet_to_replace|$newnicname|" $target/etc/cron.d/dhclient
sed -ri "s|inet_to_replace|$newnicname|" $target/etc/network/interfaces
sed -ri "s|mac_to_replace|$mac|" $target/etc/netplan/01-netcfg.yaml
```
With `{mac,inet}_to_replace` in those files (which is replaced with my sed).
Best regards,
Rémy.
Le jeu. 22 nov. 2018 à 09:54, Rémy Dernat a écrit :
> Hi,
>
> I am facing the same issue. However, I would like to adopt the new naming
> style, and so, retrieving the interface name dynamically.
>
> The Debian script previously cited seems to do the job at the FAI nfsroot
> step, during installation. However, when it reboots, it looses the ability
> to get this new name.
>
> What I did :
> - copying this debian script (
> https://github.com/faiproject/fai-config/blob/master/scripts/DEBIAN/30-interface
> ) into a BIONIC64 (ubuntu 18.04) class (script/BIONIC64/),
> - In the files/ subdirectory in fai-config directory, I created 2 files :
> - etc/cron.d/dhclient/BIONIC64 (to launch dhclient at reboot),
> - etc/netplan/01-netcfg.yaml/BIONIC64.
>
> I tried to put $NIC1 in those files thinking it would be replaced by the
> right interface name, but it is not.
>
> So, I think that FAI variable cannot be used in the files/ subdirectory
> because it is not avalaible anymore at this step (after the install).
>
> $NIC1 is used in this script
> https://github.com/faiproject/fai/blob/master/lib/get-boot-info
>
> Any idea on how to achieve this according to the new naming convention ?
>
> Kind regards,
> Rémy.
>
> Le lun. 15 oct. 2018 à 09:37, Steffen Grunewald <
> steffen.grunew...@aei.mpg.de> a écrit :
>
>> Hi,
>>
>> thanks for the responses which may turn out to be helpful in some way (if
>> it's
>> already too late to get hold of the old-style if names) - during a
>> half-hearted
>> test, the grub cmdline trick did not work for me.
>>
>> What still makes me curious is
>>
>> > > What I still haven't found is which magic FAI uses to avoid the
>> renaming
>> > > to "predictable network interface names" (a term that's completely
>> misleading
>> > > to me as I'm unable to predict the network names if I'm given a
>> brand-new
>> > > machine). Thomas, can you shed some light on this?
>>
>> If I run FAI sysinfo, I see eth0, ..., wlan0, etc., and dmesg.log doesn't
>> contain any "renamed to" strings, so this renaming must have been avoided
>> somehow, and it's the "how" I am still looking for. (I.e., was makes the
>> NFSROOT special in this respect as it uses the default kernel?)
>>
>> > > My plan is to read the (old-style) interface names assigned by FAI,
>> then
>> > > create a $target/etc/udev/rules.d/70-persistent-net.rules file from
>> that
>> > > information. Is there a hidden trap? When is it best to write that
>> file?
>>
>> As long as the un-renamed if names are available witin FAI, it's simple to
>> create that rules file. The devil may be in the detail when it's required
>> to exist (I suspect there's some initrd magic?)...
>>
>> Thanks again,
>> Steffen
>>
>
Re: Network interface names
Hi, I am facing the same issue. However, I would like to adopt the new naming style, and so, retrieving the interface name dynamically. The Debian script previously cited seems to do the job at the FAI nfsroot step, during installation. However, when it reboots, it looses the ability to get this new name. What I did : - copying this debian script ( https://github.com/faiproject/fai-config/blob/master/scripts/DEBIAN/30-interface ) into a BIONIC64 (ubuntu 18.04) class (script/BIONIC64/), - In the files/ subdirectory in fai-config directory, I created 2 files : - etc/cron.d/dhclient/BIONIC64 (to launch dhclient at reboot), - etc/netplan/01-netcfg.yaml/BIONIC64. I tried to put $NIC1 in those files thinking it would be replaced by the right interface name, but it is not. So, I think that FAI variable cannot be used in the files/ subdirectory because it is not avalaible anymore at this step (after the install). $NIC1 is used in this script https://github.com/faiproject/fai/blob/master/lib/get-boot-info Any idea on how to achieve this according to the new naming convention ? Kind regards, Rémy. Le lun. 15 oct. 2018 à 09:37, Steffen Grunewald < steffen.grunew...@aei.mpg.de> a écrit : > Hi, > > thanks for the responses which may turn out to be helpful in some way (if > it's > already too late to get hold of the old-style if names) - during a > half-hearted > test, the grub cmdline trick did not work for me. > > What still makes me curious is > > > > What I still haven't found is which magic FAI uses to avoid the > renaming > > > to "predictable network interface names" (a term that's completely > misleading > > > to me as I'm unable to predict the network names if I'm given a > brand-new > > > machine). Thomas, can you shed some light on this? > > If I run FAI sysinfo, I see eth0, ..., wlan0, etc., and dmesg.log doesn't > contain any "renamed to" strings, so this renaming must have been avoided > somehow, and it's the "how" I am still looking for. (I.e., was makes the > NFSROOT special in this respect as it uses the default kernel?) > > > > My plan is to read the (old-style) interface names assigned by FAI, > then > > > create a $target/etc/udev/rules.d/70-persistent-net.rules file from > that > > > information. Is there a hidden trap? When is it best to write that > file? > > As long as the un-renamed if names are available witin FAI, it's simple to > create that rules file. The devil may be in the detail when it's required > to exist (I suspect there's some initrd magic?)... > > Thanks again, > Steffen >
Re: nfsroot with kernel 3.X vs 4.X and kernel tftp options
Hi Thomas, Indeed, I am working with SaltStack templates to generate those tftp files. I will compare with those generated with fai-chboot. Thanks, Rémy Le mar. 9 oct. 2018 à 13:54, Thomas Lange a écrit : > >>>>> On Tue, 9 Oct 2018 12:03:24 +0200, Rémy Dernat > said: > > > If I use a kernel 3.X (3.16.0-6-amd64) created with another fai > server, my clients boot fine, but if I try to create a kernel 4.9.0-8, my > clients > > are always stuck in a "read only file system" error ( The exact > error message is the same than this one: > With the 3.x kernel FAI is using aufs as an kernel commandline > option. This activates the aufs (another overlay fs). 4.X does not > include aufs any more but instead is has overlayfs. The kernel cmdline > must be different. Please use fai-chboot for generating the pxe config > file. > > -- > regards Thomas >
nfsroot with kernel 3.X vs 4.X and kernel tftp options
Hi, I have a weird behaviour concerning my nfsroot configuration. If I use a kernel 3.X (3.16.0-6-amd64) created with another fai server, my clients boot fine, but if I try to create a kernel 4.9.0-8, my clients are always stuck in a "read only file system" error ( The exact error message is the same than this one: https://lists.uni-koeln.de/pipermail/linux-fai/2008-January/005648.html ), although my nfs configuration and the disks are the same. I had also another boot behaviour on some other clients: it is just loading vmlinuz + initrd and then, nothing, just a black screen with no warn/error message. Though, the creation of the nfsroot seems to be Ok (no error or warning in /var/log/fai/fai-make-nfsroot.log). I double checked my /etc/fai directory and particularly nfsroot.conf and my NFSROOT file, but these files are the same on both fai servers. I am thinking about a nfsroot option in my pxelinux/tftp file, and my nfs version. However, both servers are listening for nfs version 3 and 4. I found this thread, talking about nfsroot and the nfs versions on bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=922031 My tftp files look like this: # generated by fai-chboot for host 10.1.253.21 with IP 10.1.253.21 default fai-generated label fai-generated kernel vmlinuz-3.16.0-6-amd64 append initrd=initrd.img-3.16.0-6-amd64 rw aufs ip=dhcp root=/dev/nfs nfsroot=10.1.254.41:/srv/fai/nfsroot:vers=3 rd.shell FAI_FLAGS=verbose,sshd,createvt,reboot FAI_ACTION=install server=faiserver.local FAI_CONFIG_SRC=nfs://faiserver.local/srv/fai/config rd.md=0 rd.dm=0 rd.lvm=0 rd.luks=0 And, yes, I am using root=/dev/nfs and vers=3. WHen, I am using sed to switch to kernel 4.9.0-8 and the corresponding nfsroot, it does not work anymore as described above. Could it be caused by this configuration ? Kind regards, Rémy
Re: Install ZFS root with FAI?
Hi,
I am not doing this, but I think it should not be to difficult. I have a
similar script running as a post-install step; maybe you can use something
like this as a script :
#!/bin/bash
check_zpool()
{
zfs_value=`/sbin/zpool list|grep -ic "zfs_bigvol"`
}
check_zpool
if [ $zfs_value -eq 0 ];then
## Create a zfs zpool raid0 for every disks locally except /dev/sda
/sbin/modprobe zfs
ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}'
> /tmp/disks-by-id.txt
while read disk; do
/sbin/parted -s /dev/disk/by-id/$disk mklabel GPT;
done < /tmp/disks-by-id.txt
all_disks_inline=`tr '\n' ' ' < /tmp/disks-by-id.txt`
# sometimes it fails to create the zpool
for i in {1..15}; do
check_zpool
if [ $zfs_value -eq 0 ];then
/sbin/zpool create -o ashift=12 -m /media/bigvol zfs_bigvol
$all_disks_inline
sleep 5
/sbin/zfs set compression=lz4 zfs_bigvol
sleep 1
/sbin/zfs mount -a
else
break
fi
done
rm -f /tmp/disks-by-id.txt
fi
This scripts creates a RAID0 volume using parted and disks id. Note that if
you have many many disks, RAID0 is obvisouly not recommanded (perhaps you
would switch to radiz3) and you may want to use ZFS with disks by path.
Another example using apt (this will create 3 zpool with 20 disks in each
of zpool):
apt-get install zfsutils-linux parted
ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}'
|head -20 > /root/pool1.txt
ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}' |
awk 'NR>20&<=40' > pool2.txt
ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}' |
awk 'NR>40&<=60' > pool3.txt
mkdir -p /export/pool{1,2,3}
pool1_disks=`tr '\n' ' ' < pool1.txt`
pool2_disks=`tr '\n' ' ' < pool2.txt`
pool3_disks=`tr '\n' ' ' < pool3.txt`
apt-get install parted
while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; done
< pool1.txt
while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; done
< pool2.txt
while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; done
< pool3.txt
zpool create pool1 -o ashift=12 -m /export/pool1 raidz3 $pool1_disks
zpool create pool2 -o ashift=12 -m /export/pool2 raidz3 $pool2_disks
zpool create pool3 -o ashift=12 -m /export/pool3 raidz3 $pool3_disks
4 kernel modules are necessary (generally, you will only see spl and zfs) :
zfs.ko, spl.ko, znvpair.ko, zcommon.ko
If you need to build ZFS from source, you will need git (or download the
tar.gz version with wget) :
apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev
libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh gdebi
apt-get install build-essential autoconf libtool gawk alien fakeroot
linux-headers-$(uname -r)
mkdir -p /opt/zfs-source
cd $_
git clone https://github.com/zfsonlinux/spl
git clone https://github.com/zfsonlinux/zfs
cd spl
git checkout master
sh autogen.sh
./configure
make -s -j$(nproc)
make install
cd ../zfs
git checkout master
sh autogen.sh
./configure
make -s -j$(nproc)
make install
echo "/lib64" > /etc/ld.so.conf.d/zfs.conf
echo "/usr/local/lib" >> /etc/ld.so.conf.d/zfs.conf
ldconfig
updatedb
locate spl.ko |xargs -I{} ls -l {}
# overwritting old kernel modules with more recent ones
cp /opt/zfs-source/spl/module/spl/spl.ko /lib/modules/$(uname -r)
/extra/spl/spl/spl.ko
cp /opt/zfs-source/spl/module/spl/spl.ko /lib/modules/$(uname -r)
/kernel/zfs/spl/spl.ko
locate zfs.ko |xargs -I{} ls -l {}
cp /opt/zfs-source/zfs/module/zfs/zfs.ko /lib/modules/$(uname -r)
/kernel/zfs/zfs/zfs.ko
locate znvpair.ko |xargs -I{} ls -l {}
cp /opt/zfs-source/zfs/module/nvpair/znvpair.ko /lib/modules/$(uname -r)
/kernel/zfs/nvpair/znvpair.ko
locate zcommon.ko |xargs -I{} ls -l {}
cp /opt/zfs-source/zfs/module/zcommon/zcommon.ko /lib/modules/$(uname -r)
/kernel/zfs/zcommon/zcommon.ko
modprobe spl
modprobe znvpair
modprobe zcommon
modprobe zfs
So you can imagine in the `package_config` a class with all the necessary
packages and in `scripts` the same class which is running other commands. I
don't think there is anything specific in FAI to use ZFS with the FAI
`disk_config`.
Kind regards,
Rémy
Le ven. 5 oct. 2018 à 09:06, Steffen Grunewald
a écrit :
> Good morning,
>
> once again I have browsed multiple years of the list archive, without
> finding
> anything useful, but it's Firday and I may have gone blind over the past
> week.
>
> What I'm looking for is FAI installation details to set up a root
> filesystem
> on ZFS (two mirrored disks, possibly one spare) on a number of new
> machines.
> (I know that FAI isn't mandatory for this, but never underestimate the
> documentation effect of having the proper classes.)
>
> - Which additions to the nfsroot are required / recommended?
> - Which packages must be added to the machine package configuration?
> - Which hooks should I use to create the root filesystem?
>
> There must be recipes
Re: UEFI issues booting into FAI sysinfo on Dell PowerEdge
Hi, I suggest you to retrieve the version of SYSLINUX 6.04; look here for more informations : https://groups.google.com/a/lbl.gov/forum/#!msg/warewulf/klTLgX-L4nw/IJZo3-jgAAAJ BTW, sometimes an update of the firmware on the client give great results. Best regards, Rémy. Le lun. 3 sept. 2018 à 14:27, Steffen Grunewald < steffen.grunew...@aei.mpg.de> a écrit : > Hello, > > I've been using FAI for years now, and could always avoid UEFI - until last > week when a coworker asked me to "quickly" run a sysinfo for one of his new > machines. > Of course, it comes with UEFI activated. > After finding how to distinguish between BIOS and UEFI PXE requests, and > setting up the DHCP/TFTP server accordingly, it was only "yet another step" > to find and add ldlinux.e64 to get the machine booting. > It requests the kernel, and loads it. > It requests the initrd, and hangs doing so. > /var/log/daemon shows both requests and nothing more after them. > > There's a warning > "core_udb_sendto: stalling on configure with no mapping" > further up the boot screen, but I'm not sure it'd be related. > > The corresponding pxelinux.cfg file was created once, worked in legacy > mode. > > Unfortunately, UEFI-related FAI documentation is very sparse, and I may > have > overlooked something? > Machine is a Dell one (if this makes a difference), PowerEdge R840. > > Fortunately, the machines were able to boot in legacy mode, so the goal was > reached to a major extent (I couldn't test EFI detection though, not run > "efibootmgr -v" within one of the class scripts). > > > Any suggestions what to try next? > > Thanks, > Steffen > > -- > Steffen Grunewald, Cluster Administrator > Max Planck Institute for Gravitational Physics (Albert Einstein Institute) > Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany > ~~~ > Fon: +49-331-567 7274 > Mail: steffen.grunewald(at)aei.mpg.de > ~~~ >
Re: UEFI + Legacy BIOS TFTP/PXELINUX
Hi, Yes ! Thanks ! It works after modifying the file "60-misc" like described by Thomas and this line : ``` primary /boot/efi 204800K vfat rw ``` to ``` primary /boot/efi 204800K vfat rw createopts="-s2 -F32" ``` Thank you very much. Best regards, Rémy. Le lun. 9 juil. 2018 à 15:39, Thomas Lange a écrit : > The current version of 60-misc now uses this > code > > if [ -d /sys/firmware/efi ]; then > echo GRUB_EFI > elif ifclass -o I386 AMD64 ; then > echo GRUB_PC > fi > > > -- > regards Thomas >
Re: UEFI + Legacy BIOS TFTP/PXELINUX
Hi John, Toomas,
Thank you very much.
I changed my config to match your config, John, however, I still have an
error (which is a new one).
First of all, it boots fine (the first time), launching the install,
retrieving syslinux.efi, but it detects the class "GRUB_PC" instead of
"GRUB_EFI" (obviously, it should load GRUB_EFI !).
```
root@faiserv:/srv/fai/config# grep -r GRUB_PC .
./package_config/DEBIAN:PACKAGES install GRUB_PC
./package_config/UBUNTU:PACKAGES install GRUB_PC
./class/60-misc:ifclass -o GRUB_PC GRUB_EFI ||echo GRUB_PC
root@faiserv:/srv/fai/config# find . -name GRUB_PC
./scripts/GRUB_PC
./disk_config/GRUB_PC
root@faiserv:/srv/fai/config# grep -r GRUB_EFI .
./package_config/GRUB_EFI:PACKAGES aptitude GRUB_EFI
./package_config/DEBIAN:PACKAGES install GRUB_EFI
./package_config/UBUNTU:PACKAGES install GRUB_EFI
./class/60-misc:ifclass -o GRUB_PC GRUB_EFI ||echo GRUB_PC
root@faiserv:/srv/fai/config# find . -name GRUB_EFI
./package_config/GRUB_EFI
./scripts/GRUB_EFI
./disk_config/GRUB_EFI
```
I am a bit lost (...); I do not why it is loading GRUB_PC instead of
GRUB_EFI.
I also tried to switch to the DEMO class to check if it comes from my
defined class; I get the same result: it load also the GRUB_PC class.
My 60-misc class is :
```
cat class/60-misc
#! /bin/bash
ifclass -o CENTOS SLC && exit 0
if ifclass -o I386 AMD64 ; then
ifclass -o GRUB_PC GRUB_EFI ||echo GRUB_PC
fi
```
Best regards,
Rémy.
Le ven. 6 juil. 2018 à 17:14, John G Heim a écrit :
> Sorry to reply to your posts out of order but I would not follow the
> advice in that message. It is at least out dated. You do not have to
> change your dhcp config after each install and you can continue to use
> your fai-chboot config files. Syslinux does support EFI booting. It's
> in the syslinux-efi package.
>
> Here is the code in my dhcp config file that sets the filename option
> depending on EFI/BIOS boot:
>
> === begin ===
> # tell a kernel how to boot via PXE or NFS
> server-name faiserver;
> next-server faiserver;
> # Check pxe boot code #93 to determine whether we are booting via BIOS
> or EFI
> # 2018-07-04: jheim
> option architecture-type code 93 = unsigned integer 16;
> class "pxeclients" {
> match if substring (option vendor-class-identifier, 0, 9) =
> "PXEClient";
> if option architecture-type = 00:00 {
> filename "fai/pxelinux.0";
> } else {
> filename "fai/syslinux.efi64";
> }
> }
> === end ===
>
> The file syslinux.efi64 is from the debian syslinux-efi package. It's
> original name/path was /usr/lib/SYSLINUX.EFI/efi64/syslinux.efi. I
> renamed it syslinux.efi64 to distinguish it from the 32-bit syslinux.efi
> file. Then I decided not to bother with 32-bit EFI boots. You also need
> some libs in your tftp directory to support EFI booting via syslinux. I
> think if you modify your dhcp config as abobe, copy the necessary files
> from the syslinux-efi package to your tftp space, you'll successfully
> boot into the FAI install. You will also need to create new disk config
> files to partition via gpt instead of msdos. (More on that below.)
>
> 1. Cut/passte above code into your dhcp config.
> 2. apt-get install syslinux-efi
> 3a. cp /usr/lib/SYSLINUX.EFI/efi64/syslinux.efi /srv/tftp/fai/
> 3b. cp /usr/lib/syslinux/modules/efi64/ldlinux.e64 /srv/tftp/fai/
>
> Done with the boot stuff. Now the disk config stuff. Code that detects
> whether you have booted via BIOS or EFI already exists in the fai-doc
> examples. It's in a class script called 60-grub. It creates either
> GRUB_PC or GRUB_EFI class. You have to retain your old BIOS disk config
> files for older machines. I used to create a class called LARGEDISK for
> disks large enough to partition into /, /var/ usr, ... So I just renamed
> that GRUB_PC. Then I copied that file to GRUB_EFI andmodified it for
> gpt partitioning and EFI boot. It looks like this:
>
> === begin ===
> # 2018-07-04: jheim
> disk_config disk1 disklabel:gpt fstabkey:label align-at:1
> primary /boot/efi 204800K vfat rw
> primary/usr/local 157286400Kext4 rw,relatime
> createopts="-L LOCALFS"
> primary/ 72G- ext4 rw,relatime
> createopts="-L ROOTFS"
> primaryswap8Gswap rw
> createopts="-L SWAPFS"
> # EOF
> === end ===
>
>
>
>
>
>
>
> On 07/06/2018 07:27 AM, Rémy Dernat wrote:
> > Hi,
> >
> > I am trying to set up an UEFI boot pxe mainly because our hardware
> > servers do not support full legacy BIOS anymore (particularly hard disk
> > plugin on those servers).
> >
> > Our legacy pxelinux works just fine
Re: UEFI + Legacy BIOS TFTP/PXELINUX
Hmmm Now it is installing fine, but it is trying to reinstall at each
reboot, because the boot from the local disk failed (although the entry is
well written is the UEFI boot menu).
My grub.cfg is now:
```
set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
set timeout=5
# for debug:
#set pager=1
#set debug=all
insmod efi_gop
insmod efi_uga
if loadfont ${prefix}/unicode.pf2
then
insmod gfxterm
set gfxmode=auto
set gfxpayload=keep
terminal_output gfxterm
fi
menuentry "Install from FAI" {
linux vmlinuz-3.16.0-6-amd64 ip=dhcp aufs rw root=/dev/nfs
nfsroot=IP.IP.IP.IP:/srv/fai/nfsroot FAI_FLAGS=verbose,sshd,createvt,reboot
FAI_ACTION=install server=faiserv.acme.ltd
FAI_CONFIG_SRC=nfs://faiserv.acme.ltd/srv/fai/config
initrd initrd.img-3.16.0-6-amd64
}
```
It looks like FAI install is just loading the FAIBASE class. I think that
"FAI_CONFIG_SRC=nfs://faiserv.acme.ltd/srv/fai/config" is not loading at
the install step. Maybe I have some syntax error...
Best regards,
Rémy.
Le ven. 6 juil. 2018 à 14:27, Rémy Dernat a écrit :
> Hi,
>
> I am trying to set up an UEFI boot pxe mainly because our hardware servers
> do not support full legacy BIOS anymore (particularly hard disk plugin on
> those servers).
>
> Our legacy pxelinux works just fine from a while now (and thanks for it).
>
> I followed this link :
> https://lists.uni-koeln.de/pipermail/linux-fai/2014-February/010294.html
> ... with no luck. I created an "UEFI" specific class to avoid the erase of
> DEFAULT, and put everything in that new class, because we will need to keep
> the pxelinux config.
>
> My tftp fai folder is organised like this:
> ```
> root@faiserv:/srv# tree -L 2
> .
> ├── fai
> │ ├── config
> │ └── nfsroot
> ├── nfs4
> └── tftp
> └── fai
>
> root@faiserv:/srv/tftp/fai# tree -L 2
> .
> ├── c2960s-universalk9-tar.150-2a.SE9.tar
> ├── cisco-ios-150-2a.tar -> c2960s-universalk9-tar.150-2a.SE9.tar
> ├── efi
> │ ├── fonts
> │ ├── grub.cfg
> │ ├── grub.cfg.nok
> │ ├── i386-pc
> │ ├── locale
> │ ├── pxelinux.cfg -> ../pxelinux.cfg
> │ ├── unicode.pf2
> │ └── x86_64-efi
> ├── gpxelinux.0
> ├── initrd.img-3.16.0-4-amd64
> ├── initrd.img-3.16.0-6-amd64
> ├── ldlinux.c32
> ├── lpxelinux.0
> ├── pxelinux.0
> ├── pxelinux.0.back
> ├── pxelinux.cfg
> │ ├── 0A01FD03
> │ ├── 0A01FFF2
> │ ├── A226B5A1.disable
> │ ├── A226B5A2.disable
> │ ├── A226B5A3.disable
> │ ├── A226B5A4.disable
> │ ├── A226B5A5.disable
> │ ├── A226B5A6.disable
> │ ├── A226B5A7.disable
> │ ├── A226B5A9
> │ ├── A226B5AA.disable
> │ ├── A226B5AB.disable
> │ ├── A226B5B0.disable
> │ ├── A226B5B4.disable
> │ ├── A226B5B8
> │ └── default
> ├── syslinux.efi
> ├── vmlinuz-3.16.0-4-amd64
> └── vmlinuz-3.16.0-6-amd64
>
> ```
>
> I do not know if the UEFI is also searching for a filename matching its IP
> address with the hexa value, so I created a symbolic link to the
> pxelinux.cfg parent directory...
>
>
> I have no problem with DHCP, and my client is loading the grub.cfg menu.
> However, it fails while searching for the nfsroot. The error I get (with
> debug=all) is (replacing sensitive informations):
> ```
> kern/disk.c:196 : Opening `tftp,IP.IP.IP.IP'...
> disk/efi/efidisk.c:461 : opening tftp
> kern/disk.c: 281: Opening `tftp,IP.IP.IP.IP' failed.
> kern/disk.c: 295: Closing `tftp'.
> ```
>
>
> Here is my grub.cfg config :
> ```
> root@faiserv:/srv/tftp/fai/efi# cat grub.cfg
> set menu_color_normal=white/black
> set menu_color_highlight=black/light-gray
> set timeout=5
> # for debug:
> #set pager=1
> set debug=all
>
> insmod efi_gop
> insmod efi_uga
> if loadfont ${prefix}/unicode.pf2
> then
> insmod gfxterm
> set gfxmode=auto
> set gfxpayload=keep
> terminal_output gfxterm
> fi
>
> menuentry "Install from FAI" {
> linux vmlinuz-3.16.0-6-amd64 rw ip=dhcp root=/dev/nfs
> nfsroot=IP.IP.IP.IP:/srv/fai/nfsroot FAI_FLAGS=verbose,sshd,createvt,reboot
> FAI_ACTION=install server=faiserv.acme.ltd
> FAI_CONFIG_SRC=nfs://faiserv.acme.ltd/srv/fai/config
> initrd initrd.img-3.16.0-6-amd64
> }
>
> ```
>
>
> Any help would be greatly appreciated.
>
> Thanks,
> Rémy
>
UEFI + Legacy BIOS TFTP/PXELINUX
Hi,
I am trying to set up an UEFI boot pxe mainly because our hardware servers
do not support full legacy BIOS anymore (particularly hard disk plugin on
those servers).
Our legacy pxelinux works just fine from a while now (and thanks for it).
I followed this link :
https://lists.uni-koeln.de/pipermail/linux-fai/2014-February/010294.html
... with no luck. I created an "UEFI" specific class to avoid the erase of
DEFAULT, and put everything in that new class, because we will need to keep
the pxelinux config.
My tftp fai folder is organised like this:
```
root@faiserv:/srv# tree -L 2
.
├── fai
│ ├── config
│ └── nfsroot
├── nfs4
└── tftp
└── fai
root@faiserv:/srv/tftp/fai# tree -L 2
.
├── c2960s-universalk9-tar.150-2a.SE9.tar
├── cisco-ios-150-2a.tar -> c2960s-universalk9-tar.150-2a.SE9.tar
├── efi
│ ├── fonts
│ ├── grub.cfg
│ ├── grub.cfg.nok
│ ├── i386-pc
│ ├── locale
│ ├── pxelinux.cfg -> ../pxelinux.cfg
│ ├── unicode.pf2
│ └── x86_64-efi
├── gpxelinux.0
├── initrd.img-3.16.0-4-amd64
├── initrd.img-3.16.0-6-amd64
├── ldlinux.c32
├── lpxelinux.0
├── pxelinux.0
├── pxelinux.0.back
├── pxelinux.cfg
│ ├── 0A01FD03
│ ├── 0A01FFF2
│ ├── A226B5A1.disable
│ ├── A226B5A2.disable
│ ├── A226B5A3.disable
│ ├── A226B5A4.disable
│ ├── A226B5A5.disable
│ ├── A226B5A6.disable
│ ├── A226B5A7.disable
│ ├── A226B5A9
│ ├── A226B5AA.disable
│ ├── A226B5AB.disable
│ ├── A226B5B0.disable
│ ├── A226B5B4.disable
│ ├── A226B5B8
│ └── default
├── syslinux.efi
├── vmlinuz-3.16.0-4-amd64
└── vmlinuz-3.16.0-6-amd64
```
I do not know if the UEFI is also searching for a filename matching its IP
address with the hexa value, so I created a symbolic link to the
pxelinux.cfg parent directory...
I have no problem with DHCP, and my client is loading the grub.cfg menu.
However, it fails while searching for the nfsroot. The error I get (with
debug=all) is (replacing sensitive informations):
```
kern/disk.c:196 : Opening `tftp,IP.IP.IP.IP'...
disk/efi/efidisk.c:461 : opening tftp
kern/disk.c: 281: Opening `tftp,IP.IP.IP.IP' failed.
kern/disk.c: 295: Closing `tftp'.
```
Here is my grub.cfg config :
```
root@faiserv:/srv/tftp/fai/efi# cat grub.cfg
set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
set timeout=5
# for debug:
#set pager=1
set debug=all
insmod efi_gop
insmod efi_uga
if loadfont ${prefix}/unicode.pf2
then
insmod gfxterm
set gfxmode=auto
set gfxpayload=keep
terminal_output gfxterm
fi
menuentry "Install from FAI" {
linux vmlinuz-3.16.0-6-amd64 rw ip=dhcp root=/dev/nfs
nfsroot=IP.IP.IP.IP:/srv/fai/nfsroot FAI_FLAGS=verbose,sshd,createvt,reboot
FAI_ACTION=install server=faiserv.acme.ltd
FAI_CONFIG_SRC=nfs://faiserv.acme.ltd/srv/fai/config
initrd initrd.img-3.16.0-6-amd64
}
```
Any help would be greatly appreciated.
Thanks,
Rémy
