Hi, After digging some time around this issue, I am finally able to boot the RAID1 partition with EFI boot enabled on it (using this example: https://fai-project.org/download/misc/disk_config/lvm-on-luks-on-raid).
Here is my final disk_config file: ```# example of new config file for setup-storage # # <type> <mountpoint> <size> <fs type> <mount options> <misc options> disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid primary - 2G - - primary - 0- - - disk_config disk2 sameas:disk1 disk_config raid raid1 /boot/efi disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose --metadata=0.90" raid1 - disk1.2,disk2.2 - - disk_config lvm vg vg_system md1 vg_system-swap swap 16G swap sw vg_system-root / 60G-200G ext4 rw,noatime vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev ``` Best regards, Rémy Le ven. 1 déc. 2023 à 12:11, Rémy Dernat <remy...@gmail.com> a écrit : > > Hi Andrew, > > Thanks for suggestions. > > Now, my file "disk_config/GRUB_EFI_RAID1" is [*]: > > ``` > # example of new config file for setup-storage > # > # <type> <mountpoint> <size> <fs type> <mount options> <misc options> > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 > primary - 2G - - > primary /boot/efi 1G vfat rw createopts="-F32" > primary - 2G - - > primary - 0- - - > > #disk_config disk2 sameas:disk1 > disk_config disk2 disklabel:gpt fstabkey:uuid > primary - 2G - - > primary /boot/efi2 1G vfat rw createopts="-F32" > primary - 2G - - > primary - 0- - - > > disk_config raid fstabkey:uuid > raid1 /boot disk1.1,disk2.1 ext4 rw,noatime mdcreateopts="--verbose > --metadata=0.90" > raid1 swap disk1.3,disk2.3 swap sw > raid1 - disk1.4,disk2.4 - - > > disk_config lvm fstabkey:uuid > vg vg_system md2 > vg_system-root / 60G-200G ext4 rw,noatime > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > ``` > > => Even if the installation succeeded, grub does not seem to work. > > Anyway, I am wondering how you set this in the disk_config file (syntax ?): > > > I like to put a 100MB partition at the end of disks in a RAID pair to allow > > for slightly differing size of a replacement disk. Although currently it > > seems that any new disk we get is many many GB bigger than the old failed > > one. > > Thanks again, > > Best regards, > Rémy > > > > [*] there was an error for /boot partition which was not formatted the > right way (vfat does not allow symbolic links). vfat should be used > for /boot/EFI only. > > Le ven. 1 déc. 2023 à 05:43, Andrew Ruthven <and...@etc.gen.nz> a écrit : > > > > Nice, good work. > > > > A couple of suggestions... > > > > I'd put swap on RAID1, if you have a disk go offline while the system is > > running, you've lost anything that had been swapped out to the swap on the > > failed disk. > > > > I like to put a 100MB partition at the end of disks in a RAID pair to allow > > for slightly differing size of a replacement disk. Although currently it > > seems that any new disk we get is many many GB bigger than the old failed > > one. > > > > I've found that some servers allow the EFI partition to be RAID1, the > > current servers we're using support that. So no need to copy files around. > > > > Cheers, > > Andrew > > > > On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote: > > > > > Ok, FYI, I think I have a working configuration with : > > > > > > ``` > > > # example of new config file for setup-storage > > > # > > > # <type> <mountpoint> <size> <fs type> <mount options> <misc > > > options> > > > > > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1 > > > primary - 2G - - > > > primary /boot/efi 1G vfat rw createopts="-F32" > > > primary swap 2G swap sw,pri=1 > > > primary - 0- - - > > > > > > #disk_config disk2 sameas:disk1 > > > disk_config disk2 disklabel:gpt fstabkey:uuid > > > primary - 2G - - > > > primary /boot/efi2 1G vfat rw createopts="-F32" > > > primary swap 2G swap sw,pri=1 > > > primary - 0- - - > > > > > > disk_config raid fstabkey:uuid > > > #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose > > > --metadata=1.0" createopts="-L boot" > > > raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose > > > --metadata=0.90" > > > raid1 - disk1.4,disk2.4 - - > > > > > > disk_config lvm fstabkey:uuid > > > vg vg_system md1 > > > vg_system-root / 60G-200G ext4 rw,noatime > > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > > > ``` > > > > > > My previous message was crystal clear. md0 did not have any formatted > > > filesystem on it. So I added `vfat` option on /boot. > > > > > > Best regards, > > > Rémy > > > > > > > > > Le jeu. 30 nov. 2023 à 16:53, Rémy Dernat > > <[remy...@gmail.com](mailto:remy...@gmail.com)> a écrit : > > > > > > > > > > > What is strange here, is that md1 is found, as well as md0, but > > > > contrary to md0, md1 have an UUID ... > > > > > > > > However, UUID seems to be retrieve by blkid from /dev/disk/by-id, > > > > where both md array have their own mdadm UUID, but only md1 have an > > > > UUID from LVM (with pvdisplay / pvs ) ... > > > > > > > > > > > > Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat > > <[remy...@gmail.com](mailto:remy...@gmail.com)> a écrit : > > > > > > > > > > > > > > Hi Andrew, > > > > > > > > > > Indeed, this removed the syntax issue. However, I have a new error > > now: > > > > > > > > > > ``` > > > > > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2 > > > > > Command had non-zero exit code > > > > > ``` > > > > > Indeed, blkid is not able to found md0 UUID. > > > > > > > > > > I tried to modify the disk_config file to avoid this error, but I am > > > > > still stuck with it. Here is what look like my disk_config file now: > > > > > > > > > > ``` > > > > > # example of new config file for setup-storage > > > > > # > > > > > # <type> <mountpoint> <size> <fs type> <mount options> <misc > > options> > > > > > > > > > > disk_config disk1 > > > > > primary - 2G - - > > > > > primary /boot/efi 1G vfat rw createopts="-F32" > > > > > primary swap 2G swap sw,pri=1 > > > > > primary - 0- - - > > > > > > > > > > #disk_config disk2 sameas:disk1 > > > > > disk_config disk2 disklabel:gpt fstabkey:uuid > > > > > primary - 2G - - > > > > > primary /boot/efi2 1G vfat rw createopts="-F32" > > > > > primary swap 2G swap sw,pri=1 > > > > > primary - 0- - - > > > > > > > > > > disk_config raid fstabkey:uuid > > > > > raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose > > > > > --metadata=0.90" createopts="-L boot" > > > > > raid1 - disk1.4,disk2.4 - - > > > > > > > > > > disk_config lvm fstabkey:uuid > > > > > vg vg_system md1 > > > > > vg_system-root / 60G-200G ext4 rw,noatime > > > > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev > > > > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime > > > > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev > > > > > ``` > > > > > > > > > > I also tried to change the raid metadata to 1.0, but that did not > > > > > change anything. I wrote back ` fstabkey:uuid` behind disk_config raid > > > > > because it does not fail with syntax error. > > > > > > > > > > Thanks for help > > > > > > > > > > Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven > > <[and...@etc.gen.nz](mailto:and...@etc.gen.nz)> a écrit : > > > > > > > > > > > > > > > > > Hey, > > > > > > > > > > > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote: > > > > > > > > > > > > > Hi awesome Fai guys, > > > > > > > > > > > > > > I have a weird syntax issue with the following disk_config: > > > > > > > > > > > > > > > > > > > [snip] > > > > > > > > > > > > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1 > > > > > > > > > > > > > > > > > > This line is your problem, remove everything after the word raid. So > > it > > > > > > should just be: > > > > > > > > > > > > disk_config raid > > > > > > > > > > > > Cheers, > > > > > > Andrew > > > > > > -- > > > > > > Andrew Ruthven, Wellington, New Zealand > > > > > > [and...@etc.gen.nz](mailto:and...@etc.gen.nz) | > > > > > > Catalyst Cloud: | This space intentionally left blank > > > > > > [https://catalystcloud.nz](https://catalystcloud.nz) | > > > > > > > > > > > > > > > > > > > > > > > > ``` > > -- > > Andrew Ruthven, Wellington, New Zealand > > and...@etc.gen.nz | > > Catalyst Cloud: | This space intentionally left blank > > https://catalystcloud.nz | > > > > > > ``` > > > >