UEFI boot mode is indeed a good option, as it does not rely on the ZFS pool for booting the system (it uses a separate VFAT partition for the bootloader). However, this method does not solve another problem which is the redundancy of the bootloader, as the UEFI partition can be installed only on a single (boot) drive. If that drive fails, then one must rebuild the UEFI partition on a different or on the replacement drive.
On Mon, 14 Sep 2020 at 09:14, Arjen via pve-user <[email protected]> wrote: > > > > ---------- Forwarded message ---------- > From: Arjen <[email protected]> > To: Proxmox VE user list <[email protected]> > Cc: > Bcc: > Date: Mon, 14 Sep 2020 08:04:03 +0000 > Subject: Re: [PVE-User] zfs root after last update grub unknown filesystem > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Monday, September 14, 2020 9:07 AM, Aaron Lauterer < > [email protected]> wrote: > > > Alternatively you can try to install the machine in UEFI mode. In UEFI > mode with root ZFS the installer will set up systemd-boot instead of grub. > This alleviates the problem with grubs limited ZFS compatibility. > > > > See the docs for more infos: > https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysboot > > > > HTH, > > Aaron > > I had the same problem with a zpool feature that could not be turned off. > I used this work-around for a while in the hope that the bug would be fixed > (GRUB patch existed for 5 years but it was never tested/reviewed): > Put a new Proxmox VE installation on USB and copy /boot from the hard > drives to the USB installation. > With that, it could boot from USB, after which it would pickup on the > rpool on the hard drives (probably by lucky chance). > Eventually, reinstalling with UEFI+GPT+ESP+systemd-boot (which did not > exists when I encountered the issue) fixed it. > > I would suggest taking the pain now, and reinstall using UEFI, to prevent > such issues in the future. > > > On 9/14/20 12:16 AM, Gianni Milo wrote: > > > > > GRUB does not support all zfs features, so it's quite common for it to > fail > > > to recognise the rpool during boot if it has a feature which is > > > incompatible with it. In your case, I believe that is the "encyption" > > > feature. > > > Do you recall the issue starting after enabling encryption on > rpool/data > > > dataset? If so, you may have to rebuild the pool and leave rpool/data > > > unencrypted. > > > Note that even though you enabled encryption only on rpool/data, the > > > feature will take effect at the pool level, hence the GRUB issue you > > > are facing. > > > Because of this issue, people have started using a separate boot pool > > > (bpool), with limited zfs features enabled and a different data pool > > > (rpool) for the OS and the data. I believe that the PVE installer > should be > > > modified to follow this approach (if it hasn't done it already) to > overcome > > > similar issues. > > > Gianni > > > On Sun, 13 Sep 2020 at 22:29, Stephan Leemburg > [email protected] > > > wrote: > > > > > > > Hi All, > > > > I have a proxmox system at netcup that was clean installed about two > > > > weeks ago. > > > > It is full zfs, so root on zfs. I am migrating from one netcup system > > > > with less storage to this system. > > > > This system is using the pve-no-subscription repository, but after > > > > migration I will move the subscription from the 'old' system to this > > > > system. > > > > The rpool/data dataset is zfs encrypted. > > > > Today I did zfs send/recv from the 'old' system to this system to the > > > > rpool/data dataset. > > > > After that I did an apt update and noticed there where updates > available. > > > > After the upgrade and the mandatory reboot, the system does not come > up > > > > anymore. It is stuck in grub rescue. > > > > Grub mentions that it has a 'unknown filesystem'. > > > > Has anyone else experienced this same situation? > > > > If so and you could recover, what was the reason and fix? > > > > I am still researching what is causing this. If I boot a .iso then I > can > > > > import the pool and see all datasets and subvolumes. > > > > So, it seems that the zpool itself and the datasets are ok, it is > just > > > > that grub is unable to recognize them for some reason. > > > > I have read about other situations like this where large_dnode > seemed to > > > > be the cause. I noticed that on the zpool large_dnode is enabled. > And it > > > > is a creation only setting. It cannot be changed to disabled > afterwards. > > > > This must have been done by the Proxmox installer. As booting is > about > > > > the root filesystem, I guess the zfs send / recv to the rpool/data > > > > dataset would have nothing to do with it, but I could be wrong. > > > > I will continue my research tomorrow evening after some other > > > > obligations, but if anyone has an idea, please share it. If it is > just > > > > how to get more debugging info out of the zfs module in grub. > > > > Because 'unknown filesystem', with the zfs module loaded is kind of > not > > > > helping enough.. > > > > Best regards, > > > > Stephan > > > > > > > > pve-user mailing list > > > > [email protected] > > > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > pve-user mailing list > > > [email protected] > > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > pve-user mailing list > > [email protected] > > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > > > > > > > ---------- Forwarded message ---------- > From: Arjen via pve-user <[email protected]> > To: Proxmox VE user list <[email protected]> > Cc: Arjen <[email protected]> > Bcc: > Date: Mon, 14 Sep 2020 08:04:03 +0000 > Subject: Re: [PVE-User] zfs root after last update grub unknown filesystem > _______________________________________________ > pve-user mailing list > [email protected] > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > _______________________________________________ pve-user mailing list [email protected] https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
