On 14-09-2020 11:29, Gianni Milo wrote:
UEFI boot mode is indeed a good option, as it does not rely on the ZFS pool
for booting the system (it uses a separate VFAT partition for the
bootloader).
Yes, that would be good, but the netcup system does not 'support' UEFI.
So, I guess I will take the pain and reinstall the system with a
traditional lvm as root and a luks encrypted container with the zpool in it.
luks provides more than one keyslot. LVM will do ok for root and works
at least with grub.
I could access the rpool by booting into the debug installer. So I could
make a tar of the rpool/ROOT/pve-1 filesystem. That will make a recovery
more easy, without loosing too much work.
BTW Also the 'rescue mode' of the Proxmox installer barks that there is
'no rpool'. Must be related.
However, this method does not solve another problem which is the redundancy
of the bootloader, as the UEFI partition can be installed only on a single
(boot) drive. If that drive fails, then one must rebuild the UEFI partition
on a different or on the replacement drive.
On Mon, 14 Sep 2020 at 09:14, Arjen via pve-user <[email protected]>
wrote:
---------- Forwarded message ----------
From: Arjen <[email protected]>
To: Proxmox VE user list <[email protected]>
Cc:
Bcc:
Date: Mon, 14 Sep 2020 08:04:03 +0000
Subject: Re: [PVE-User] zfs root after last update grub unknown filesystem
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, September 14, 2020 9:07 AM, Aaron Lauterer <
[email protected]> wrote:
Alternatively you can try to install the machine in UEFI mode. In UEFI
mode with root ZFS the installer will set up systemd-boot instead of grub.
This alleviates the problem with grubs limited ZFS compatibility.
See the docs for more infos:
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysboot
HTH,
Aaron
I had the same problem with a zpool feature that could not be turned off.
I used this work-around for a while in the hope that the bug would be fixed
(GRUB patch existed for 5 years but it was never tested/reviewed):
Put a new Proxmox VE installation on USB and copy /boot from the hard
drives to the USB installation.
With that, it could boot from USB, after which it would pickup on the
rpool on the hard drives (probably by lucky chance).
Eventually, reinstalling with UEFI+GPT+ESP+systemd-boot (which did not
exists when I encountered the issue) fixed it.
I would suggest taking the pain now, and reinstall using UEFI, to prevent
such issues in the future.
On 9/14/20 12:16 AM, Gianni Milo wrote:
GRUB does not support all zfs features, so it's quite common for it to
fail
to recognise the rpool during boot if it has a feature which is
incompatible with it. In your case, I believe that is the "encyption"
feature.
Do you recall the issue starting after enabling encryption on
rpool/data
dataset? If so, you may have to rebuild the pool and leave rpool/data
unencrypted.
Note that even though you enabled encryption only on rpool/data, the
feature will take effect at the pool level, hence the GRUB issue you
are facing.
Because of this issue, people have started using a separate boot pool
(bpool), with limited zfs features enabled and a different data pool
(rpool) for the OS and the data. I believe that the PVE installer
should be
modified to follow this approach (if it hasn't done it already) to
overcome
similar issues.
Gianni
On Sun, 13 Sep 2020 at 22:29, Stephan Leemburg
[email protected]
wrote:
Hi All,
I have a proxmox system at netcup that was clean installed about two
weeks ago.
It is full zfs, so root on zfs. I am migrating from one netcup system
with less storage to this system.
This system is using the pve-no-subscription repository, but after
migration I will move the subscription from the 'old' system to this
system.
The rpool/data dataset is zfs encrypted.
Today I did zfs send/recv from the 'old' system to this system to the
rpool/data dataset.
After that I did an apt update and noticed there where updates
available.
After the upgrade and the mandatory reboot, the system does not come
up
anymore. It is stuck in grub rescue.
Grub mentions that it has a 'unknown filesystem'.
Has anyone else experienced this same situation?
If so and you could recover, what was the reason and fix?
I am still researching what is causing this. If I boot a .iso then I
can
import the pool and see all datasets and subvolumes.
So, it seems that the zpool itself and the datasets are ok, it is
just
that grub is unable to recognize them for some reason.
I have read about other situations like this where large_dnode
seemed to
be the cause. I noticed that on the zpool large_dnode is enabled.
And it
is a creation only setting. It cannot be changed to disabled
afterwards.
This must have been done by the Proxmox installer. As booting is
about
the root filesystem, I guess the zfs send / recv to the rpool/data
dataset would have nothing to do with it, but I could be wrong.
I will continue my research tomorrow evening after some other
obligations, but if anyone has an idea, please share it. If it is
just
how to get more debugging info out of the zfs module in grub.
Because 'unknown filesystem', with the zfs module loaded is kind of
not
helping enough..
Best regards,
Stephan
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
---------- Forwarded message ----------
From: Arjen via pve-user <[email protected]>
To: Proxmox VE user list <[email protected]>
Cc: Arjen <[email protected]>
Bcc:
Date: Mon, 14 Sep 2020 08:04:03 +0000
Subject: Re: [PVE-User] zfs root after last update grub unknown filesystem
_______________________________________________
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
_______________________________________________
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
_______________________________________________
pve-user mailing list
[email protected]
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
