In case it's related, can you confirm the kernel version you are using now, and the kernel version from before the upgrade?
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1635115 Title: grub fails zfs root filesystem detection due to failed checksum Status in grub2 package in Ubuntu: Confirmed Status in zfs-linux package in Ubuntu: New Bug description: root@q-1-d.sv1:~# lsb_release -rd Description: Ubuntu 16.04.1 LTS Release: 16.04 root@q-1-d.sv1:~# apt-cache policy grub2-common grub2-common: Installed: 2.02~beta2-36ubuntu3.2 Candidate: 2.02~beta2-36ubuntu3.2 Version table: *** 2.02~beta2-36ubuntu3.2 500 500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages 100 /var/lib/dpkg/status 2.02~beta2-36ubuntu3 500 500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages I followed https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-ZFS to make a ZFS native root disk system. Installation went fine, systems works like a champ... but then, after a while, it's time to apt-get update ; apt-get upgrade to update my kernel. That apt-get install fails because grub can't detect it's root file system: root@q-2-d.sv1:~# grub-probe / grub-probe: error: unknown filesystem. Strange, this use to work. Add in a -vv and I see: grub-core/kern/fs.c:56: Detecting zfs... grub-core/osdep/hostdisk.c:415: opening the device `/dev/sda1' in open_device() grub-core/fs/zfs/zfs.c:1192: label ok 0 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:1007: check 2 passed grub-core/fs/zfs/zfs.c:1018: check 3 passed grub-core/fs/zfs/zfs.c:1025: check 4 passed grub-core/fs/zfs/zfs.c:1035: check 6 passed grub-core/fs/zfs/zfs.c:1043: check 7 passed grub-core/fs/zfs/zfs.c:1054: check 8 passed grub-core/fs/zfs/zfs.c:1064: check 9 passed grub-core/fs/zfs/zfs.c:1086: check 11 passed grub-core/fs/zfs/zfs.c:1112: check 10 passed grub-core/fs/zfs/zfs.c:1128: str=com.delphix:hole_birth grub-core/fs/zfs/zfs.c:1128: str=com.delphix:embedded_data grub-core/fs/zfs/zfs.c:1137: check 12 passed (feature flags) grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 2048/2048 grub-core/fs/zfs/zfs.c:1898: endian = -1 grub-core/fs/zfs/zfs.c:595: dva=8, b00258 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:442: checksum fletcher4 verification failed grub-core/fs/zfs/zfs.c:447: actual checksum 00000059d95adad1 00006322fba304b3 00430f321ad7f439 21e873b51c15e3bc grub-core/fs/zfs/zfs.c:452: expected checksum 000000035ca143fb 0000062fc3a48ff8 0005b49e0d96b07e 03841abf1fed586a grub-core/fs/zfs/zfs.c:1919: incorrect checksum grub-core/kern/fs.c:78: zfs detection failed. Now here's the fun part. Rebooting the system will fix the checksum issue for a short amount of time. Here is what it looks like immediately after a reboot: root@q-1-d.sv1:~# grub-probe / zfs grub-core/kern/fs.c:56: Detecting zfs... grub-core/osdep/hostdisk.c:415: opening the device `/dev/sda1' in open_device() grub-core/fs/zfs/zfs.c:1192: label ok 0 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:1007: check 2 passed grub-core/fs/zfs/zfs.c:1018: check 3 passed grub-core/fs/zfs/zfs.c:1025: check 4 passed grub-core/fs/zfs/zfs.c:1035: check 6 passed grub-core/fs/zfs/zfs.c:1043: check 7 passed grub-core/fs/zfs/zfs.c:1054: check 8 passed grub-core/fs/zfs/zfs.c:1064: check 9 passed grub-core/fs/zfs/zfs.c:1086: check 11 passed grub-core/fs/zfs/zfs.c:1112: check 10 passed grub-core/fs/zfs/zfs.c:1128: str=com.delphix:hole_birth grub-core/fs/zfs/zfs.c:1128: str=com.delphix:embedded_data grub-core/fs/zfs/zfs.c:1137: check 12 passed (feature flags) grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 2048/2048 grub-core/fs/zfs/zfs.c:1898: endian = -1 grub-core/fs/zfs/zfs.c:595: dva=8, c001c0 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:2678: endian = -1, blkid=0 grub-core/fs/zfs/zfs.c:2020: endian = -1 grub-core/fs/zfs/zfs.c:2051: endian = -1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = -1 grub-core/fs/zfs/zfs.c:595: dva=8, c001b8 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, c001a0 grub-core/fs/zfs/zfs.c:2682: alive grub-core/fs/zfs/zfs.c:2493: looking for 'features_for_read' grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 200 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:2503: zap read grub-core/fs/zfs/zfs.c:2516: fat zap grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, bb0c48 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:2276: fzap: length 18 grub-core/fs/zfs/zfs.c:2520: returned 0 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 512/512 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 70 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:2109: zap: name = org.illumos:lz4_compress, value = 1, cd = 0 grub-core/fs/zfs/zfs.c:2109: zap: name = com.delphix:hole_birth, value = 1, cd = 0 grub-core/fs/zfs/zfs.c:2109: zap: name = com.delphix:extensible_dataset, value = 0, cd = 0 grub-core/fs/zfs/zfs.c:2109: zap: name = com.delphix:embedded_data, value = 1, cd = 0 grub-core/fs/zfs/zfs.c:2109: zap: name = org.open-zfs:large_blocks, value = 0, cd = 0 grub-core/fs/zfs/zfs.c:2109: zap: name = , value = 0, cd = 0 grub-core/fs/zfs/zfs.c:2109: zap: name = , value = 0, cd = 0 grub-core/fs/zfs/zfs.c:3250: alive grub-core/fs/zfs/zfs.c:3062: endian = 1 grub-core/fs/zfs/zfs.c:2678: endian = 1, blkid=0 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, c001b8 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, c001a0 grub-core/fs/zfs/zfs.c:2682: alive grub-core/fs/zfs/zfs.c:3069: alive grub-core/fs/zfs/zfs.c:2493: looking for 'root_dataset' grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 200 grub-core/fs/zfs/zfs.c:2503: zap read grub-core/fs/zfs/zfs.c:2516: fat zap grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, bb0c48 grub-core/fs/zfs/zfs.c:2276: fzap: length 13 grub-core/fs/zfs/zfs.c:2520: returned 0 grub-core/fs/zfs/zfs.c:3075: alive grub-core/fs/zfs/zfs.c:3081: alive grub-core/fs/zfs/zfs.c:3259: alive grub-core/fs/zfs/zfs.c:3263: endian = 0 grub-core/fs/zfs/zfs.c:3272: endian = 1 grub-core/fs/zfs/zfs.c:3127: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 2048/2048 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 60 grub-core/fs/zfs/zfs.c:3352: endian = 1 grub-core/fs/zfs/zfs.c:3127: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 2048/2048 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 60 grub-core/fs/zfs/zfs.c:2678: endian = 1, blkid=0 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 50 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 48 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 40 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 38 grub-core/osdep/hostdisk.c:394: reusing open device `/dev/sda1' grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 30 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2051: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 28 grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 16384/4096 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 20 grub-core/fs/zfs/zfs.c:2682: alive grub-core/fs/zfs/zfs.c:2493: looking for 'ROOT' grub-core/fs/zfs/zfs.c:2020: endian = 1 grub-core/fs/zfs/zfs.c:2046: endian = 1 grub-core/fs/zfs/zfs.c:1875: zio_read: E 0: size 512/512 grub-core/fs/zfs/zfs.c:1898: endian = 1 grub-core/fs/zfs/zfs.c:595: dva=8, 0 grub-core/fs/zfs/zfs.c:2503: zap read grub-core/fs/zfs/zfs.c:2507: micro zap grub-core/fs/zfs/zfs.c:2510: returned 0 zfs grub-core/kern/disk.c:295: Closing `hostdisk//dev/sda'. I see this issue across many servers that's I've done this to. Any help would be appreciated. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: grub2-common 2.02~beta2-36ubuntu3.2 ProcVersionSignature: Ubuntu 4.4.0-43.63-generic 4.4.21 Uname: Linux 4.4.0-43-generic x86_64 NonfreeKernelModules: zfs zunicode zcommon znvpair zavl ApportVersion: 2.20.1-0ubuntu2.1 Architecture: amd64 Date: Thu Oct 20 04:36:35 2016 ProcEnviron: TERM=xterm PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: grub2 UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1635115/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp