Package: zfsutils-linux Version: 2.0.3-1 Severity: normal Hi,
After creating a pool ('store') with a file system dataset ('store/scratch'), everything was fine to me until a reboot after which all the setting seemed to be lost. I started to address by simply running: $ zpool import store cannot import 'store': no such pool available Then I checked systemd services and found 'zfs-import-cache.service' in a failed state. So using 'journalctl' I saw the following: mars 01 13:31:26 chipiron systemd[1]: Starting Install ZFS kernel module... mars 01 13:31:26 chipiron systemd[1]: Finished Install ZFS kernel module. mars 01 13:31:26 chipiron kernel: ZFS: Loaded module v2.0.3-1, ZFS pool version 5000, ZFS filesystem version 5 mars 01 13:31:26 chipiron systemd[1]: Starting Import ZFS pools by cache file... mars 01 13:31:27 chipiron zpool[2397]: cannot import 'store': one or more devices is currently unavailable mars 01 13:31:27 chipiron systemd[1]: zfs-import-cache.service: Main process exited, code=exited, status=1/FAILURE mars 01 13:31:27 chipiron systemd[1]: zfs-import-cache.service: Failed with result 'exit-code'. mars 01 13:31:27 chipiron systemd[1]: Failed to start Import ZFS pools by cache file. mars 01 13:31:27 chipiron systemd[1]: Reached target ZFS pool import target. mars 01 13:31:27 chipiron systemd[1]: Starting Mount ZFS filesystems... mars 01 13:31:27 chipiron systemd[1]: Starting Wait for ZFS Volume (zvol) links in /dev... mars 01 13:31:27 chipiron systemd[1]: Finished Mount ZFS filesystems. Then looking at this service, I realized that 'zool import' needed to explain the cache file unlike all the other zpool commands, weird! But at least I was able to reproduced the error running: $ zpool import -c /etc/zfs/zpool.cache -aN cannot import 'store': one or more devices is currently unavailable Here is the 'zdb' output: $ zdb store: version: 5000 name: 'store' state: 0 txg: 4 pool_guid: 14836119471077797826 errata: 0 hostid: 1739059050 hostname: 'chipiron' com.delphix:has_per_vdev_zaps vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 14836119471077797826 create_txg: 4 children[0]: type: 'raidz' id: 0 guid: 9286456085254631639 nparity: 1 metaslab_array: 134 metaslab_shift: 34 ashift: 12 asize: 7201382989824 is_log: 0 create_txg: 4 com.delphix:vdev_zap_top: 129 children[0]: type: 'disk' id: 0 guid: 14694741801978214455 path: '/dev/sdb1' devid: 'scsi-35000039a38598c81-part1' phys_path: 'pci-0000:33:00.0-sas-phy2-lun-0' whole_disk: 1 create_txg: 4 com.delphix:vdev_zap_leaf: 130 children[1]: type: 'disk' id: 1 guid: 1592048366807419050 path: '/dev/sdc1' devid: 'scsi-35000039a385991e9-part1' phys_path: 'pci-0000:33:00.0-sas-phy4-lun-0' whole_disk: 1 create_txg: 4 com.delphix:vdev_zap_leaf: 131 children[2]: type: 'disk' id: 2 guid: 18150426818928785530 path: '/dev/sdd1' devid: 'scsi-35000039a38598709-part1' phys_path: 'pci-0000:33:00.0-sas-phy7-lun-0' whole_disk: 1 create_txg: 4 com.delphix:vdev_zap_leaf: 132 children[3]: type: 'disk' id: 3 guid: 11296186532713133135 path: '/dev/sde1' devid: 'scsi-35000039a385986d1-part1' phys_path: 'pci-0000:33:00.0-sas-phy5-lun-0' whole_disk: 1 create_txg: 4 com.delphix:vdev_zap_leaf: 133 features_for_read: com.delphix:hole_birth com.delphix:embedded_data Here is also the 'fdisk' output: $ LANG=C fdisk -l /dev/sdb /dev/sdc /dev/sdd /dev/sde Disk /dev/sdb: 1.64 TiB, 1800360124416 bytes, 3516328368 sectors Disk model: AL15SEB18EQY Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: 271363D9-0F42-4F4E-BDE1-C8E2190DC266 Device Start End Sectors Size Type /dev/sdb1 2048 3516311551 3516309504 1.6T Solaris /usr & Apple ZFS /dev/sdb9 3516311552 3516327935 16384 8M Solaris reserved 1 Disk /dev/sdc: 1.64 TiB, 1800360124416 bytes, 3516328368 sectors Disk model: AL15SEB18EQY Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: C41EC26F-0CB5-D946-9495-E4E0733B86AD Device Start End Sectors Size Type /dev/sdc1 2048 3516311551 3516309504 1.6T Solaris /usr & Apple ZFS /dev/sdc9 3516311552 3516327935 16384 8M Solaris reserved 1 Disk /dev/sdd: 1.64 TiB, 1800360124416 bytes, 3516328368 sectors Disk model: AL15SEB18EQY Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: 403E9EF5-20DB-BF41-ACB3-DECD5EF64884 Device Start End Sectors Size Type /dev/sdd1 2048 3516311551 3516309504 1.6T Solaris /usr & Apple ZFS /dev/sdd9 3516311552 3516327935 16384 8M Solaris reserved 1 Disk /dev/sde: 1.64 TiB, 1800360124416 bytes, 3516328368 sectors Disk model: AL15SEB18EQY Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: EFB61062-2F43-D14D-888E-79844E17A7CA Device Start End Sectors Size Type /dev/sde1 2048 3516311551 3516309504 1.6T Solaris /usr & Apple ZFS /dev/sde9 3516311552 3516327935 16384 8M Solaris reserved 1 Is there an issue behind this? Or is there something wrong on my side? Finally is there a way with 'zdb' to check the file system dataset config? (tried many options without success) Thanks, Patrice -- System Information: Debian Release: bullseye/sid APT prefers testing APT policy: (500, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 5.10.0-3-amd64 (SMP w/176 CPU threads) Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_FIRMWARE_WORKAROUND, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages zfsutils-linux depends on: ii init-system-helpers 1.60 ii libblkid1 2.36.1-7 ii libc6 2.31-9 ii libnvpair3linux 2.0.3-1 ii libuuid1 2.36.1-7 ii libuutil3linux 2.0.3-1 ii libzfs4linux 2.0.3-1 ii libzpool4linux 2.0.3-1 ii python3 3.9.1-1 Versions of packages zfsutils-linux recommends: ii lsb-base 11.1.0 ii zfs-dkms [zfs-modules] 2.0.3-1 ii zfs-zed 2.0.3-1 Versions of packages zfsutils-linux suggests: pn nfs-kernel-server <none> pn samba-common-bin <none> pn zfs-initramfs | zfs-dracut <none> -- no debconf information