Output from "apport-cli --save 5.15.0-37-generic.apport -p linux --file- bug"
** Attachment added: "5.15.0-37-generic.apport" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1978347/+attachment/5597248/+files/5.15.0-37-generic.apport -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1978347 Title: horrible IO degradation with encrypted zfs root on kernels past 5.15.0-27 Status in linux package in Ubuntu: Confirmed Bug description: IO on encrypted zfs root has fallen off a cliff in kernel versions after 5.15.0-27 (the degradation is observed since version 5.15.0-30, also seen on -33 and -37, -25 and -27 work like a charm). Heavy usage almost hangs a new laptop (building large singularity images or synthetic testing with dd). I have confirmed that things are working as expected on a default + upgraded Ubuntu 22.04 desktop LVM+LUKS installation using another NVMe SSD on the same laptop. There seems to be a regression when using the native zfs encryption (did aes-ni acceleration get turned off?) How to reproduce: - install ubuntu 22.04 desktop from iso, don't install web updates, check use zfs and encryption - sudo apt update && apt install dstat htop - create a dataset with compression disabled so that dd actually writes things to disk * sudo zfs create rpool/dummy * sudo zfs set compress=off rpool/dummy * sudo chown -R myusername. /dummy - start dstat and htop in the background (show kernel threads in the htop config) - dd if=/dev/zero of=/dummy/bigfile bs=1M count=16384 - sudo apt upgrade and reboot on the latest kernel, repeat Expected: some cpu load, dstat reports write speeds about as much as the SSD can sustain (2.9-3GiB/s with a 2TiB Samsung 970 EVO Plus for a 16GiB write test, 1.4GiB/s for a few seconds then 800MiB/s sustained for whatever WD 512GiB model I had laying around). Observed on versions -30 and later: 700% or more system cpu load, mostly in z_wr_iss threads, writes top at around 150-180MiB/s, the system becomes somewhat unresponsive. Reads are also not good but I have not benchmarked. Booting the system and launching apps seems about normal due to the low IO load. ProblemType: Bug DistroRelease: Ubuntu 22.04 Package: linux-image-5.15.0-37-generic 5.15.0-37.39 ProcVersionSignature: Ubuntu 5.15.0-37.39-generic 5.15.35 Uname: Linux 5.15.0-37-generic x86_64 NonfreeKernelModules: zfs zunicode zcommon znvpair zavl icp ApportVersion: 2.20.11-0ubuntu82.1 Architecture: amd64 AudioDevicesInUse: USER PID ACCESS COMMAND /dev/snd/controlC0: laperlej 4100 F.... pulseaudio CRDA: N/A CasperMD5CheckResult: pass CurrentDesktop: ubuntu:GNOME Date: Fri Jun 10 15:52:12 2022 HibernationDevice: RESUME=none InstallationDate: Installed on 2022-05-10 (31 days ago) InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Release amd64 (20220419) MachineType: HP HP EliteBook 850 G8 Notebook PC ProcFB: 0 i915drmfb ProcKernelCmdLine: BOOT_IMAGE=/BOOT/ubuntu_rgwvzq@/vmlinuz-5.15.0-37-generic root=ZFS=rpool/ROOT/ubuntu_rgwvzq ro quiet splash vt.handoff=1 RelatedPackageVersions: linux-restricted-modules-5.15.0-37-generic N/A linux-backports-modules-5.15.0-37-generic N/A linux-firmware 20220329.git681281e4-0ubuntu3.2 SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 01/11/2022 dmi.bios.release: 8.0 dmi.bios.vendor: HP dmi.bios.version: T76 Ver. 01.08.00 dmi.board.name: 8846 dmi.board.vendor: HP dmi.board.version: KBC Version 30.37.00 dmi.chassis.type: 10 dmi.chassis.vendor: HP dmi.ec.firmware.release: 48.55 dmi.modalias: dmi:bvnHP:bvrT76Ver.01.08.00:bd01/11/2022:br8.0:efr48.55:svnHP:pnHPEliteBook850G8NotebookPC:pvr:rvnHP:rn8846:rvrKBCVersion30.37.00:cvnHP:ct10:cvr:sku4V1S3UP#ABL: dmi.product.family: 103C_5336AN HP EliteBook dmi.product.name: HP EliteBook 850 G8 Notebook PC dmi.product.sku: 4V1S3UP#ABL dmi.sys.vendor: HP To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1978347/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
