Script used to tune bcache devices ** Attachment added: "tune-bcache" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1806015/+attachment/5217652/+files/tune-bcache
** Description changed: Whilst diagnosing a disk performance issue on our QA cloud, I did some performance testing of bcache fronted spindles to compare xenial (4.4 kernel) and bionic (4.14 kernel) installs on the same hardware. A vanilla install (with no tuning of bcache configuration) resulted in the following performance metrics (using the sysbench fileio rndrw benchmark): Xenial 4 threads: - 45.77 MiB/sec read - 30.52 MiB/sec write + 45.77 MiB/sec read + 30.52 MiB/sec write 48 threads (matching core count): - 138.72 MiB/sec read - 92.52 MiB/sec write + 138.72 MiB/sec read + 92.52 MiB/sec write Bionic 4 threads: - 29.51 MiB/sec - 19.67 MiB/sec + 29.51 MiB/sec + 19.67 MiB/sec 48 threads (matching core count): - 41.35 MiB/sec - 27.59 MiB/sec + 41.35 MiB/sec + 27.59 MiB/sec - - After tuning (disabling the congested_{read|write}_threshold_us and disabling sequential cutoff) + After tuning (disabling the congested_{read|write}_threshold_us and + disabling sequential cutoff) Xenial 48 threads (matching core count): - 153.60 MiB/sec - 102.40 MiB/sec + 153.60 MiB/sec + 102.40 MiB/sec Bionic 48 threads (matching core count): - 161.49 MiB/sec - 107.67 MiB/sec + 161.49 MiB/sec + 107.67 MiB/sec - suggestion is this might be something todo with the move from deadline - to cfq as the default IO scheduler between 4.4 and 4.13 but as you can - see the baseline vanilla performance is significantly slower. + suggestion is this might be something todo with the move from deadline to cfq as the default IO scheduler between 4.4 and 4.13 but as you can see + the baseline vanilla performance is significantly slower. + + As a further reference point, the IO performance on the NVMe device + supporting the bcache device is: + + 4 threads: + 554.56 MiB/sec read + 369.71 MiB/sec write + ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-image-generic 4.15.0.39.41 ProcVersionSignature: Ubuntu 4.15.0-39.42-generic 4.15.18 Uname: Linux 4.15.0-39-generic x86_64 AlsaDevices: - total 0 - crw-rw---- 1 root audio 116, 1 Nov 29 11:38 seq - crw-rw---- 1 root audio 116, 33 Nov 29 11:38 timer + total 0 + crw-rw---- 1 root audio 116, 1 Nov 29 11:38 seq + crw-rw---- 1 root audio 116, 33 Nov 29 11:38 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.5 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Date: Fri Nov 30 10:11:54 2018 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' Lsusb: - Bus 002 Device 002: ID 8087:8002 Intel Corp. - Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub - Bus 001 Device 003: ID 413c:a001 Dell Computer Corp. Hub - Bus 001 Device 002: ID 8087:800a Intel Corp. - Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 002 Device 002: ID 8087:8002 Intel Corp. + Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 001 Device 003: ID 413c:a001 Dell Computer Corp. Hub + Bus 001 Device 002: ID 8087:800a Intel Corp. + Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub MachineType: Dell Inc. PowerEdge R630 PciMultimedia: - + ProcEnviron: - TERM=xterm-256color - PATH=(custom, no user) - XDG_RUNTIME_DIR=<set> - LANG=C.UTF-8 - SHELL=/bin/bash + TERM=xterm-256color + PATH=(custom, no user) + XDG_RUNTIME_DIR=<set> + LANG=C.UTF-8 + SHELL=/bin/bash ProcFB: 0 mgadrmfb ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-39-generic root=UUID=a361a524-47eb-46c3-8a04-e5eaa65188c9 ro hugepages=103117 iommu=pt intel_iommu=on RelatedPackageVersions: - linux-restricted-modules-4.15.0-39-generic N/A - linux-backports-modules-4.15.0-39-generic N/A - linux-firmware 1.173.2 + linux-restricted-modules-4.15.0-39-generic N/A + linux-backports-modules-4.15.0-39-generic N/A + linux-firmware 1.173.2 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 11/08/2016 dmi.bios.vendor: Dell Inc. dmi.bios.version: 2.3.4 dmi.board.name: 02C2CP dmi.board.vendor: Dell Inc. dmi.board.version: A03 dmi.chassis.type: 23 dmi.chassis.vendor: Dell Inc. dmi.modalias: dmi:bvnDellInc.:bvr2.3.4:bd11/08/2016:svnDellInc.:pnPowerEdgeR630:pvr:rvnDellInc.:rn02C2CP:rvrA03:cvnDellInc.:ct23:cvr: dmi.product.name: PowerEdge R630 dmi.sys.vendor: Dell Inc. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1806015 Title: bcache: performance regression without tuning under bionic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1806015/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs