I/O Error Test 3
================
commit "bcache: fix inaccurate io state for detached bcache devices"
Probçem: on detached bcache devices, iostat reports non-zero 'avgqu-sz'
and '%util' even after IO stopped.
Original kernel: iostat fields are non-zero on I/O idle.
Modified kernel: iostat fields are all zero on I/O idle.
Original
--------
# uname -rv
4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019
# ./setup.sh >/dev/null 2>&1
[ 118.371172] bcache: register_bdev() registered backing device dm-0
[ 118.377221] bcache: run_cache_set() invalidating existing data
[ 118.386614] bcache: register_cache() registered cache device dm-1
[ 120.373682] bcache: bch_cached_dev_attach() Caching dm-0 as bcache0 on set
4f2b6518-1a58-42e0-8720-42b535cb826f
# lsblk -e 252
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
└─fake-loop0 253:0 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
loop1 7:1 0 1G 0 loop
└─fake-loop1 253:1 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
# echo 1 > /sys/block/bcache0/bcache/detach
[ 131.847360] bcache: cached_dev_detach_finish() Caching disabled for dm-0
# lsblk -e 252
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
└─fake-loop0 253:0 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
loop1 7:1 0 1G 0 loop
└─fake-loop1 253:1 0 1024M 0 dm
# dd if=/dev/zero of=/dev/bcache0 bs=4k
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 3.86316 s, 278 MB/s
# sync
# iostat -xy 1 10 /dev/bcache0 | grep bcache0
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 262224.00 0.00 0.00 0.00 100.00
Modified
--------
# uname -rv
4.15.0-55-generic #60+test20190703build1bcache1-Ubuntu SMP Wed Jul 3 21:41:37
UTC
# ./setup.sh >/dev/null 2>&1
[ 26.923296] bcache: register_bdev() registered backing device dm-0
[ 26.928998] bcache: run_cache_set() invalidating existing data
[ 26.940144] bcache: register_cache() registered cache device dm-1
[ 28.920772] bcache: bch_cached_dev_attach() Caching dm-0 as bcache0 on set
bf256397-22a4-40ac-8ff3-0026fd28354c
# lsblk -e 252
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
└─fake-loop0 253:0 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
loop1 7:1 0 1G 0 loop
└─fake-loop1 253:1 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
# echo 1 > /sys/block/bcache0/bcache/detach
[ 42.292538] bcache: cached_dev_detach_finish() Caching disabled for dm-0
# lsblk -e 252
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
└─fake-loop0 253:0 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
loop1 7:1 0 1G 0 loop
└─fake-loop1 253:1 0 1024M 0 dm
# dd if=/dev/zero of=/dev/bcache0 bs=4k
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 3.86123 s, 278 MB/s
# sync
# iostat -xy 1 10 /dev/bcache0 | grep bcache0
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
bcache0 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1829563
Title:
bcache: risk of data loss on I/O errors in backing or caching devices
Status in linux package in Ubuntu:
Invalid
Status in linux source package in Bionic:
In Progress
Status in linux source package in Cosmic:
In Progress
Bug description:
[Impact]
* The bcache code in Bionic lacks several fixes to handle
I/O errors in both backing devices and caching devices.
* Partial or permanent errors in backing or caching devices,
specially in writeback mode, can lead to data loss and/or
the application is not notified about failed I/O requests.
* The bcache device might remain available for I/O requests
even if backing device is offline, so writes are undefined.
[Test Case]
* Detailed test cases/steps for the behavior of almost every
patch with code logic changes are provided in bug comments.
* The patchset has been tested for regressions on each cache
mode (writethrough, writeback, writearound, none) with the
xfstests test suite (on ext4), fio (random read-write) and
iozone (several read/write tests).
[Regression Potential]
* The patchset is relatively large and touches several areas
in bcache code, however, synthetic testing of the patches
has been performed, and extensive regression/stress tests
were run (as mentioned in Test Case section).
* Many patches in the patchset are 'Fixes' patches to other
patches, and no further 'Fixes' currently exist upstream.
[Other Info]
* Canonical Field Eng. deploys bcache+writeback extensively
(e.g., BootStack, UA cloud, except rare all-flash cases).
[Original Bug Description]
This is a request for a backport of the following upstream patch from
4.18:
"bcache: stop bcache device when backing device is offline"
https://github.com/torvalds/linux/commit/0f0709e6bfc3ce4e8e1c0e8573490c45f76cfeee
Field engineering uses bcache quite extensively and it would be good
to have this in the GA/bionic kernel.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829563/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
