I/O Error Test 4
================

commit "bcache: add io_disable to struct cached_dev"

Problem: in case of the backing device hits I/o errors or is
disconected, the I/O can still be accepted to the bcache device.

Original kernel: dd writes in writeback mode to failed backing device complete.
Modified kernel: the bcache0 device is removed after some I/O errors in backing 
device.


Original
--------

# uname -rv
4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019


# ./setup.sh >/dev/null 2>&1                                                    
                                        
[   24.820401] bcache: register_bdev() registered backing device dm-0           
                                                            
[   24.833268] bcache: run_cache_set() invalidating existing data
[   24.848314] bcache: register_cache() registered cache device dm-1
[   26.824645] bcache: bch_cached_dev_attach() Caching dm-0 as bcache0 on set 
dd465fa7-4e85-484b-89dd-353c24c6b041

# echo writeback > /sys/block/bcache0/bcache/cache_mode

# cat /sys/block/bcache0/bcache/cache_mode
writethrough [writeback] writearound none

# ./dm_fake_dev.sh /dev/loop0 bad
[   41.439684] Buffer I/O error on dev dm-0, logical block 262128, async page 
read
[   41.445284] Buffer I/O error on dev dm-0, logical block 262128, async page 
read
[   41.451846] bcache: register_bcache() error /dev/dm-0: device already 
registered (emitting change event)
[   41.454704] Buffer I/O error on dev bcache0, logical block 262112, async 
page read
[   41.457685] Buffer I/O error on dev bcache0, logical block 262112, async 
page read
[   41.457743] Buffer I/O error on dev bcache0, logical block 1, async page read

# dd if=/dev/zero of=/dev/bcache0 bs=4k
[   49.048036] Buffer I/O error on dev bcache0, logical block 0, lost async 
page write
[   49.051702] Buffer I/O error on dev bcache0, logical block 1, lost async 
page write
[   49.054062] Buffer I/O error on dev bcache0, logical block 2, lost async 
page write
[   49.056466] Buffer I/O error on dev bcache0, logical block 3, lost async 
page write
[   49.058867] Buffer I/O error on dev bcache0, logical block 4, lost async 
page write
[   49.072020] Buffer I/O error on dev bcache0, logical block 5, lost async 
page write
[   49.074440] Buffer I/O error on dev bcache0, logical block 6, lost async 
page write
[   49.078658] Buffer I/O error on dev bcache0, logical block 7, lost async 
page write
[   49.079008] Buffer I/O error on dev bcache0, logical block 6834, lost async 
page write
[   49.079022] Buffer I/O error on dev bcache0, logical block 6835, lost async 
page write
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.58342 s, 416 MB/s

# dd if=/dev/zero of=/dev/bcache0 bs=4k
[   62.696034] buffer_io_error: 260992 callbacks suppressed
[   62.696037] Buffer I/O error on dev bcache0, logical block 0, lost async 
page write
[   62.701996] Buffer I/O error on dev bcache0, logical block 1, lost async 
page write
[   62.704394] Buffer I/O error on dev bcache0, logical block 2, lost async 
page write
[   62.706763] Buffer I/O error on dev bcache0, logical block 3, lost async 
page write
[   62.716025] Buffer I/O error on dev bcache0, logical block 4, lost async 
page write
[   62.718421] Buffer I/O error on dev bcache0, logical block 5, lost async 
page write
[   62.720821] Buffer I/O error on dev bcache0, logical block 6, lost async 
page write
[   62.723193] Buffer I/O error on dev bcache0, logical block 7, lost async 
page write
[   62.725584] Buffer I/O error on dev bcache0, logical block 8, lost async 
page write
[   62.725763] Buffer I/O error on dev bcache0, logical block 5405, lost async 
page write
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.88915 s, 372 MB/s

# dd if=/dev/zero of=/dev/bcache0 bs=4k
[   67.700114] buffer_io_error: 290043 callbacks suppressed
[   67.700117] Buffer I/O error on dev bcache0, logical block 40750, lost async 
page write
[   67.706230] Buffer I/O error on dev bcache0, logical block 40751, lost async 
page write
[   67.709846] Buffer I/O error on dev bcache0, logical block 40752, lost async 
page write
[   67.713503] Buffer I/O error on dev bcache0, logical block 40753, lost async 
page write
[   67.717241] Buffer I/O error on dev bcache0, logical block 40754, lost async 
page write
[   67.720938] Buffer I/O error on dev bcache0, logical block 40755, lost async 
page write
[   67.741395] Buffer I/O error on dev bcache0, logical block 40756, lost async 
page write
dd: error writing '/dev/bcache0': No space left on device
[   67.748145] Buffer I/O error on dev bcache0, logical block 40757, lost async 
page write
[   67.752352] Buffer I/O error on dev bcache0, logical block 41038, lost async 
page write
[   67.756642] Buffer I/O error on dev bcache0, logical block 41313, lost async 
page write
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.99741 s, 358 MB/s

# lsblk -e 252
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0          7:0    0    1G  0 loop
loop1          7:1    0    1G  0 loop
└─fake-loop1 253:1    0 1024M  0 dm
  └─bcache0  251:0    0 1024M  0 disk
fake-loop0   253:0    0    1G  0 dm
└─bcache0    251:0    0 1024M  0 disk


Modified
--------

# uname -rv
4.15.0-55-generic #60+test20190703build1bcache1-Ubuntu SMP Wed Jul 3 21:41:37 
UTC

# ./setup.sh >/dev/null 2>&1                                                    
[   22.202972] bcache: run_cache_set() invalidating existing data               
                    
[   22.213346] bcache: register_cache() registered cache device dm-1            
                    
[   22.226165] bcache: register_bdev() registered backing device dm-0           
                    
[   24.198940] bcache: bch_cached_dev_attach() Caching dm-0 as bcache0 on set 
cabfdb60-4301-46e0-940a-eb96e801c816

# echo writeback > /sys/block/bcache0/bcache/cache_mode                         
# cat /sys/block/bcache0/bcache/cache_mode                                      
writethrough [writeback] writearound none                                       
                    

# ./dm_fake_dev.sh /dev/loop0 bad                                               
[   40.025536] Buffer I/O error on dev dm-0, logical block 262128, async page 
read                  
[   40.030156] Buffer I/O error on dev dm-0, logical block 262128, async page 
read                  
[   40.035808] bcache: register_bcache() error /dev/dm-0: device already 
registered (emitting change event)
[   40.038534] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   40.038567] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   40.038574] Buffer I/O error on dev bcache0, logical block 262112, async 
page read               
[   40.041268] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   40.041284] Buffer I/O error on dev bcache0, logical block 262112, async 
page read               
[   40.041319] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   40.041341] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   40.041346] Buffer I/O error on dev bcache0, logical block 1, async page 
read                    

# dd if=/dev/zero of=/dev/bcache0 bs=4k                                         
[   48.178854] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.181495] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.183988] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.186469] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.188962] Buffer I/O error on dev bcache0, logical block 0, lost async 
page write              
[   48.191117] Buffer I/O error on dev bcache0, logical block 1, lost async 
page write
[   48.193295] Buffer I/O error on dev bcache0, logical block 2, lost async 
page write
[   48.195457] Buffer I/O error on dev bcache0, logical block 3, lost async 
page write
[   48.197607] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.200116] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.202597] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.205087] Buffer I/O error on dev bcache0, logical block 4, lost async 
page write
[   48.207229] Buffer I/O error on dev bcache0, logical block 5, lost async 
page write
[   48.209377] Buffer I/O error on dev bcache0, logical block 6, lost async 
page write
...
[   48.362824] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.365085] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.367294] bcache: bch_count_backing_io_errors() dm-0: IO error on backing 
device, unrecoverable
[   48.369457] bcache: bch_cached_dev_error() stop bcache0: too many IO errors 
on backing device dm-0
[   48.369457]
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in[   48.866726] bcache: bcache_device_free() bcache0 stopped

262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.27785 s, 471 MB/s

# lsblk -e 252
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0          7:0    0    1G  0 loop 
loop1          7:1    0    1G  0 loop 
└─fake-loop1 253:1    0 1024M  0 dm   
fake-loop0   253:0    0    1G  0 dm   

# ls /dev/bcache0
ls: cannot access '/dev/bcache0': No such file or directory

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1829563

Title:
  bcache: risk of data loss on I/O errors in backing or caching devices

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829563/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to