Control: tags -1 + moreinfo On Fri, Sep 26, 2025 at 10:34:22AM +0300, Kevin Lumik wrote: > Package: src:linux > Version: 6.12.48-1 > Severity: normal > > Dear Maintainer, > > After upgrading from Debian Bookworm to Trixie, an issue within KVM guests > when creating an LVM snapshot of its volume > has surfaced. When a LVM snapshot is taken from the host, the guest starts to > get I/O errors. The issue seems to only > appear when aio=io_uring is specified in the KVM drive parameters and also > seems to resolve when downgrading the kernel > package down to 6.1. The issue is also not reproducible when using > aio=native. > > KVM args for the drive: "-drive id=drive- > virtio0,format=raw,file=/dev/dom/vps_testsql,cache=none,aio=io_uring,index=0,media=disk,if=virtio" > > An IO workload is being created in the VM using "fio --randrepeat=1 > --ioengine=io_uring --direct=1 --gtod_reduce=1 -- > name=randwrite --filename=/root/test.bin --bs=4k --iodepth=64 --runtime=60 > --numjobs=32 --readwrite=randwrite --size=1G > --rwmixread=75 --group_reporting" > > After running "lvcreate -s /dev/dom/vps_testsql -n test -L 1G" on the host we > can observe fio erroring out: > > ... > fio: io_u error on file /root/test.bin: Input/output error: write > offset=46977024, buflen=4096 > fio: pid=7367, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: io_u error on file /root/test.bin: Input/output error: write > offset=261505024, buflen=4096 > fio: pid=7374, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: io_u error on file /root/test.bin: Input/output error: write > offset=973840384, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=9637888, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=159965184, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=857505792, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=90787840, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=26427392, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=955621376, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=96169984, buflen=4096 > fio: pid=7372, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: pid=7362, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: io_u error on file /root/test.bin: Input/output error: write > offset=203702272, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=814649344, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=91467776, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=948256768, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=105295872, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=75247616, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=1062293504, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=111955968, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=942563328, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=117354496, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=1050402816, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=125419520, buflen=4096 > fio: io_u error on file /root/test.bin: Input/output error: write > offset=129044480, buflen=4096 > fio: pid=7369, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: pid=7373, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: pid=7348, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: pid=7375, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: pid=7358, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > fio: pid=7359, err=5/file:io_u.c:1876, func=io_u error, error=Input/output > error > > randwrite: (groupid=0, jobs=32): err= 5 (file:io_u.c:1876, func=io_u error, > error=Input/output error): pid=7344: Thu Sep > 25 16:28:30 2025 > write: IOPS=194k, BW=758MiB/s (795MB/s)(3976MiB/5244msec); 0 zone resets > bw ( KiB/s): min=657104, max=1009816, per=100.00%, avg=785345.80, > stdev=4331.73, samples=320 > iops : min=164276, max=252454, avg=196336.40, stdev=1082.93, > samples=320 > cpu : usr=0.33%, sys=1.07%, ctx=18912, majf=0, minf=455 > IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.8% > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, > >=64=0.0% > complete : 0=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, > >=64=0.0% > issued rwts: total=0,1019955,0,0 short=0,0,0,0 dropped=0,0,0,0 > latency : target=0, window=0, percentile=100.00%, depth=64 > > Run status group 0 (all jobs): > WRITE: bw=758MiB/s (795MB/s), 758MiB/s-758MiB/s (795MB/s-795MB/s), > io=3976MiB (4169MB), run=5244-5244msec > > Disk stats (read/write): > vda: ios=1/1001361, merge=0/2, ticks=0/10314531, in_queue=10314549, > util=97.82% > Bus error > > --- > > And the error is also visible in the kernel error log of the VM: > [ 361.962970] I/O error, dev vda, sector 83277192 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.963945] I/O error, dev vda, sector 83067480 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964489] I/O error, dev vda, sector 82881208 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964499] I/O error, dev vda, sector 83031976 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964501] I/O error, dev vda, sector 83089832 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964503] I/O error, dev vda, sector 83156184 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964522] I/O error, dev vda, sector 83183384 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964524] I/O error, dev vda, sector 83310704 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964525] I/O error, dev vda, sector 83311144 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > [ 361.964532] I/O error, dev vda, sector 83315272 op 0x1:(WRITE) flags > 0x8800 phys_seg 1 prio class 2 > > The host does not generate any erros, they only seem to occur within the VM.
Thanks for your report. Two questions: - Can you temporarily test as well the upper kernels from unstable (6.16.8-1 right now, but soonish 6.16.9-1 as well) to see if the problem is triggerable there as well? - Depending on the above question, would you be able to bisect upstream changes to determine either the introducing commit, or if we find it is fixed in 6.16.y to determine the fixing commit, so we might assess why it is missing in 6.12.y? Note: If we find the problem is prosent in 6.16.y then we need to find the breaking change between 6.1.y and 6.12.y. It might be asier here if we first make the range of potential upstream versions more narrow, so you might first do a "manual bisect" on Debian revisions (For that use the snapshot.debian.org service to retrieve linux-images uploaded to Debian. E.g. go down from 6.12 in "major version" steps until we have close enough range of broken/fixed upstream major versions. Then perform a normal bisect between the upstream versions. Let me know if you need more details on the bisection steps. Regards, Salvatore
