Hi, I have been investigating strange stalls of virtual machines, and realized that the VMs were (silently) paused because qemu thinks there were I/O errors when writing to the host.
After using "werror=report,rerror=report" with "-drive" we now see actual reporting of I/O errors to the guest, where they look like this:
end_request: I/O error, dev vda, sector 7243680 EXT4-fs warning (device vda1): ext4_end_bio:259: I/O error writing to inode 951097 (offset 3096576 size 4096 starting block 905461) end_request: I/O error, dev vda, sector 22018120 Buffer I/O error on device vda1, logical block 2752009 lost page write due to I/O error on vda1 end_request: I/O error, dev vda, sector 12857032 JBD2: Detected IO errors while flushing file data on vda1-8 Aborting journal on device vda1-8.
The qemu instance in question is using an executable compiled from current sources, running on vanilla linux-4.4.2 - and qemu is started directly, not via any library or VM management framework. The guest drive parameters are:
-drive "file=image.raw,if=virtio,format=raw,media=disk,cache=unsafe,werror=report,rerror=report"
I've searched the Web and found some people reporting similar symptoms, but related to either time-outs with NFS, direct use of LVM / DRBD partitions on the host and such - these circumstances do not apply here. (We do use DRBD and LVM, but qemu is not accessing raw partitions, just an ordinary file on an XFS filesystem, and the host does not report any I/O errors on the device or filesystem layers.) There seems to be a relationship between the occurence of the I/O-errors reported to the guest and the load on the I/O system of the host - the errors become more frequent (like "once per day") when there is high load. Is there any kind of timeout or something that might make qemu assume a write operation on the host has failed? Can you provide any hint on how to pursue the cause of these errors? (I thought about using "strace -f -p ..." on qemu, but I don't know what exactly to look for in the output - some failed "pwrite()" to the image file?) Regards, Lutz Vieweg
