On Fri, Oct 25, 2013 at 05:01:54PM +0200, Jack Wang wrote: > We've seen guest block io lost in a VM.any response will be helpful > > environment is: > guest os: Ubuntu 1304 > running busy database workload with xfs on a disk export with virtio-blk > > the exported vdb has very high infight io over 300. Some times later a > lot io process in D state, looks a lot requests is lost in below storage > stack.
Is the image file on a local file system or are you using a network storage system (e.g. NFS, Gluster, Ceph, Sheepdog)? If you run "vmstat 5" inside the guest, do you see "bi"/"bo" block I/O activity? If that number is very low or zero then there may be a starvation problem. If that number is reasonable then the workload is simply bottlenecked on disk I/O. virtio-blk only has 128 descriptors available so it's not possible to have 300 requests pending at the virtio-blk layer. If you suspect QEMU, try building qemu.git/master from source in case the bug has already been fixed. If you want to trace I/O requests, you might find this blog post on writing trace analysis scripts useful: http://blog.vmsplice.net/2011/03/how-to-write-trace-analysis-scripts-for.html Stefan