[Bug 592327] Re: KVM becomes unresponsive due to waiting for aio threads
Exact instructions on how we are running the FIO tests: cut http://freshmeat.net/projects/fio/ - testing utility # yum install libaio-devel # wget http://freshmeat.net/urls/3aa21b8c106cab742bf1f20d60629e3f # tar xvfz fio-1.38.tar.gz # cd fio-1.38 # make # make install # cd .. # mkdir -p /tmp/fio-testing/data # vi four-threads-randio.fio [global] rw=randread size=256m directory=/tmp/fio-testing/data ioengine=libaio iodepth=4 invalidate=1 direct=1 [bgwriter] rw=randwrite iodepth=32 ;[queryA] ;iodepth=1 ;ioengine=mmap ;direct=0 ;thinktime=3 ;[queryB] ;iodepth=1 ;ioengine=mmap ;direct=0 ;thinktime=5 [bgupdater] rw=randrw iodepth=16 thinktime=40 size=32m # fio four-threads-randio.fio /cut The RedHat bug is indeed private, but checking http://store.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.5/html- single/Technical_Notes/ seems to indicate that this is related. An excerpt from the technical notes: cut KVM and virtualized guests would become unresponsive due to waiting infinitely for an aio threads to return. The updated packages resolve this issue by correctly timing out threads which do not return. (BZ#525114) /cut ** Bug watch added: Red Hat Bugzilla #525114 https://bugzilla.redhat.com/show_bug.cgi?id=525114 ** Also affects: qemu-kvm (CentOS) via https://bugzilla.redhat.com/show_bug.cgi?id=525114 Importance: Unknown Status: Unknown -- KVM becomes unresponsive due to waiting for aio threads https://bugs.launchpad.net/bugs/592327 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 592327] [NEW] KVM becomes unresponsive due to waiting for aio threads
Public bug reported: Binary package hint: qemu-kvm 1) # lsb_release -rd Description:Ubuntu 10.04 LTS Release:10.04 2) # apt-cache policy qemu-kvm qemu-kvm: Installed: 0.12.3+noroms-0ubuntu9 Candidate: 0.12.3+noroms-0ubuntu9 Version table: *** 0.12.3+noroms-0ubuntu9 0 500 http://70.84.184.243/ubuntu/ lucid/main Packages 100 /var/lib/dpkg/status 3) We expect that the aio threads that do not return should time out. 4) instead, KVM and virtualized guests become unresponsive when the guest run performance tests against the libaio engine. This is reproducable 100% by running fio with the libaio engine. In some cases, the test must be run multiple times. The more vCPUs a guest instance has, the more likely it is to crash quickly. In some cases, one guest instance can also cause performance problems and crashes for other guests on the same physical hardware. The only way to regain control of KVM is to reboot the server. This appears to be the same problem addressed by the KVM package in RHEL 5.5 (BZ#525114). ProblemType: Bug DistroRelease: Ubuntu 10.04 Package: qemu-kvm 0.12.3+noroms-0ubuntu9 Uname: Linux 2.6.34-drclaw-iscsimod x86_64 Architecture: amd64 Date: Thu Jun 10 16:21:57 2010 KvmCmdLine: Error: command ['ps', '-C', 'kvm', '-F'] failed with exit code 1: UIDPID PPID CSZ RSS PSR STIME TTY TIME CMD MachineType: Dell Inc. PowerEdge R710 ProcCmdLine: BOOT_IMAGE=vds-kernel-nfs initrd=iscsimod.lzma rw root=/dev/ram0 quiet BOOTIF=01-a4-ba-db-10-a5-d0 ProcEnviron: PATH=(custom, no user) SHELL=/bin/bash ProcModules: sch_htb 12262 - - Live 0xa0032000 iscsi_tcp 7928 - - Live 0xa002a000 libiscsi_tcp 11533 - - Live 0xa0021000 libiscsi 31730 - - Live 0xa001 scsi_transport_iscsi 25224 - - Live 0xa000 SourcePackage: qemu-kvm dmi.bios.date: 10/30/2009 dmi.bios.vendor: Dell Inc. dmi.bios.version: 1.3.6 dmi.board.name: 0YDJK3 dmi.board.vendor: Dell Inc. dmi.board.version: A02 dmi.chassis.type: 23 dmi.chassis.vendor: Dell Inc. dmi.modalias: dmi:bvnDellInc.:bvr1.3.6:bd10/30/2009:svnDellInc.:pnPowerEdgeR710:pvr:rvnDellInc.:rn0YDJK3:rvrA02:cvnDellInc.:ct23:cvr: dmi.product.name: PowerEdge R710 dmi.sys.vendor: Dell Inc. ** Affects: qemu-kvm (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug lucid -- KVM becomes unresponsive due to waiting for aio threads https://bugs.launchpad.net/bugs/592327 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 592327] Re: KVM becomes unresponsive due to waiting for aio threads
** Attachment added: BootDmesg.txt http://launchpadlibrarian.net/50080644/BootDmesg.txt ** Attachment added: CurrentDmesg.txt http://launchpadlibrarian.net/50080645/CurrentDmesg.txt ** Attachment added: Dependencies.txt http://launchpadlibrarian.net/50080646/Dependencies.txt ** Attachment added: Lspci.txt http://launchpadlibrarian.net/50080647/Lspci.txt ** Attachment added: Lsusb.txt http://launchpadlibrarian.net/50080648/Lsusb.txt ** Attachment added: ProcCpuinfo.txt http://launchpadlibrarian.net/50080649/ProcCpuinfo.txt ** Attachment added: ProcInterrupts.txt http://launchpadlibrarian.net/50080650/ProcInterrupts.txt ** Attachment added: RelatedPackageVersions.txt http://launchpadlibrarian.net/50080651/RelatedPackageVersions.txt ** Attachment added: UdevDb.txt http://launchpadlibrarian.net/50080652/UdevDb.txt ** Attachment added: UdevLog.txt http://launchpadlibrarian.net/50080653/UdevLog.txt -- KVM becomes unresponsive due to waiting for aio threads https://bugs.launchpad.net/bugs/592327 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
