Re: [kvm-devel] disappointing speed with virtio_blk
Hi Marcelo,
http://www.mail-archive.com/kvm-devel@lists.sourceforge.net/msg14732.html
I tried it this evening with kvm 66 - which should include your patch,
right?
No its not included. The issue is being worked on.
my bad, sorry.
Now I know I really have that patch: qemu-kvm hangs :(
I was trying kvm 66 with only the patch listed above applied on an otherwise
perfectly working vm with virtio_blk root partition:
Last line of the booting kernel in my vnc window:
Serial: 8250/16550 driver $Revision 1.90... (you know the rest)
an strace of the qemu-kvm gave the following in rapid succession:
clock_gettime(CLOCK_MONOTONIC, {2565, 306799672}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307065342}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307354930}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307618803}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307886312}) = 0
timer_gettime(0, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0, 0, {it_interval={0, 0}, it_value={0, 3300}}, NULL) = 0
rt_sigtimedwait([USR1 USR2 ALRM IO], {si_signo=SIGALRM, si_code=SI_TIMER,
si_pid=0, si_uid=0, si_value={int=0, ptr=0}}, 0xbfe5af88, 8) = 14
rt_sigaction(SIGALRM, NULL, {0x804d8f8, ~[KILL STOP RTMIN RT_1], 0}, 8) = 0
select(12, [6 11], [], [], {0, 0}) = 0 (Timeout)
select(0, [], NULL, NULL, {0, 0}) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {2565, 342895116}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 343164113}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 343454002}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 343716804}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 343980012}) = 0
timer_gettime(0, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0, 0, {it_interval={0, 0}, it_value={0, 3300}}, NULL) = 0
rt_sigtimedwait([USR1 USR2 ALRM IO], {si_signo=SIGALRM, si_code=SI_TIMER,
si_pid=0, si_uid=0, si_value={int=0, ptr=0}}, 0xbfe5af88, 8) = 14
rt_sigaction(SIGALRM, NULL, {0x804d8f8, ~[KILL STOP RTMIN RT_1], 0}, 8) = 0
select(12, [6 11], [], [], {0, 0}) = 0 (Timeout)
select(0, [], NULL, NULL, {0, 0}) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {2565, 379035364}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 379307884}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 379589434}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 379919100}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 380183834}) = 0
timer_gettime(0, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0, 0, {it_interval={0, 0}, it_value={0, 3300}}, NULL) = 0
rt_sigtimedwait([USR1 USR2 ALRM IO], {si_signo=SIGALRM, si_code=SI_TIMER,
si_pid=0, si_uid=0, si_value={int=0, ptr=0}}, 0xbfe5af88, 8) = 14
rt_sigaction(SIGALRM, NULL, {0x804d8f8, ~[KILL STOP RTMIN RT_1], 0}, 8) = 0
select(12, [6 11], [], [], {0, 0}) = 0 (Timeout)
select(0, [], NULL, NULL, {0, 0}) = 0 (Timeout)
...
Hope that helps.
Kind regards,
Gerd
--
Address (better: trap) for people I really don't want to get mail from:
[EMAIL PROTECTED]
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
Hi Gerd,
On Fri, Apr 18, 2008 at 11:27:58PM +0200, Gerd von Egidy wrote:
Hi Marcelo,
http://www.mail-archive.com/kvm-devel@lists.sourceforge.net/msg14732.html
I tried it this evening with kvm 66 - which should include your patch,
right?
No its not included. The issue is being worked on.
my bad, sorry.
Now I know I really have that patch: qemu-kvm hangs :(
I was trying kvm 66 with only the patch listed above applied on an otherwise
perfectly working vm with virtio_blk root partition:
Last line of the booting kernel in my vnc window:
Serial: 8250/16550 driver $Revision 1.90... (you know the rest)
When the hang happens, can you run kvm-stat --once (script can be found
kvm-66 directory) and paste the result?
Can you confirm that reverting the patch fixes it?
an strace of the qemu-kvm gave the following in rapid succession:
clock_gettime(CLOCK_MONOTONIC, {2565, 306799672}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307065342}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307354930}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307618803}) = 0
clock_gettime(CLOCK_MONOTONIC, {2565, 307886312}) = 0
timer_gettime(0, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0, 0, {it_interval={0, 0}, it_value={0, 3300}}, NULL) = 0
rt_sigtimedwait([USR1 USR2 ALRM IO], {si_signo=SIGALRM, si_code=SI_TIMER,
si_pid=0, si_uid=0, si_value={int=0, ptr=0}}, 0xbfe5af88, 8) = 14
rt_sigaction(SIGALRM, NULL, {0x804d8f8, ~[KILL STOP RTMIN RT_1], 0}, 8) = 0
This won't help much.
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
Hi Marcelo, virtio-blk is doing synchronous IO which blocks the guest CPU. This is especially bad for write intensive loads where the guest will hang in the host write throttling logic. In the meantime please try the following patch: http://www.mail-archive.com/kvm-devel@lists.sourceforge.net/msg14732.html Thank you for this hint. I tried it this evening with kvm 66 - which should include your patch, right? The result, at least with bonnie++, is nearly the same. I looked at the patch for the guest kernel you sent with this patch but looking at the discussion it did not work. Kind regards, Gerd -- Address (better: trap) for people I really don't want to get mail from: [EMAIL PROTECTED] - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
On Thu, Apr 17, 2008 at 01:05:50AM +0200, Gerd von Egidy wrote: Hi Marcelo, virtio-blk is doing synchronous IO which blocks the guest CPU. This is especially bad for write intensive loads where the guest will hang in the host write throttling logic. In the meantime please try the following patch: http://www.mail-archive.com/kvm-devel@lists.sourceforge.net/msg14732.html Thank you for this hint. I tried it this evening with kvm 66 - which should include your patch, right? Hi Gerd, No its not included. The issue is being worked on. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] disappointing speed with virtio_blk
Hi, I just tried the virtio block device with the intent to boost disk throughput for my vm. I ran bonnie++ -r 512 -s 2048 -u nobody -d /tmp: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP virtio_blk 2G 14274 42 20206 14 22363 37 31116 92 66731 81 140.8 13 kvm-ide 2G 26065 83 26435 28 24146 33 26587 84 57991 18 91.52 The host is a Xeon 3040, 1G RAM (I know that that is a bit few, it's just a test machine...), the vm gets 512MB of that. The data is stored on two SATA disks, mirrored (RAID1) with md, lvm running on top of that. Host and Client are running 2.6.25-0.200.rc8.git3.i686. This is a kernel from Fedora-Rawhide with kvm manually enabled by me. KVM version is 64. Especially writing seems to be slower using virtio, but reading isn't that much faster. I thought virtio would improve io speed significantly because of fewer steps needed to communicate betweeen host and client. What might be the reason that I can't see a speed boost? - Wrong setup (The virtio-client boots from /dev/vda1, so I think virtio is working) - virtio_blk is not matured/tuned enough to give a real speed boost - I'm missing a patch that is not included into 2.6.26-rc8 but can be found in kvm-git - The output of bonnie++ is bogus because timing is not that accurate within a kvm-client Any ideas welcome. Kind regards, Gerd -- Address (better: trap) for people I really don't want to get mail from: [EMAIL PROTECTED] - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
My understanding is that virtio is very latency-sensitive. Do you know whether your system is configured for high-resolution timer support, and (more to the point) which clock source KVM is using on your host? - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
Hi Charles, thanks for the quick reply. My understanding is that virtio is very latency-sensitive. Do you know whether your system is configured for high-resolution timer support, and (more to the point) which clock source KVM is using on your host? On the host: grep Clock Event Device /proc/timer_list Clock Event Device: hpet Clock Event Device: lapic Clock Event Device: lapic Within the vm: grep Clock Event Device /proc/timer_list Clock Event Device: pit Clock Event Device: lapic Host and vm use the same kernel, CONFIG_HIGH_RES_TIMERS=y How do I find out which clock source KVM is using? Kind regards, Gerd -- Address (better: trap) for people I really don't want to get mail from: [EMAIL PROTECTED] - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
On Sun, Apr 13, 2008 at 11:04:44PM +0200, Gerd von Egidy wrote: Hi, I just tried the virtio block device with the intent to boost disk throughput for my vm. I ran bonnie++ -r 512 -s 2048 -u nobody -d /tmp: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP virtio_blk 2G 14274 42 20206 14 22363 37 31116 92 66731 81 140.8 13 kvm-ide 2G 26065 83 26435 28 24146 33 26587 84 57991 18 91.5 2 The host is a Xeon 3040, 1G RAM (I know that that is a bit few, it's just a test machine...), the vm gets 512MB of that. The data is stored on two SATA disks, mirrored (RAID1) with md, lvm running on top of that. Host and Client are running 2.6.25-0.200.rc8.git3.i686. This is a kernel from Fedora-Rawhide with kvm manually enabled by me. KVM version is 64. Especially writing seems to be slower using virtio, but reading isn't that much faster. I thought virtio would improve io speed significantly because of fewer steps needed to communicate betweeen host and client. What might be the reason that I can't see a speed boost? - Wrong setup (The virtio-client boots from /dev/vda1, so I think virtio is working) - virtio_blk is not matured/tuned enough to give a real speed boost - I'm missing a patch that is not included into 2.6.26-rc8 but can be found in kvm-git - The output of bonnie++ is bogus because timing is not that accurate within a kvm-client Any ideas welcome. virtio-blk is doing synchronous IO which blocks the guest CPU. This is especially bad for write intensive loads where the guest will hang in the host write throttling logic. In the meantime please try the following patch: http://www.mail-archive.com/kvm-devel@lists.sourceforge.net/msg14732.html Some larger changes will take place to allow similar behaviour with an API for asynchronous vectored IO. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] disappointing speed with virtio_blk
Gerd von Egidy wrote: Hi, I just tried the virtio block device with the intent to boost disk throughput for my vm. We really haven't optimized virtio block yet under KVM. Most of the effort so far has been focused on virtio_net. We'll get there though in the near future. Regards, Anthony Liguori I ran bonnie++ -r 512 -s 2048 -u nobody -d /tmp: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP virtio_blk 2G 14274 42 20206 14 22363 37 31116 92 66731 81 140.8 13 kvm-ide 2G 26065 83 26435 28 24146 33 26587 84 57991 18 91.5 2 The host is a Xeon 3040, 1G RAM (I know that that is a bit few, it's just a test machine...), the vm gets 512MB of that. The data is stored on two SATA disks, mirrored (RAID1) with md, lvm running on top of that. Host and Client are running 2.6.25-0.200.rc8.git3.i686. This is a kernel from Fedora-Rawhide with kvm manually enabled by me. KVM version is 64. Especially writing seems to be slower using virtio, but reading isn't that much faster. I thought virtio would improve io speed significantly because of fewer steps needed to communicate betweeen host and client. What might be the reason that I can't see a speed boost? - Wrong setup (The virtio-client boots from /dev/vda1, so I think virtio is working) - virtio_blk is not matured/tuned enough to give a real speed boost - I'm missing a patch that is not included into 2.6.26-rc8 but can be found in kvm-git - The output of bonnie++ is bogus because timing is not that accurate within a kvm-client Any ideas welcome. Kind regards, Gerd - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
