Re: Awful FreeBSD 9 block IO performance in KVM
On 07/22/2012 03:19 AM, Wojciech Puchar wrote: You are right. It is not capped at that speed: root@freebsd:/root # dd if=/dev/zero of=/dev/da1 bs=16384 count=262144 262144+0 records in 262144+0 records out 4294967296 bytes transferred in 615.840721 secs (6974153 bytes/sec) you did test da1 while dmesg are about da0? is it OK and da1 is another qemu-kvm vdisk? If so, check dd if=/dev/zero of=/dev/da1 bs=512 count=256k and compare speed. i bet at something near 250kB/s and i think it is long I/O service pathlength in qemu-kvm SCSI device simulator. Just my bet i don't run FreeBSD on any VM (as opposed to running Windows under FreeBSD in VBox) check out how much CPU is used on the host side when you do that test. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org The test was on da1, but that does not explain how Linux manages to get over 100MB/s when writing to a file on ext4. I will do some more comprehensive tests as soon as I find time. signature.asc Description: OpenPGP digital signature
Re: Awful FreeBSD 9 block IO performance in KVM
da0: 3.300MB/s transfers da0: Command Queueing enabled root@freebsd:/root # dd if=/dev/zero of=/dev/da1 bs=16384 count=262144 4294967296 bytes transferred in 615.840721 secs (6974153 bytes/sec) 1) Does a larger block size (bs=1m) help? 2) That's roughly the speed I'd expect without queueing. Is it really making effective use of queueing, or is something limiting queueing to one transfer at a time? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
You are right. It is not capped at that speed: root@freebsd:/root # dd if=/dev/zero of=/dev/da1 bs=16384 count=262144 262144+0 records in 262144+0 records out 4294967296 bytes transferred in 615.840721 secs (6974153 bytes/sec) you did test da1 while dmesg are about da0? is it OK and da1 is another qemu-kvm vdisk? If so, check dd if=/dev/zero of=/dev/da1 bs=512 count=256k and compare speed. i bet at something near 250kB/s and i think it is long I/O service pathlength in qemu-kvm SCSI device simulator. Just my bet i don't run FreeBSD on any VM (as opposed to running Windows under FreeBSD in VBox) check out how much CPU is used on the host side when you do that test. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
2) That's roughly the speed I'd expect without queueing. Is it really making effective use of queueing, or is something limiting queueing to one transfer at a time? still 400-500 IOPS is way too little. FreeBSD without VM machine can do well over 1 IOPS of 512 byte sequential read with single process on my laptop using less than 20% of single not blazing fast CPU. dd if=/dev/ada0 of=/dev/null bs=512 117593+0 records in 117593+0 records out 60207616 bytes transferred in 7.709451 secs (7809585 bytes/sec) CPU: Pentium(R) Dual-Core CPU T4500 @ 2.30GHz (2294.56-MHz K8-class CPU) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
- Original Message - From: Dieter BSD dieter...@engineer.com To: hack...@freebsd.org, curr...@freebsd.org Sent: Sunday, July 22, 2012 1:19:32 AM Subject: Re: Awful FreeBSD 9 block IO performance in KVM da0: 3.300MB/s transfers da0: Command Queueing enabled root@freebsd:/root # dd if=/dev/zero of=/dev/da1 bs=16384 count=262144 4294967296 bytes transferred in 615.840721 secs (6974153 bytes/sec) 1) Does a larger block size (bs=1m) help? 2) That's roughly the speed I'd expect without queueing. Is it really making effective use of queueing, or is something limiting queueing to one transfer at a time? The likely fix here is basically do vtblk_startio() in a separate kproc that vtblk_strategy() enqueues bio's to. This has been on my todo for a while, but haven't had the time. Also, the use of bioq_disksort() probably doesn't gain much for virtualized disks, but I never found much of a difference in my testing. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
kproc that vtblk_strategy() enqueues bio's to. This has been on my todo for a while, but haven't had the time. Also, the use of bioq_disksort() probably doesn't gain much for virtualized disks, definitely. it is done by the host. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
da0: QEMU QEMU HARDDISK 1.1. Fixed Direct Access SCSI-5 device da0: 3.300MB/s transfers da0: Command Queueing enabled da0: 409600MB (838860800 512 byte sectors: 255H 63S/T 52216C) It does not explain why virtio is slow though, although I still need to test virtio against the latest code. I will do ivan's raw block test against virtio-blk, mainly because there is no point in doing it against a device whose transfers have been capped to 3.3MB/sec. are you sure it is really capped? i am not. just emulated sym device reports low speed. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
On 07/21/2012 04:15 AM, Wojciech Puchar wrote: da0: QEMU QEMU HARDDISK 1.1. Fixed Direct Access SCSI-5 device da0: 3.300MB/s transfers da0: Command Queueing enabled da0: 409600MB (838860800 512 byte sectors: 255H 63S/T 52216C) It does not explain why virtio is slow though, although I still need to test virtio against the latest code. I will do ivan's raw block test against virtio-blk, mainly because there is no point in doing it against a device whose transfers have been capped to 3.3MB/sec. are you sure it is really capped? i am not. just emulated sym device reports low speed. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org You are right. It is not capped at that speed: root@freebsd:/root # dd if=/dev/zero of=/dev/da1 bs=16384 count=262144 262144+0 records in 262144+0 records out 4294967296 bytes transferred in 615.840721 secs (6974153 bytes/sec) signature.asc Description: OpenPGP digital signature
Re: Awful FreeBSD 9 block IO performance in KVM
On 19/07/2012 20:27, Richard Yao wrote: Dear Everyone, FreeBSD 9 has awful block IO performance in KVM. I have experienced it and others have experienced it. Someone posted slides to slideshare with benchmarks documenting it: http://www.slideshare.net/TakeshiHasegawa1/runningfreebsdonlinuxkvm Slides 13 and 20 are particular eye openers. Does anyone know what is wrong? I'm interested in seeing if the difference is still large while writing to a raw device (dd of=/dev/xxx bs=1m count=1000) vs writing to the file system. Can you test this? signature.asc Description: OpenPGP digital signature
Re: Awful FreeBSD 9 block IO performance in KVM
On 19 July 2012 11:27, Richard Yao r...@gentoo.org wrote: Dear Everyone, FreeBSD 9 has awful block IO performance in KVM. I have experienced it and others have experienced it. Someone posted slides to slideshare with benchmarks documenting it: http://www.slideshare.net/TakeshiHasegawa1/runningfreebsdonlinuxkvm Slides 13 and 20 are particular eye openers. Does anyone know what is wrong? For those watching at home - this is bad performance _with_ the virtio drivers themselves, not just with SCSI emulation. Slide 17 is very telling - the operation latency is quite high. Richard, are you able to easily test out things on FreeBSD-HEAD guest in a Linux KVM? If so, some of the storage/block/GEOM driver people may be able to step up and start offering some ideas. Thanks, Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
On 07/20/2012 03:44 PM, Adrian Chadd wrote: On 19 July 2012 11:27, Richard Yao r...@gentoo.org wrote: Dear Everyone, FreeBSD 9 has awful block IO performance in KVM. I have experienced it and others have experienced it. Someone posted slides to slideshare with benchmarks documenting it: http://www.slideshare.net/TakeshiHasegawa1/runningfreebsdonlinuxkvm Slides 13 and 20 are particular eye openers. Does anyone know what is wrong? For those watching at home - this is bad performance _with_ the virtio drivers themselves, not just with SCSI emulation. Slide 17 is very telling - the operation latency is quite high. Richard, are you able to easily test out things on FreeBSD-HEAD guest in a Linux KVM? If so, some of the storage/block/GEOM driver people may be able to step up and start offering some ideas. Thanks, Adrian Dear Adrian, I am in the process of setting up a VM instance specifically for this. While installing it, I noticed that qemu-kvm printed 'lsi_scsi: error: ORDERED queue not implemented', which might be a clue as to why the block device performance is bad. Also, I will try testing raw disk IO for Ivan after I have it setup. Yours truly, Richard Yao signature.asc Description: OpenPGP digital signature
Re: Awful FreeBSD 9 block IO performance in KVM
On 20 July 2012 15:26, Richard Yao r...@gentoo.org wrote: I am in the process of setting up a VM instance specifically for this. While installing it, I noticed that qemu-kvm printed 'lsi_scsi: error: ORDERED queue not implemented', which might be a clue as to why the block device performance is bad. Also, I will try testing raw disk IO for Ivan after I have it setup. Thanks for setting this up. Setting up an easily reproducible environment is by far the biggest and most helpful step here. Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Awful FreeBSD 9 block IO performance in KVM
On 07/20/2012 06:26 PM, Richard Yao wrote: On 07/20/2012 03:44 PM, Adrian Chadd wrote: On 19 July 2012 11:27, Richard Yao r...@gentoo.org wrote: Dear Everyone, FreeBSD 9 has awful block IO performance in KVM. I have experienced it and others have experienced it. Someone posted slides to slideshare with benchmarks documenting it: http://www.slideshare.net/TakeshiHasegawa1/runningfreebsdonlinuxkvm Slides 13 and 20 are particular eye openers. Does anyone know what is wrong? For those watching at home - this is bad performance _with_ the virtio drivers themselves, not just with SCSI emulation. Slide 17 is very telling - the operation latency is quite high. Richard, are you able to easily test out things on FreeBSD-HEAD guest in a Linux KVM? If so, some of the storage/block/GEOM driver people may be able to step up and start offering some ideas. Thanks, Adrian Dear Adrian, I am in the process of setting up a VM instance specifically for this. While installing it, I noticed that qemu-kvm printed 'lsi_scsi: error: ORDERED queue not implemented', which might be a clue as to why the block device performance is bad. Also, I will try testing raw disk IO for Ivan after I have it setup. Yours truly, Richard Yao I now have FreeBSD 9.1-BETA1 installed in a virtual machine. I noticed the following in dmesg which might explain why the emulated SCSI support is so slow: da0 at sym0 bus 0 scbus2 target 0 lun 0 da0: QEMU QEMU HARDDISK 1.1. Fixed Direct Access SCSI-5 device da0: 3.300MB/s transfers da0: Command Queueing enabled da0: 409600MB (838860800 512 byte sectors: 255H 63S/T 52216C) It does not explain why virtio is slow though, although I still need to test virtio against the latest code. I will do ivan's raw block test against virtio-blk, mainly because there is no point in doing it against a device whose transfers have been capped to 3.3MB/sec. signature.asc Description: OpenPGP digital signature
Awful FreeBSD 9 block IO performance in KVM
Dear Everyone, FreeBSD 9 has awful block IO performance in KVM. I have experienced it and others have experienced it. Someone posted slides to slideshare with benchmarks documenting it: http://www.slideshare.net/TakeshiHasegawa1/runningfreebsdonlinuxkvm Slides 13 and 20 are particular eye openers. Does anyone know what is wrong? Yours truly, Richard Yao signature.asc Description: OpenPGP digital signature
