Errata: You need to consider what you are doing. You are piling a virtualiser (KVM) on an OS (OpenIndiana) and on top of the virtualiser you are piling another OS (windows) and after creating this pile you wonder why it won't stand stable...
Keep thing where they belong. ZFS is lighting fast when it runs on Solaris or OpenIndiana, but its performance degrades as soon as you start piling up other things on it, like a windows harddrivelayout. So when it comes to store or distribute data keep the data on the layer that is fastest, in this case the OpenIndiana OS. -----Original Message----- From: Open Indiana [mailto:openindi...@out-side.nl] Sent: woensdag 4 januari 2012 9:35 To: 'Discussion list for OpenIndiana' Subject: Re: [OpenIndiana-discuss] Appalling disk performance for KVM What you could have done was: 1. use Virtualbox for your virtual OS's 2. create a minimal C: drive for the OS only 3. create 3 iscsi drives on OpenIndiana from a ZFS drive 4. import these drives as storage in your VM's Apart from step 1 you could do it right now. I guess you will see improved performance, going up to 50MB/s -----Original Message----- From: Russell Hansen [mailto:russ...@new-swankton.net] Sent: woensdag 4 januari 2012 5:51 To: Discussion list for OpenIndiana Subject: [OpenIndiana-discuss] Appalling disk performance for KVM I'm seeing absolutely terrible disk performance for my virtual machines running under KVM. My ZFS pool is 6 Western Digital AV-25 500GB disks arranged in 3 mirrored vdevs connected to an LSI 9210 (reflashed IBM M1015) in IT mode. My KVM startup script: /usr/bin/qemu-kvm \ -enable-kvm \ -smp 2 \ -m 1024 \ -no-hpet \ -rtc base=localtime,driftfix=slew \ -drive file=/dev/zvol/dsk/oi_data/kvm/system/disk0,if=virtio,format=raw,cache=none, index=0 \ -drive file=/dev/zvol/dsk/oi_data/kvm/system/disk1,if=virtio,format=raw,cache=none, index=1 \ -net nic,vlan=0,name=lan0,model=virtio,macaddr=$MAC0 \ -net vnic,vlan=0,name=lan0,ifname=$ETH0 \ -net nic,vlan=1,name=int0,model=virtio,macaddr=$MAC1 \ -net vnic,vlan=1,name=int0,ifname=$ETH1 \ -vnc $LAN_ADDR:$1 \ -monitor telnet:127.0.0.1:444$1,server,nowait \ -usbdevice tablet \ -daemonize My zvol settings: NAME PROPERTY VALUE SOURCE oi_data/kvm/sachiel/disk1 type volume - oi_data/kvm/sachiel/disk1 creation Tue Nov 8 11:56 2011 - oi_data/kvm/sachiel/disk1 used 298G - oi_data/kvm/sachiel/disk1 available 844G - oi_data/kvm/sachiel/disk1 referenced 139G - oi_data/kvm/sachiel/disk1 compressratio 1.00x - oi_data/kvm/sachiel/disk1 reservation none default oi_data/kvm/sachiel/disk1 volsize 144G local oi_data/kvm/sachiel/disk1 volblocksize 8K - oi_data/kvm/sachiel/disk1 checksum on default oi_data/kvm/sachiel/disk1 compression off default oi_data/kvm/sachiel/disk1 readonly off default oi_data/kvm/sachiel/disk1 copies 1 default oi_data/kvm/sachiel/disk1 refreservation 149G local oi_data/kvm/sachiel/disk1 primarycache all default oi_data/kvm/sachiel/disk1 secondarycache all default oi_data/kvm/sachiel/disk1 usedbysnapshots 11.8G - oi_data/kvm/sachiel/disk1 usedbydataset 139G - oi_data/kvm/sachiel/disk1 usedbychildren 0 - oi_data/kvm/sachiel/disk1 usedbyrefreservation 147G - oi_data/kvm/sachiel/disk1 logbias latency default oi_data/kvm/sachiel/disk1 dedup off default oi_data/kvm/sachiel/disk1 mlslabel none default oi_data/kvm/sachiel/disk1 sync standard default oi_data/kvm/sachiel/disk1 refcompressratio 1.00x - Representative iostat output while moving some 4GB of digital photos (approx 2MB each): capacity operations bandwidth pool alloc free read write read write ------------------------- ----- ----- ----- ----- ----- ----- oi_data 373G 1019G 71 377 569K 3.98M mirror 124G 340G 15 127 125K 1.34M c2t50014EE2AFBDC3CBd0 - - 2 71 118K 1.34M c2t50014EE25A683C10d0 - - 0 69 7.01K 1.34M mirror 124G 340G 31 123 256K 1.33M c2t50014EE20518D4F6d0 - - 4 49 132K 1.33M c2t50014EE2AFBD87B4d0 - - 4 48 123K 1.33M mirror 124G 340G 23 125 189K 1.32M c2t50014EE205154E0Bd0 - - 3 52 35.1K 1.32M c2t50014EE25A6829D6d0 - - 4 51 156K 1.32M ------------------------- ----- ----- ----- ----- ----- ----- A Bonnie result from a napp-it install: Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP 2011.12.12 32G 61586 99 151550 16 95819 17 49188 99 273833 18 1166 3 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 32070 99 +++++ +++ +++++ +++ 32736 100 +++++ +++ +++++ +++ 2011.12.12,32G,61586,99,151550,16,95819,17,49188,99,273833,18,1166.1,3,16,32 070,99,+++++,+++,+++++,+++,32736,100,+++++,+++,+++++,+++ As you can see, it basically hovers around 4-6 MB/s average with an approximate 10KB I/O size so it appears I'm 100% iops limited. Do I need to recreate my zvols with a larger volblocksize to get decent performance or is it something else I need to do on the KVM config side or within the virtual machine? I have one Windows Server 2003 (32-bit) VM and 2x Windows Server 2008 R2 (64-bit) VMs and they all behave the same way when it comes to disk perf. Thanks, -Russ _______________________________________________ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss _______________________________________________ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss _______________________________________________ OpenIndiana-discuss mailing list OpenIndiana-discuss@openindiana.org http://openindiana.org/mailman/listinfo/openindiana-discuss