I'm seeing absolutely terrible disk performance for my virtual machines running
under KVM.
My ZFS pool is 6 Western Digital AV-25 500GB disks arranged in 3 mirrored vdevs
connected to an LSI 9210 (reflashed IBM M1015) in IT mode.
My KVM startup script:
/usr/bin/qemu-kvm \
-enable-kvm \
-smp 2 \
-m 1024 \
-no-hpet \
-rtc base=localtime,driftfix=slew \
-drive
file=/dev/zvol/dsk/oi_data/kvm/system/disk0,if=virtio,format=raw,cache=none,index=0
\
-drive
file=/dev/zvol/dsk/oi_data/kvm/system/disk1,if=virtio,format=raw,cache=none,index=1
\
-net nic,vlan=0,name=lan0,model=virtio,macaddr=$MAC0 \
-net vnic,vlan=0,name=lan0,ifname=$ETH0 \
-net nic,vlan=1,name=int0,model=virtio,macaddr=$MAC1 \
-net vnic,vlan=1,name=int0,ifname=$ETH1 \
-vnc $LAN_ADDR:$1 \
-monitor telnet:127.0.0.1:444$1,server,nowait \
-usbdevice tablet \
-daemonize
My zvol settings:
NAME PROPERTY VALUE
SOURCE
oi_data/kvm/sachiel/disk1 type volume
-
oi_data/kvm/sachiel/disk1 creation Tue Nov 8 11:56
2011 -
oi_data/kvm/sachiel/disk1 used 298G
-
oi_data/kvm/sachiel/disk1 available 844G
-
oi_data/kvm/sachiel/disk1 referenced 139G
-
oi_data/kvm/sachiel/disk1 compressratio 1.00x
-
oi_data/kvm/sachiel/disk1 reservation none
default
oi_data/kvm/sachiel/disk1 volsize 144G
local
oi_data/kvm/sachiel/disk1 volblocksize 8K
-
oi_data/kvm/sachiel/disk1 checksum on
default
oi_data/kvm/sachiel/disk1 compression off
default
oi_data/kvm/sachiel/disk1 readonly off
default
oi_data/kvm/sachiel/disk1 copies 1
default
oi_data/kvm/sachiel/disk1 refreservation 149G
local
oi_data/kvm/sachiel/disk1 primarycache all
default
oi_data/kvm/sachiel/disk1 secondarycache all
default
oi_data/kvm/sachiel/disk1 usedbysnapshots 11.8G
-
oi_data/kvm/sachiel/disk1 usedbydataset 139G
-
oi_data/kvm/sachiel/disk1 usedbychildren 0
-
oi_data/kvm/sachiel/disk1 usedbyrefreservation 147G
-
oi_data/kvm/sachiel/disk1 logbias latency
default
oi_data/kvm/sachiel/disk1 dedup off
default
oi_data/kvm/sachiel/disk1 mlslabel none
default
oi_data/kvm/sachiel/disk1 sync standard
default
oi_data/kvm/sachiel/disk1 refcompressratio 1.00x
-
Representative iostat output while moving some 4GB of digital photos (approx
2MB each):
capacity operations bandwidth
pool alloc free read write read write
------------------------- ----- ----- ----- ----- ----- -----
oi_data 373G 1019G 71 377 569K 3.98M
mirror 124G 340G 15 127 125K 1.34M
c2t50014EE2AFBDC3CBd0 - - 2 71 118K 1.34M
c2t50014EE25A683C10d0 - - 0 69 7.01K 1.34M
mirror 124G 340G 31 123 256K 1.33M
c2t50014EE20518D4F6d0 - - 4 49 132K 1.33M
c2t50014EE2AFBD87B4d0 - - 4 48 123K 1.33M
mirror 124G 340G 23 125 189K 1.32M
c2t50014EE205154E0Bd0 - - 3 52 35.1K 1.32M
c2t50014EE25A6829D6d0 - - 4 51 156K 1.32M
------------------------- ----- ----- ----- ----- ----- -----
A Bonnie result from a napp-it install:
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
2011.12.12 32G 61586 99 151550 16 95819 17 49188 99 273833 18 1166
3
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 32070 99 +++++ +++ +++++ +++ 32736 100 +++++ +++ +++++ +++
2011.12.12,32G,61586,99,151550,16,95819,17,49188,99,273833,18,1166.1,3,16,32070,99,+++++,+++,+++++,+++,32736,100,+++++,+++,+++++,+++
As you can see, it basically hovers around 4-6 MB/s average with an approximate
10KB I/O size so it appears I'm 100% iops limited. Do I need to recreate my
zvols with a larger volblocksize to get decent performance or is it something
else I need to do on the KVM config side or within the virtual machine?
I have one Windows Server 2003 (32-bit) VM and 2x Windows Server 2008 R2
(64-bit) VMs and they all behave the same way when it comes to disk perf.
Thanks,
-Russ
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
