Hi Stefan, thanks for reply.
Unfortunately it didn't work.
disk config:
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='directsync' io='native'
discard='unmap'/>
<auth username='compute'>
<secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
</auth>
<source protocol='rbd'
name='volumes-nvme/volume-ce247187-a625-49f1-bacd-fc03df215395'>
<host name='172.31.0.1' port='6789'/>
<host name='172.31.0.2' port='6789'/>
<host name='172.31.0.3' port='6789'/>
</source>
<target dev='sdb' bus='scsi'/>
<serial>ce247187-a625-49f1-bacd-fc03df215395</serial>
<address type='drive' controller='0' bus='0' target='0' unit='2'/>
</disk>
Controller config:
<controller type='scsi' index='0' model='virtio-scsi'>
<driver queues='8'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
</controller>
benchmark command: fio --randrepeat=1 --ioengine=libaio --direct=1
--name=test --filename=test --bs=4k --iodepth=64 --size=1G
--readwrite=randwrite --time_based --runtime=60
--write_iops_log=write_results --numjobs=8
And I'm still getting very low random write IOPS inside KVM instance with
8vcores(3-5k compared to 20k+ outside KVM)
Maybe do You have any idea how to deal with it?
wt., 26 cze 2018 o 09:37 Stefan Kooman <ste...@bit.nl> napisał(a):
> Quoting Damian Dabrowski (scoot...@gmail.com):
> > Hello,
> >
> > When I mount rbd image with -o queue_depth=1024 I can see much
> improvement,
> > generally on writes(random write improvement from 3k IOPS on standard
> > queue_depth to 24k IOPS on queue_depth=1024).
> >
> > But is there any way to attach rbd disk to KVM instance with custom
> > queue_depth? I can't find any information about it.
>
> Not specifically "queue depth", but if you use virtio-scsi, and have a
> VM with more than 1 vCPU, you can give each vCPU it's own queue [1]:
>
> <controller type='scsi' index='0' model='virtio-scsi'>
> <driver queues='N' />
> </controller>
>
>
> Gr. Stefan
>
> [1]:
> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-blockio-techniques
>
>
> --
> | BIT BV http://www.bit.nl/ Kamer van Koophandel 09090351
> | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
