Re: [ovirt-users] gluster performance oVirt 3.4
Citējot Sahina Bose sab...@redhat.com : On 05/13/2014 07:27 PM, Vadims Korsaks wrote: Citējot Humble Devassy Chirammal humble.deva...@gmail.com : | | Citējot Vijay Bellur vbel...@redhat.com : | On 05/11/2014 02:04 AM, Vadims Korsaks wrote: | HI! | | Created 2 node setup with oVirt 3.4 and | CentOS 6.5, for storage created | 2 node replicated gluster (3.5) fs on same | hosts with oVirt. | mount looks like this: | 127.0.0.1:/gluster01 on | | /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 | type fuse.glusterfs | | (rw,default_permissions,allow_other,max_read=131072) | | when i making gluster test with dd, something | like | dd if=/dev/zero bs=1M count=2 | | of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka | i'm gettting speed ~ 110 MB/s, so this is | 1Gbps speed of ethernet adapter | | but with in VM created in oVirt speed is | lower than 20 MB/s | | why there is so huge difference? | how can improve VMs disks speed? | | | What are your gluster volume settings? Have you | applied the following | performance tunables in gluster's virt profile: | | eager-lock=enable | remote-dio=enable | | Regards, | Vijay | | setting were: | [root@centos155 ~]# gluster volume info gluster01 | | Volume Name: gluster01 | Type: Replicate | Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 | Status: Started | Number of Bricks: 1 x 2 = 2 | Transport-type: tcp | Bricks: | Brick1: 10.2.75.152:/mnt/gluster01/brick | Brick2: 10.2.75.155:/mnt/gluster01/brick | Options Reconfigured: | storage.owner-gid: 36 | storage.owner-uid: 36 | | | add your settings settings now it looks | | [root@centos155 ~]# gluster volume info gluster01 | | Volume Name: gluster01 | Type: Replicate | Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 | Status: Started | Number of Bricks: 1 x 2 = 2 | Transport-type: tcp | Bricks: | Brick1: 10.2.75.152:/mnt/gluster01/brick | Brick2: 10.2.75.155:/mnt/gluster01/brick | Options Reconfigured: | network.remote-dio: enable | cluster.eager-lock: enable | storage.owner-gid: 36 | storage.owner-uid: 36 | | | but this didn't affected performace in any big way | should hosts to be restarted? | glusterfs storage domain configuration GUI got Optimize for virt. store option which have to be enabled when configuring for virt store. Ref#http://www.ovirt.org/Features/GlusterFS_Storage_Domain If the configuration is manual, you need to set the group to 'virt' as shown below #gluster volume set VOLNAME group virt This will enable below options on gluster volume : quick-read=off read-ahead=off io-cache=off stat-prefetch=off eager-lock=enable remote-dio=on Can you please make sure group has set properly ? Also, invoke dd with oflag=direct option and check whether it helps .. --Humble tnx a lot!! now it's much better - from VM i can get dd with ~ 60 MB/s this is still ~ x2 lower than from host, but x3 better than it was before :) BTW could not found GUI got Optimize for virt. store option in oVirt 3.5 The option Optimize for Virt store is available when you select a volume in Ovirt - both as a right click menu option as well as a button in the top sub navigation bar. You also can check this option while creating a gluster volume using the oVirt GUI i have glusterfs as my master storage, but there is nothing in volumes, empty. And i can't create gluster volume, there is no choise in Data Center and Volume Cluster. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] gluster performance oVirt 3.4
Citējot Sahina Bose sab...@redhat.com: On 05/14/2014 02:36 PM, Vadims Korsaks wrote: Citējot Sahina Bose sab...@redhat.com : On 05/13/2014 07:27 PM, Vadims Korsaks wrote: Citējot Humble Devassy Chirammal humble.deva...@gmail.com : Citējot Vijay Bellur vbel...@redhat.com : On 05/11/2014 02:04 AM, Vadims Korsaks wrote: HI! Created 2 node setup with oVirt 3.4 and CentOS 6.5, for storage created 2 node replicated gluster (3.5) fs on same hosts with oVirt. mount looks like this: 127.0.0.1:/gluster01 on /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) when i making gluster test with dd, something like dd if=/dev/zero bs=1M count=2 of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka i'm gettting speed ~ 110 MB/s, so this is 1Gbps speed of ethernet adapter but with in VM created in oVirt speed is lower than 20 MB/s why there is so huge difference? how can improve VMs disks speed? What are your gluster volume settings? Have you applied the following performance tunables in gluster's virt profile: eager-lock=enable remote-dio=enable Regards, Vijay setting were: [root@centos155 ~]# gluster volume info gluster01 Volume Name: gluster01 Type: Replicate Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.2.75.152:/mnt/gluster01/brick Brick2: 10.2.75.155:/mnt/gluster01/brick Options Reconfigured: storage.owner-gid: 36 storage.owner-uid: 36 add your settings settings now it looks [root@centos155 ~]# gluster volume info gluster01 Volume Name: gluster01 Type: Replicate Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.2.75.152:/mnt/gluster01/brick Brick2: 10.2.75.155:/mnt/gluster01/brick Options Reconfigured: network.remote-dio: enable cluster.eager-lock: enable storage.owner-gid: 36 storage.owner-uid: 36 but this didn't affected performace in any big way should hosts to be restarted? glusterfs storage domain configuration GUI got Optimize for virt. store option which have to be enabled when configuring for virt store. Ref#http://www.ovirt.org/Features/GlusterFS_Storage_Domain If the configuration is manual, you need to set the group to 'virt' as shown below #gluster volume set VOLNAME group virt This will enable below options on gluster volume : quick-read=off read-ahead=off io-cache=off stat-prefetch=off eager-lock=enable remote-dio=on Can you please make sure group has set properly ? Also, invoke dd with oflag=direct option and check whether it helps .. --Humble tnx a lot!! now it's much better - from VM i can get dd with ~ 60 MB/s this is still ~ x2 lower than from host, but x3 better than it was before :) BTW could not found GUI got Optimize for virt. store option in oVirt 3.5 The option Optimize for Virt store is available when you select a volume in Ovirt - both as a right click menu option as well as a button in the top sub navigation bar. You also can check this option while creating a gluster volume using the oVirt GUI i have glusterfs as my master storage, but there is nothing in volumes, empty. And i can't create gluster volume, there is no choise in Data Center and Volume Cluster. To be able to manage gluster (i.e create volume, add bricks and peer probe hosts) from Ovirt UI, the cluster should have the Enable gluster service option checked. I think you may not have enabled this when you created the cluster. To use gluster volume as storage domain, this option is not required. Hope this helps. Ok, i see, tnx a lot. any chances to change this for Default Cluster, because i have self hosted engine and just 2 hosts? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] gluster performance oVirt 3.4
Citējot Humble Devassy Chirammal humble.deva...@gmail.com : | | Citējot Vijay Bellur vbel...@redhat.com : | On 05/11/2014 02:04 AM, Vadims Korsaks wrote: |HI! | |Created 2 node setup with oVirt 3.4 and | CentOS 6.5, for storage created |2 node replicated gluster (3.5) fs on same | hosts with oVirt. |mount looks like this: |127.0.0.1:/gluster01 on | | /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 | type fuse.glusterfs | | (rw,default_permissions,allow_other,max_read=131072) | |when i making gluster test with dd, something | like |dd if=/dev/zero bs=1M count=2 | | of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka |i'm gettting speed ~ 110 MB/s, so this is | 1Gbps speed of ethernet adapter | |but with in VM created in oVirt speed is | lower than 20 MB/s | |why there is so huge difference? |how can improve VMs disks speed? | | | What are your gluster volume settings? Have you | applied the following | performance tunables in gluster's virt profile: | | eager-lock=enable | remote-dio=enable | | Regards, | Vijay | | setting were: | [root@centos155 ~]# gluster volume info gluster01 | | Volume Name: gluster01 | Type: Replicate | Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 | Status: Started | Number of Bricks: 1 x 2 = 2 | Transport-type: tcp | Bricks: | Brick1: 10.2.75.152:/mnt/gluster01/brick | Brick2: 10.2.75.155:/mnt/gluster01/brick | Options Reconfigured: | storage.owner-gid: 36 | storage.owner-uid: 36 | | | add your settings settings now it looks | | [root@centos155 ~]# gluster volume info gluster01 | | Volume Name: gluster01 | Type: Replicate | Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 | Status: Started | Number of Bricks: 1 x 2 = 2 | Transport-type: tcp | Bricks: | Brick1: 10.2.75.152:/mnt/gluster01/brick | Brick2: 10.2.75.155:/mnt/gluster01/brick | Options Reconfigured: | network.remote-dio: enable | cluster.eager-lock: enable | storage.owner-gid: 36 | storage.owner-uid: 36 | | | but this didn't affected performace in any big way | should hosts to be restarted? | glusterfs storage domain configuration GUI got Optimize for virt. store option which have to be enabled when configuring for virt store. Ref#http://www.ovirt.org/Features/GlusterFS_Storage_Domain If the configuration is manual, you need to set the group to 'virt' as shown below #gluster volume set VOLNAME group virt This will enable below options on gluster volume : quick-read=off read-ahead=off io-cache=off stat-prefetch=off eager-lock=enable remote-dio=on Can you please make sure group has set properly ? Also, invoke dd with oflag=direct option and check whether it helps .. --Humble tnx a lot!! now it's much better - from VM i can get dd with ~ 60 MB/s this is still ~ x2 lower than from host, but x3 better than it was before :) BTW could not found GUI got Optimize for virt. store option in oVirt 3.5 ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] gluster performance oVirt 3.4
Citējot Gianluca Cecchi gianluca.cec...@gmail.com : On Mon, May 12, 2014 at 2:58 PM, Tobias Honacker tob...@honacker.info wrote: I've got the same issue using the same versions as Korsaks, using CentOS 6.5, too. I could launch vms with gluster block backend driver using the standard package of libvirt from centos repo. but ovirt does not run the VM with the qemu GFAPI integration, the path of the disk using the fuse mount. [snip] or am i wrong? This same question asked many times latest months. If you search through archives we remained here: http://lists.ovirt.org/pipermail/users/2014-April/023825.html BTW, Itamar, are there any bug trackers for this still remaining problems with live migration? Could we put anything on the wiki? If so, where, so that I can fill this up? Gianluca ok i'm new with gluster, so, please, can anyone explain why dd to gluster from host itself gets 110MB/s, looks like 1Gbps ethernet adapter limitation and from VM only 20MB/s ? 5x (or more with 10Gbps) it's huge difference. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] gluster performance oVirt 3.4
On 05/11/2014 02:04 AM, Vadims Korsaks wrote: HI! Created 2 node setup with oVirt 3.4 and CentOS 6.5, for storage created 2 node replicated gluster (3.5) fs on same hosts with oVirt. mount looks like this: 127.0.0.1:/gluster01 on /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) when i making gluster test with dd, something like dd if=/dev/zero bs=1M count=2 of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka i'm gettting speed ~ 110 MB/s, so this is 1Gbps speed of ethernet adapter but with in VM created in oVirt speed is lower than 20 MB/s why there is so huge difference? how can improve VMs disks speed? What are your gluster volume settings? Have you applied the following performance tunables in gluster's virt profile: eager-lock=enable remote-dio=enable Regards, Vijay setting were:[root@centos155 ~]# gluster volume info gluster01Volume Name: gluster01Type: ReplicateVolume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6Status: StartedNumber of Bricks: 1 x 2 = 2Transport-type: tcpBricks:Brick1: 10.2.75.152:/mnt/gluster01/brickBrick2: 10.2.75.155:/mnt/gluster01/brickOptions Reconfigured:storage.owner-gid: 36storage.owner-uid: 36 add your settings settings now it looks [root@centos155 ~]# gluster volume info gluster01Volume Name: gluster01Type: ReplicateVolume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6Status: StartedNumber of Bricks: 1 x 2 = 2Transport-type: tcpBricks:Brick1: 10.2.75.152:/mnt/gluster01/brickBrick2: 10.2.75.155:/mnt/gluster01/brickOptions Reconfigured:network.remote-dio: enablecluster.eager-lock: enablestorage.owner-gid: 36storage.owner-uid: 36but this didn't affected performace in any big wayshould hosts to be restarted? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] gluster performance oVirt 3.4
Citējot Vijay Bellur vbel...@redhat.com : On 05/11/2014 02:04 AM, Vadims Korsaks wrote: HI! Created 2 node setup with oVirt 3.4 and CentOS 6.5, for storage created 2 node replicated gluster (3.5) fs on same hosts with oVirt. mount looks like this: 127.0.0.1:/gluster01 on /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) when i making gluster test with dd, something like dd if=/dev/zero bs=1M count=2 of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kaka i'm gettting speed ~ 110 MB/s, so this is 1Gbps speed of ethernet adapter but with in VM created in oVirt speed is lower than 20 MB/s why there is so huge difference? how can improve VMs disks speed? What are your gluster volume settings? Have you applied the following performance tunables in gluster's virt profile: eager-lock=enable remote-dio=enable Regards, Vijay setting were: [root@centos155 ~]# gluster volume info gluster01 Volume Name: gluster01 Type: Replicate Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.2.75.152:/mnt/gluster01/brick Brick2: 10.2.75.155:/mnt/gluster01/brick Options Reconfigured: storage.owner-gid: 36 storage.owner-uid: 36 add your settings settings now it looks [root@centos155 ~]# gluster volume info gluster01 Volume Name: gluster01 Type: Replicate Volume ID: 436edaa3-ac8b-421f-aa35-68b5bd7064b6 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 10.2.75.152:/mnt/gluster01/brick Brick2: 10.2.75.155:/mnt/gluster01/brick Options Reconfigured: network.remote-dio: enable cluster.eager-lock: enable storage.owner-gid: 36 storage.owner-uid: 36 but this didn't affected performace in any big way should hosts to be restarted? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] gluster performance oVirt 3.4
HI!Created 2 node setup with oVirt 3.4 and CentOS 6.5, for storage created 2 node replicated gluster (3.5) fs on same hosts with oVirt.mount looks like this:127.0.0.1:/gluster01 on /rhev/data-center/mnt/glusterSD/127.0.0.1:_gluster01 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)when i making gluster test with dd, something like dd if=/dev/zero bs=1M count=2 of=/rhev/data-center/mnt/glusterSD/127.0.0.1\:_gluster01/kakai'm gettting speed ~ 110 MB/s, so this is 1Gbps speed of ethernet adapterbut with in VM created in oVirt speed is lower than 20 MB/swhy there is so huge difference? how can improve VMs disks speed? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
