Re: [Gluster-users] client is terrible with large amount of small files
On 04/30/2015 02:32 PM, gjprabu wrote: Hi bturner, I am getting below error while adding server.event gluster v set integvol server.event-threads 3 volume set: failed: option : server.event-threads does not exist Did you mean server.gid-timeout or ...manage-gids? This option is not available in 3.6, its going to come in 3.7 Glusterfs version has been upgraded to 3.6.3 Also os kernel upgraded to 6.6 kernel Yes two brick are running in KVM and one is physical machine and we are not using thinp. Regards G.J On Thu, 30 Apr 2015 00:37:44 +0530 Ben Turnerlt;btur...@redhat.comgt; wrote - Original Message - gt; From: gjprabu lt;gjpr...@zohocorp.comgt; gt; To: A Ghoshal lt;a.ghos...@tcs.comgt; gt; Cc: gluster-users@gluster.org, gluster-users-boun...@gluster.org gt; Sent: Wednesday, April 29, 2015 9:07:07 AM gt; Subject: Re: [Gluster-users] client is terrible with large amount of small files gt; gt; Hi Ghoshal, gt; gt; Please find the details below. gt; gt; A) Glusterfs version gt; glusterfs 3.6.2 Upgrade to 3.6.3 and set client.event-threads and server.event-threads to at least 4. Here is a guide on tuning MT epoll: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html gt; gt; B) volume configuration (gluster v lt;volnamegt; info) gt; gluster volume info gt; gt; gt; Volume Name: integvol gt; Type: Replicate gt; Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492 gt; Status: Started gt; Number of Bricks: 1 x 3 = 3 gt; Transport-type: tcp gt; Bricks: gt; Brick1: integ-gluster2:/srv/sdb1/brick gt; Brick2: integ-gluster1:/srv/sdb1/brick gt; Brick3: integ-gluster3:/srv/sdb1/brick gt; gt; gt; C) host linux version gt; CentOS release 6.5 (Final) Are your bricks on LVM? Are you using thinp? If so update to the latest kernel as thinp perf was really bad in 6.5 and early 6.6 kernels. gt; gt; D) details about the kind of network you use to connect your servers making gt; up your storage pool. gt; We are connecting LAN to LAN there is no special network configuration done gt; gt; Frome client we use to mount like below gt; mount -t glusterfs gluster1:/integvol /mnt/gluster/ gt; gt; gt; Regards gt; Prabu gt; gt; gt; gt; On Wed, 29 Apr 2015 17:58:16 +0530 A Ghoshallt;a.ghos...@tcs.comgt; wrote gt; gt; gt; gt; gt; Performance would largely depend upon setup. While I cannot think of any gt; setup that would cause write to be this slow, if would help if you share the gt; following details: gt; gt; A) Glusterfs version gt; B) volume configuration (gluster v lt;volnamegt; info) gt; C) host linux version gt; D) details about the kind of network you use to connect your servers making gt; up your storage pool. gt; gt; Thanks, gt; Anirban gt; gt; gt; gt; From: gjprabu lt; gjpr...@zohocorp.com gt; gt; To: lt; gluster-users@gluster.org gt; gt; Date: 04/29/2015 05:52 PM gt; Subject: Re: [Gluster-users] client is terrible with large amount of small gt; files gt; Sent by: gluster-users-boun...@gluster.org gt; gt; gt; gt; gt; Hi Team, gt; gt; If anybody know the solution please share us. gt; gt; Regards gt; Prabu gt; gt; gt; gt; On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu lt; gjpr...@zohocorp.com gt; gt; wrote gt; Hi Team, gt; gt; We are using glusterfs newly and testing data transfer part in client using gt; fuse.glusterfs file system but it is terrible with large amount of small gt; files (Large amount of small file 150MB of size it's writing around 18min). gt; I can able copy small files and syncing between the server brick are working gt; fine but it is terrible with large amount of small files. gt; gt; if anybody please share the solution for the above issue. gt; gt; Regards gt; Prabu gt; gt; ___ gt; Gluster-users mailing list gt; Gluster-users@gluster.org gt; http://www.gluster.org/mailman/listinfo/gluster-users gt; gt; ___ gt; Gluster-users mailing list gt; Gluster-users@gluster.org gt; http://www.gluster.org/mailman/listinfo/gluster-users gt; gt; gt; =-=-= gt; Notice: The information contained in this e-mail gt; message and/or attachments to it may contain gt; confidential or privileged information. If you are gt; not the intended recipient, any dissemination, use, gt; review, distribution, printing or copying of the gt; information contained in this e-mail message gt; and/or attachments to it are strictly prohibited. If gt; you have received this communication in error, gt; please notify us by reply e-mail or telephone and gt; immediately and permanently delete the message
Re: [Gluster-users] client is terrible with large amount of small files
Hi bturner, I am getting below error while adding server.event gluster v set integvol server.event-threads 3 volume set: failed: option : server.event-threads does not exist Did you mean server.gid-timeout or ...manage-gids? Glusterfs version has been upgraded to 3.6.3 Also os kernel upgraded to 6.6 kernel Yes two brick are running in KVM and one is physical machine and we are not using thinp. Regards G.J On Thu, 30 Apr 2015 00:37:44 +0530 Ben Turnerlt;btur...@redhat.comgt; wrote - Original Message - gt; From: gjprabu lt;gjpr...@zohocorp.comgt; gt; To: A Ghoshal lt;a.ghos...@tcs.comgt; gt; Cc: gluster-users@gluster.org, gluster-users-boun...@gluster.org gt; Sent: Wednesday, April 29, 2015 9:07:07 AM gt; Subject: Re: [Gluster-users] client is terrible with large amount of small files gt; gt; Hi Ghoshal, gt; gt; Please find the details below. gt; gt; A) Glusterfs version gt; glusterfs 3.6.2 Upgrade to 3.6.3 and set client.event-threads and server.event-threads to at least 4. Here is a guide on tuning MT epoll: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html gt; gt; B) volume configuration (gluster v lt;volnamegt; info) gt; gluster volume info gt; gt; gt; Volume Name: integvol gt; Type: Replicate gt; Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492 gt; Status: Started gt; Number of Bricks: 1 x 3 = 3 gt; Transport-type: tcp gt; Bricks: gt; Brick1: integ-gluster2:/srv/sdb1/brick gt; Brick2: integ-gluster1:/srv/sdb1/brick gt; Brick3: integ-gluster3:/srv/sdb1/brick gt; gt; gt; C) host linux version gt; CentOS release 6.5 (Final) Are your bricks on LVM? Are you using thinp? If so update to the latest kernel as thinp perf was really bad in 6.5 and early 6.6 kernels. gt; gt; D) details about the kind of network you use to connect your servers making gt; up your storage pool. gt; We are connecting LAN to LAN there is no special network configuration done gt; gt; Frome client we use to mount like below gt; mount -t glusterfs gluster1:/integvol /mnt/gluster/ gt; gt; gt; Regards gt; Prabu gt; gt; gt; gt; On Wed, 29 Apr 2015 17:58:16 +0530 A Ghoshallt;a.ghos...@tcs.comgt; wrote gt; gt; gt; gt; gt; Performance would largely depend upon setup. While I cannot think of any gt; setup that would cause write to be this slow, if would help if you share the gt; following details: gt; gt; A) Glusterfs version gt; B) volume configuration (gluster v lt;volnamegt; info) gt; C) host linux version gt; D) details about the kind of network you use to connect your servers making gt; up your storage pool. gt; gt; Thanks, gt; Anirban gt; gt; gt; gt; From: gjprabu lt; gjpr...@zohocorp.com gt; gt; To: lt; gluster-users@gluster.org gt; gt; Date: 04/29/2015 05:52 PM gt; Subject: Re: [Gluster-users] client is terrible with large amount of small gt; files gt; Sent by: gluster-users-boun...@gluster.org gt; gt; gt; gt; gt; Hi Team, gt; gt; If anybody know the solution please share us. gt; gt; Regards gt; Prabu gt; gt; gt; gt; On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu lt; gjpr...@zohocorp.com gt; gt; wrote gt; Hi Team, gt; gt; We are using glusterfs newly and testing data transfer part in client using gt; fuse.glusterfs file system but it is terrible with large amount of small gt; files (Large amount of small file 150MB of size it's writing around 18min). gt; I can able copy small files and syncing between the server brick are working gt; fine but it is terrible with large amount of small files. gt; gt; if anybody please share the solution for the above issue. gt; gt; Regards gt; Prabu gt; gt; ___ gt; Gluster-users mailing list gt; Gluster-users@gluster.org gt; http://www.gluster.org/mailman/listinfo/gluster-users gt; gt; ___ gt; Gluster-users mailing list gt; Gluster-users@gluster.org gt; http://www.gluster.org/mailman/listinfo/gluster-users gt; gt; gt; =-=-= gt; Notice: The information contained in this e-mail gt; message and/or attachments to it may contain gt; confidential or privileged information. If you are gt; not the intended recipient, any dissemination, use, gt; review, distribution, printing or copying of the gt; information contained in this e-mail message gt; and/or attachments to it are strictly prohibited. If gt; you have received this communication in error, gt; please notify us by reply e-mail or telephone and gt; immediately and permanently delete the message gt; and any attachments. Thank you gt; gt; gt; gt; ___ gt; Gluster-users mailing list gt; Gluster-users@gluster.org gt; http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] client is terrible with large amount of small files
- Original Message - From: Atin Mukherjee amukh...@redhat.com To: gjprabu gjpr...@zohocorp.com Cc: Ben Turner btur...@redhat.com, gluster-users@gluster.org Sent: Thursday, April 30, 2015 7:37:19 AM Subject: Re: [Gluster-users] client is terrible with large amount of small files On 04/30/2015 03:09 PM, gjprabu wrote: Hi Amukher, How to resolve this issue, till we need to wait for 3.7 release or any work around is there. You will have to as this feature is in for 3.7. My apologies, I didn't realize that MT epoll didn't land in 3.6. If you want to test it out there is an alpha build available: http://download.gluster.org/pub/gluster/glusterfs/nightly/glusterfs-3.7/epel-6-x86_64 I wouldn't run this in production until 3.7 is released though. Again sorry for the confusion. -b RegardsPrabu On Thu, 30 Apr 2015 14:49:46 +0530 Atin Mukherjeelt;amukh...@redhat.comgt; wrote On 04/30/2015 02:32 PM, gjprabu wrote: gt; Hi bturner, gt; gt; gt; I am getting below error while adding server.event gt; gt; gluster v set integvol server.event-threads 3 gt; volume set: failed: option : server.event-threads does not exist gt; Did you mean server.gid-timeout or ...manage-gids? This option is not available in 3.6, its going to come in 3.7 gt; gt; gt; Glusterfs version has been upgraded to 3.6.3 gt; Also os kernel upgraded to 6.6 kernel gt; Yes two brick are running in KVM and one is physical machine and we are not using thinp. gt; gt; Regards gt; G.J gt; gt; gt; gt; gt; gt; On Thu, 30 Apr 2015 00:37:44 +0530 Ben Turneramp;lt;btur...@redhat.comamp;gt; wrote gt; gt; - Original Message - gt; amp;gt; From: gjprabu amp;lt;gjpr...@zohocorp.comamp;gt; gt; amp;gt; To: A Ghoshal amp;lt;a.ghos...@tcs.comamp;gt; gt; amp;gt; Cc: gluster-users@gluster.org, gluster-users-boun...@gluster.org gt; amp;gt; Sent: Wednesday, April 29, 2015 9:07:07 AM gt; amp;gt; Subject: Re: [Gluster-users] client is terrible with large amount of small files gt; amp;gt; gt; amp;gt; Hi Ghoshal, gt; amp;gt; gt; amp;gt; Please find the details below. gt; amp;gt; gt; amp;gt; A) Glusterfs version gt; amp;gt; glusterfs 3.6.2 gt; gt; Upgrade to 3.6.3 and set client.event-threads and server.event-threads to at least 4. Here is a guide on tuning MT epoll: gt; gt; https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html gt; gt; amp;gt; gt; amp;gt; B) volume configuration (gluster v amp;lt;volnameamp;gt; info) gt; amp;gt; gluster volume info gt; amp;gt; gt; amp;gt; gt; amp;gt; Volume Name: integvol gt; amp;gt; Type: Replicate gt; amp;gt; Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492 gt; amp;gt; Status: Started gt; amp;gt; Number of Bricks: 1 x 3 = 3 gt; amp;gt; Transport-type: tcp gt; amp;gt; Bricks: gt; amp;gt; Brick1: integ-gluster2:/srv/sdb1/brick gt; amp;gt; Brick2: integ-gluster1:/srv/sdb1/brick gt; amp;gt; Brick3: integ-gluster3:/srv/sdb1/brick gt; amp;gt; gt; amp;gt; gt; amp;gt; C) host linux version gt; amp;gt; CentOS release 6.5 (Final) gt; gt; Are your bricks on LVM? Are you using thinp? If so update to the latest kernel as thinp perf was really bad in 6.5 and early 6.6 kernels. gt; gt; amp;gt; gt; amp;gt; D) details about the kind of network you use to connect your servers making gt; amp;gt; up your storage pool. gt; amp;gt; We are connecting LAN to LAN there is no special network configuration done gt; amp;gt; gt; amp;gt; Frome client we use to mount like below gt; amp;gt; mount -t glusterfs gluster1:/integvol /mnt/gluster/ gt; amp;gt; gt; amp;gt; gt; amp;gt; Regards gt; amp;gt; Prabu gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; On Wed, 29 Apr 2015 17:58:16 +0530 A Ghoshalamp;lt;a.ghos...@tcs.comamp;gt; wrote gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; Performance would largely depend upon setup. While I cannot think of any gt; amp;gt; setup that would cause write to be this slow, if would help if you share the gt; amp;gt; following details: gt; amp;gt; gt; amp;gt; A) Glusterfs version gt; amp;gt; B) volume configuration (gluster v amp;lt;volnameamp;gt; info) gt; amp;gt; C) host linux version gt; amp;gt; D) details about the kind of network you use to connect your servers making gt; amp;gt; up your storage pool. gt; amp;gt; gt; amp;gt; Thanks, gt; amp;gt; Anirban gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; From: gjprabu amp;lt; gjpr...@zohocorp.com amp;gt; gt; amp;gt; To: amp;lt; gluster-users@gluster.org amp;gt; gt; amp;gt; Date: 04/29/2015 05:52 PM gt; amp;gt; Subject: Re: [Gluster-users] client is terrible with large amount of small gt; amp;gt; files gt; amp;gt; Sent by:
Re: [Gluster-users] client is terrible with large amount of small files
On 04/30/2015 03:09 PM, gjprabu wrote: Hi Amukher, How to resolve this issue, till we need to wait for 3.7 release or any work around is there. You will have to as this feature is in for 3.7. RegardsPrabu On Thu, 30 Apr 2015 14:49:46 +0530 Atin Mukherjeelt;amukh...@redhat.comgt; wrote On 04/30/2015 02:32 PM, gjprabu wrote: gt; Hi bturner, gt; gt; gt; I am getting below error while adding server.event gt; gt; gluster v set integvol server.event-threads 3 gt; volume set: failed: option : server.event-threads does not exist gt; Did you mean server.gid-timeout or ...manage-gids? This option is not available in 3.6, its going to come in 3.7 gt; gt; gt; Glusterfs version has been upgraded to 3.6.3 gt; Also os kernel upgraded to 6.6 kernel gt; Yes two brick are running in KVM and one is physical machine and we are not using thinp. gt; gt; Regards gt; G.J gt; gt; gt; gt; gt; gt; On Thu, 30 Apr 2015 00:37:44 +0530 Ben Turneramp;lt;btur...@redhat.comamp;gt; wrote gt; gt; - Original Message - gt; amp;gt; From: gjprabu amp;lt;gjpr...@zohocorp.comamp;gt; gt; amp;gt; To: A Ghoshal amp;lt;a.ghos...@tcs.comamp;gt; gt; amp;gt; Cc: gluster-users@gluster.org, gluster-users-boun...@gluster.org gt; amp;gt; Sent: Wednesday, April 29, 2015 9:07:07 AM gt; amp;gt; Subject: Re: [Gluster-users] client is terrible with large amount of small files gt; amp;gt; gt; amp;gt; Hi Ghoshal, gt; amp;gt; gt; amp;gt; Please find the details below. gt; amp;gt; gt; amp;gt; A) Glusterfs version gt; amp;gt; glusterfs 3.6.2 gt; gt; Upgrade to 3.6.3 and set client.event-threads and server.event-threads to at least 4. Here is a guide on tuning MT epoll: gt; gt; https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html gt; gt; amp;gt; gt; amp;gt; B) volume configuration (gluster v amp;lt;volnameamp;gt; info) gt; amp;gt; gluster volume info gt; amp;gt; gt; amp;gt; gt; amp;gt; Volume Name: integvol gt; amp;gt; Type: Replicate gt; amp;gt; Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492 gt; amp;gt; Status: Started gt; amp;gt; Number of Bricks: 1 x 3 = 3 gt; amp;gt; Transport-type: tcp gt; amp;gt; Bricks: gt; amp;gt; Brick1: integ-gluster2:/srv/sdb1/brick gt; amp;gt; Brick2: integ-gluster1:/srv/sdb1/brick gt; amp;gt; Brick3: integ-gluster3:/srv/sdb1/brick gt; amp;gt; gt; amp;gt; gt; amp;gt; C) host linux version gt; amp;gt; CentOS release 6.5 (Final) gt; gt; Are your bricks on LVM? Are you using thinp? If so update to the latest kernel as thinp perf was really bad in 6.5 and early 6.6 kernels. gt; gt; amp;gt; gt; amp;gt; D) details about the kind of network you use to connect your servers making gt; amp;gt; up your storage pool. gt; amp;gt; We are connecting LAN to LAN there is no special network configuration done gt; amp;gt; gt; amp;gt; Frome client we use to mount like below gt; amp;gt; mount -t glusterfs gluster1:/integvol /mnt/gluster/ gt; amp;gt; gt; amp;gt; gt; amp;gt; Regards gt; amp;gt; Prabu gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; On Wed, 29 Apr 2015 17:58:16 +0530 A Ghoshalamp;lt;a.ghos...@tcs.comamp;gt; wrote gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; Performance would largely depend upon setup. While I cannot think of any gt; amp;gt; setup that would cause write to be this slow, if would help if you share the gt; amp;gt; following details: gt; amp;gt; gt; amp;gt; A) Glusterfs version gt; amp;gt; B) volume configuration (gluster v amp;lt;volnameamp;gt; info) gt; amp;gt; C) host linux version gt; amp;gt; D) details about the kind of network you use to connect your servers making gt; amp;gt; up your storage pool. gt; amp;gt; gt; amp;gt; Thanks, gt; amp;gt; Anirban gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; From: gjprabu amp;lt; gjpr...@zohocorp.com amp;gt; gt; amp;gt; To: amp;lt; gluster-users@gluster.org amp;gt; gt; amp;gt; Date: 04/29/2015 05:52 PM gt; amp;gt; Subject: Re: [Gluster-users] client is terrible with large amount of small gt; amp;gt; files gt; amp;gt; Sent by: gluster-users-boun...@gluster.org gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; Hi Team, gt; amp;gt; gt; amp;gt; If anybody know the solution please share us. gt; amp;gt; gt; amp;gt; Regards gt; amp;gt; Prabu gt; amp;gt; gt; amp;gt; gt; amp;gt; gt; amp;gt; On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu amp;lt; gjpr...@zohocorp.com amp;gt; gt; amp;gt; wrote gt; amp;gt; Hi Team, gt; amp;gt; gt; amp;gt; We are using glusterfs newly and testing data transfer part in client using gt; amp;gt; fuse.glusterfs file system but it is terrible with large amount of small gt; amp;gt; files (Large amount of
Re: [Gluster-users] client is terrible with large amount of small files
Hi Ghoshal, Please find the details below. A) Glusterfs version glusterfs 3.6.2 B) volume configuration (gluster v lt;volnamegt; info) gluster volume info Volume Name: integvol Type: Replicate Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: integ-gluster2:/srv/sdb1/brick Brick2: integ-gluster1:/srv/sdb1/brick Brick3: integ-gluster3:/srv/sdb1/brick C) host linux version CentOS release 6.5 (Final) D) details about the kind of network you use to connect your servers making up your storage pool. We are connecting LAN to LAN there is no special network configuration done Frome client we use to mount like below mount -t glusterfs gluster1:/integvol /mnt/gluster/ Regards Prabu On Wed, 29 Apr 2015 17:58:16 +0530 A Ghoshallt;a.ghos...@tcs.comgt; wrote Performance would largely depend upon setup. While I cannot think of any setup that would cause write to be this slow, if would help if you share the following details: A) Glusterfs version B) volume configuration (gluster v lt;volnamegt; info) C) host linux version D) details about the kind of network you use to connect your servers making up your storage pool. Thanks, Anirban From:gjprabu lt;gjpr...@zohocorp.comgt; To:lt;gluster-users@gluster.orggt; Date:04/29/2015 05:52 PM Subject:Re: [Gluster-users] client is terrible with large amount of small files Sent by:gluster-users-boun...@gluster.org Hi Team, If anybody know the solution please share us. Regards Prabu On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu lt;gjpr...@zohocorp.comgt; wrote Hi Team, We are using glusterfs newly and testing data transfer part in client using fuse.glusterfs file system but it is terrible with large amount of small files (Large amount of small file 150MB of size it's writing around 18min). I can able copy small files and syncing between the server brick are working fine but it is terrible with large amount of small files. if anybody please share the solution for the above issue. Regards Prabu ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] client is terrible with large amount of small files
Hi Team, If anybody know the solution please share us. Regards Prabu On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu lt;gjpr...@zohocorp.comgt; wrote Hi Team, We are using glusterfs newly and testing data transfer part in client using fuse.glusterfs file system but it is terrible with large amount of small files (Large amount of small file 150MB of size it's writing around 18min). I can able copy small files and syncing between the server brick are working fine but it is terrible with large amount of small files. if anybody please share the solution for the above issue. Regards Prabu ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] client is terrible with large amount of small files
Performance would largely depend upon setup. While I cannot think of any setup that would cause write to be this slow, if would help if you share the following details: A) Glusterfs version B) volume configuration (gluster v volname info) C) host linux version D) details about the kind of network you use to connect your servers making up your storage pool. Thanks, Anirban From: gjprabu gjpr...@zohocorp.com To: gluster-users@gluster.org Date: 04/29/2015 05:52 PM Subject:Re: [Gluster-users] client is terrible with large amount of small files Sent by:gluster-users-boun...@gluster.org Hi Team, If anybody know the solution please share us. Regards Prabu On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu gjpr...@zohocorp.com wrote Hi Team, We are using glusterfs newly and testing data transfer part in client using fuse.glusterfs file system but it is terrible with large amount of small files (Large amount of small file 150MB of size it's writing around 18min). I can able copy small files and syncing between the server brick are working fine but it is terrible with large amount of small files. if anybody please share the solution for the above issue. Regards Prabu ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] client is terrible with large amount of small files
- Original Message - From: gjprabu gjpr...@zohocorp.com To: A Ghoshal a.ghos...@tcs.com Cc: gluster-users@gluster.org, gluster-users-boun...@gluster.org Sent: Wednesday, April 29, 2015 9:07:07 AM Subject: Re: [Gluster-users] client is terrible with large amount of small files Hi Ghoshal, Please find the details below. A) Glusterfs version glusterfs 3.6.2 Upgrade to 3.6.3 and set client.event-threads and server.event-threads to at least 4. Here is a guide on tuning MT epoll: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html B) volume configuration (gluster v volname info) gluster volume info Volume Name: integvol Type: Replicate Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: integ-gluster2:/srv/sdb1/brick Brick2: integ-gluster1:/srv/sdb1/brick Brick3: integ-gluster3:/srv/sdb1/brick C) host linux version CentOS release 6.5 (Final) Are your bricks on LVM? Are you using thinp? If so update to the latest kernel as thinp perf was really bad in 6.5 and early 6.6 kernels. D) details about the kind of network you use to connect your servers making up your storage pool. We are connecting LAN to LAN there is no special network configuration done Frome client we use to mount like below mount -t glusterfs gluster1:/integvol /mnt/gluster/ Regards Prabu On Wed, 29 Apr 2015 17:58:16 +0530 A Ghoshala.ghos...@tcs.com wrote Performance would largely depend upon setup. While I cannot think of any setup that would cause write to be this slow, if would help if you share the following details: A) Glusterfs version B) volume configuration (gluster v volname info) C) host linux version D) details about the kind of network you use to connect your servers making up your storage pool. Thanks, Anirban From: gjprabu gjpr...@zohocorp.com To: gluster-users@gluster.org Date: 04/29/2015 05:52 PM Subject: Re: [Gluster-users] client is terrible with large amount of small files Sent by: gluster-users-boun...@gluster.org Hi Team, If anybody know the solution please share us. Regards Prabu On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu gjpr...@zohocorp.com wrote Hi Team, We are using glusterfs newly and testing data transfer part in client using fuse.glusterfs file system but it is terrible with large amount of small files (Large amount of small file 150MB of size it's writing around 18min). I can able copy small files and syncing between the server brick are working fine but it is terrible with large amount of small files. if anybody please share the solution for the above issue. Regards Prabu ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] client is terrible with large amount of small files
Le 28/04/2015 16:02, gjprabu a écrit : Hi Team, We are using glusterfs newly and testing data transfer part in client using *fuse.glusterfs* file system but it is terrible with large amount of small files (Large amount of small file 150MB of size it's writing around 18min). I can able copy small files and syncing between the server brick are working fine but it is terrible with large amount of small files. if anybody please share the solution for the above issue. Regards Prabu * * ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users Hello, What are you calling small files ? B, KB, MB sized ? -- Best regards, Pierre Barre, System administrator and developer at openmailbox.org, France. ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
