[Gluster-users] stripped volume in 3.4.0qa5 with horrible read performance
Dear folks, I've been tried to use replicated stripped volumes with 3.3. unsuccessfully due to https://bugzilla.redhat.com/show_bug.cgi?id=861423 and I then proceed to try 3.4.0qa5. I then find out that the bug was solved and I could use replicated stripped volume with the new version. Amazingly, write performance was quite astonishing. The problem I'm facing now is in the read process: It's horribly slow. When I open a file to edit using the gluster native client, it takes a few seconds and sometimes I got an error refering to file has been modified while I was editing it. There's a ruby application reading the files and I got continuously timeout errors. I'm using 4 bricks with Centos 6.3 with the following structure: Type: Striped-Replicate Volume ID: 23dbb8dd-5cb3-4c71-9702-7c16ee9a3b3b Status: Started Number of Bricks: 1 x 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.0.51.31:/gfs Brick2: 10.0.51.32:/gfs Brick3: 10.0.51.33:/gfs Brick4: 10.0.51.34:/gfs Options Reconfigured: performance.quick-read: on performance.io-thread-count: 32 performance.cache-max-file-size: 128MB performance.cache-size: 256MB performance.io-cache: on cluster.stripe-block-size: 2MB nfs.disable: on I started profiling and found out one node with absurd latency figures. I stopped the node and the problem moved to another brick: %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop - --- --- --- 99.94 551292.41 us 10.00 us 1996709.00 us361FINODELK Could anyone provide some information how to debug this problem? Currently the volume is not usable due to the horrible delay. Thank you very much in advance, Samuel. ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] stripped volume in 3.4.0qa5 with horrible read performance
Please file a bug. There might be time to fix read performance before the 1st beta release. -JM - Original Message - Dear folks, I've been tried to use replicated stripped volumes with 3.3. unsuccessfully due to https://bugzilla.redhat.com/show_bug.cgi?id=861423 and I then proceed to try 3.4.0qa5. I then find out that the bug was solved and I could use replicated stripped volume with the new version. Amazingly, write performance was quite astonishing. The problem I'm facing now is in the read process: It's horribly slow. When I open a file to edit using the gluster native client, it takes a few seconds and sometimes I got an error refering to file has been modified while I was editing it. There's a ruby application reading the files and I got continuously timeout errors. I'm using 4 bricks with Centos 6.3 with the following structure: Type: Striped-Replicate Volume ID: 23dbb8dd-5cb3-4c71-9702-7c16ee9a3b3b Status: Started Number of Bricks: 1 x 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.0.51.31:/gfs Brick2: 10.0.51.32:/gfs Brick3: 10.0.51.33:/gfs Brick4: 10.0.51.34:/gfs Options Reconfigured: performance.quick-read: on performance.io-thread-count: 32 performance.cache-max-file-size: 128MB performance.cache-size: 256MB performance.io-cache: on cluster.stripe-block-size: 2MB nfs.disable: on I started profiling and found out one node with absurd latency figures. I stopped the node and the problem moved to another brick: %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop - --- --- --- 99.94 551292.41 us 10.00 us 1996709.00 us 361 FINODELK Could anyone provide some information how to debug this problem? Currently the volume is not usable due to the horrible delay. Thank you very much in advance, Samuel. ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] stripped volume in 3.4.0qa5 with horrible read performance
Done. https://bugzilla.redhat.com/show_bug.cgi?id=888174 While testing the system, we found 3.3.0 enables stripped-replicated volumes and seems to offer a right read behaviour in some tests. Thanks in advance and, please, contact me in case I can offer further help. Best regards, Samuel. On 17 December 2012 16:20, John Mark Walker johnm...@redhat.com wrote: Please file a bug. There might be time to fix read performance before the 1st beta release. -JM -- Dear folks, I've been tried to use replicated stripped volumes with 3.3. unsuccessfully due to https://bugzilla.redhat.com/show_bug.cgi?id=861423and I then proceed to try 3.4.0qa5. I then find out that the bug was solved and I could use replicated stripped volume with the new version. Amazingly, write performance was quite astonishing. The problem I'm facing now is in the read process: It's horribly slow. When I open a file to edit using the gluster native client, it takes a few seconds and sometimes I got an error refering to file has been modified while I was editing it. There's a ruby application reading the files and I got continuously timeout errors. I'm using 4 bricks with Centos 6.3 with the following structure: Type: Striped-Replicate Volume ID: 23dbb8dd-5cb3-4c71-9702-7c16ee9a3b3b Status: Started Number of Bricks: 1 x 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: 10.0.51.31:/gfs Brick2: 10.0.51.32:/gfs Brick3: 10.0.51.33:/gfs Brick4: 10.0.51.34:/gfs Options Reconfigured: performance.quick-read: on performance.io-thread-count: 32 performance.cache-max-file-size: 128MB performance.cache-size: 256MB performance.io-cache: on cluster.stripe-block-size: 2MB nfs.disable: on I started profiling and found out one node with absurd latency figures. I stopped the node and the problem moved to another brick: %-latency Avg-latency Min-Latency Max-Latency No. of calls Fop - --- --- --- 99.94 551292.41 us 10.00 us 1996709.00 us361FINODELK Could anyone provide some information how to debug this problem? Currently the volume is not usable due to the horrible delay. Thank you very much in advance, Samuel. ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-users
