Re: [Gluster-users] Slow write times to gluster disk

Hi, We redid the dd tests (this time using conv=sync oflag=sync to avoid caching questions). The profile results are in http://mseas.mit.edu/download/phaley/GlusterUsers/profile_gluster_fuse_test On 05/05/2017 12:47 PM, Ravishankar N wrote: On 05/05/2017 08:42 PM, Pat Haley wrote: Hi

Re: [Gluster-users] disperse volume brick counts limits in RHES

What network do you have? On 5 May 2017 at 09:51, Serkan Çoban wrote: > In our use case every node has 26 bricks. I am using 60 nodes, one 9PB > volume with 16+4 EC configuration, each brick in a sub-volume is on > different host. > We put 15-20k 2GB files every day into

Re: [Gluster-users] ?==?utf-8?q? disperse volume brick counts limits in RHES

Hi, not sure about the exact reason of the slow self-heal, but we can try to see what's taking time. I would consider 3 factors: 1. Erasure code algorithm 2. Small I/O blocks 3. Latencies Some testing on 1) shows that a single core of an Intel Xeon E5-2630L 2 GHz, can encode and decode at

Re: [Gluster-users] Gluster and NFS-Ganesha - cluster is down after reboot

On 05/05/2017 08:04 PM, Adam Ru wrote: Hi Soumya, Thank you for the answer. Enabling Pacemaker? Yes, you’re completely right, I didn’t do it. Thank you. I spent some time by testing and I have some results. This is what I did: - Clean installation of CentOS 7.3 with all updates, 3x node,

Re: [Gluster-users] Arbiter and Hot Tier

Thank you very much for the assistance. -Walter Deignan -Uline IT, Systems Architect From: Ravishankar N To: Walter Deignan Cc: gluster-users@gluster.org Date: 05/05/2017 11:43 AM Subject:Re: [Gluster-users] Arbiter and Hot Tier

Re: [Gluster-users] Slow write times to gluster disk

On 05/05/2017 08:42 PM, Pat Haley wrote: Hi Pranith, I presume you are asking for some version of the profile data that just shows the dd test (or a repeat of the dd test). If yes, how do I extract just that data? Yes, that is what he is asking for. Just clear the existing profile info

Re: [Gluster-users] Arbiter and Hot Tier

Okay, just tried it in 3.9 Even though attaching a non arbiter volume (say a replica-2 ) as hot-tier is succeeding at the CLI level, the brick volfiles seem to be generated incorrectly (I see that the arbiter translator is getting loaded in one of the replica-2 bricks too, which is

Re: [Gluster-users] Arbiter and Hot Tier

Did that change between 3.9 and 3.10? When I originally saw some references on the Redhat storage packaged solution about a possible incompatibility I assumed it just meant that the hot tier itself couldn't be an arbiter volume. I was tripped up by the change in apparent support for the cold

Re: [Gluster-users] Arbiter and Hot Tier

Hi Walter, Yes, arbiter volumes are currently not supported with tiering. -Ravi On 05/05/2017 08:54 PM, Walter Deignan wrote: I've been googling this to no avail so apologies if this is explained somewhere I missed. Is there a known incompatibility between using arbiters and hot tiering?

[Gluster-users] Arbiter and Hot Tier

I've been googling this to no avail so apologies if this is explained somewhere I missed. Is there a known incompatibility between using arbiters and hot tiering? Experience on 3.9 Original volume - replica 3 arbiter 1 Attach replica 2 arbiter 1 hot tier - failure Attach replica 3 hot tier -

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith, I presume you are asking for some version of the profile data that just shows the dd test (or a repeat of the dd test). If yes, how do I extract just that data? Thanks Pat On 05/05/2017 10:58 AM, Pranith Kumar Karampuri wrote: hi Pat, Let us concentrate on the

Re: [Gluster-users] Slow write times to gluster disk

hi Pat, Let us concentrate on the performance numbers part for now. We will look at the permissions one after this? As per the profile info, only 2.6% of the work-load is writes. There are too many Lookups. Would it be possible to get the data for just the dd test you were doing earlier?

Re: [Gluster-users] Slow write times to gluster disk

Hi Pranith & Ravi, A couple of quick questions We have profile turned on. Are there specific queries we should make that would help debug our configuration? (The default profile info was previously sent in http://lists.gluster.org/pipermail/gluster-users/2017-May/030840.html but I'm not

Re: [Gluster-users] Gluster and NFS-Ganesha - cluster is down after reboot

Hi Soumya, Thank you for the answer. Enabling Pacemaker? Yes, you’re completely right, I didn’t do it. Thank you. I spent some time by testing and I have some results. This is what I did: - Clean installation of CentOS 7.3 with all updates, 3x node, resolvable IPs and VIPs - Stopped

Re: [Gluster-users] disperse volume brick counts limits in RHES

Wondering if Xavi knows something. On Fri, May 5, 2017 at 7:24 PM, Pranith Kumar Karampuri wrote: > > > On Fri, May 5, 2017 at 7:21 PM, Serkan Çoban > wrote: > >> In our use case every node has 26 bricks. I am using 60 nodes, one 9PB >> volume with

Re: [Gluster-users] disperse volume brick counts limits in RHES

On Fri, May 5, 2017 at 7:21 PM, Serkan Çoban wrote: > In our use case every node has 26 bricks. I am using 60 nodes, one 9PB > volume with 16+4 EC configuration, each brick in a sub-volume is on > different host. > We put 15-20k 2GB files every day into 10-15 folders. So

Re: [Gluster-users] disperse volume brick counts limits in RHES

In our use case every node has 26 bricks. I am using 60 nodes, one 9PB volume with 16+4 EC configuration, each brick in a sub-volume is on different host. We put 15-20k 2GB files every day into 10-15 folders. So it is 1500K files/folder. Our gluster version is 3.7.11. Heal speed in this

Re: [Gluster-users] Remove-brick failed

Thanks My plans with the servers is to reinstall them all with rhel7 and the newest gluster-version. So for now my focus is just to free glustertst03 and glustertst04 for reinstallation. So glusterfs-rdma is not installed and I guess rdma is not working…. :-). Anyhow. Do I need to install rdma

Re: [Gluster-users] [Gluster-Maintainers] [Gluster-devel] Release 3.11: Has been Branched (and pending feature notes)

I merged this and posted the backport merging the two patches into one: https://review.gluster.org/#/c/17192 Regressions started just now. On Fri, May 5, 2017 at 3:31 PM, Kaushal M wrote: > On Thu, May 4, 2017 at 6:40 PM, Kaushal M wrote: > > On Thu,

Re: [Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

On Fri, May 5, 2017 at 5:40 PM, Pranith Kumar Karampuri wrote: > > > On Fri, May 5, 2017 at 5:36 PM, Abhijit Paul > wrote: > >> So should i start using gluster-block with elasticsearch in kubernetes >> environment? >> >> My expectation from

Re: [Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

On Fri, May 5, 2017 at 5:36 PM, Abhijit Paul wrote: > So should i start using gluster-block with elasticsearch in kubernetes > environment? > > My expectation from gluster-block is, it should not CorruptIndex > of elasticsearch...and issue facing in previous mails. > >

Re: [Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

So should i start using gluster-block with elasticsearch in kubernetes environment? My expectation from gluster-block is, it should not CorruptIndex of elasticsearch...and issue facing in previous mails. Please let me know whether should i processed with above mentioned combination. On Fri, May

Re: [Gluster-users] disperse volume brick counts limits in RHES

On Fri, May 5, 2017 at 5:19 PM, Pranith Kumar Karampuri wrote: > > > On Fri, May 5, 2017 at 2:38 PM, Serkan Çoban > wrote: > >> It is the over all time, 8TB data disk healed 2x faster in 8+2 >> configuration. >> > > Wow, that is counter intuitive for

[Gluster-users] www.JamesCoyle.net hijacked?

James Colye has written a lot of blog entries on gluster and they are often referenced here, in fact I check his site out recently. But now its got an invalid ssl cert for cloud.skizzip.com, the content is gone and it just got a logon page for "VESTA" -- Lindsay Mathieson

Re: [Gluster-users] disperse volume brick counts limits in RHES

On Fri, May 5, 2017 at 2:38 PM, Serkan Çoban wrote: > It is the over all time, 8TB data disk healed 2x faster in 8+2 > configuration. > Wow, that is counter intuitive for me. I will need to explore about this to find out why that could be. Thanks a lot for this feedback!

Re: [Gluster-users] Remove-brick failed

Hi, You need to check the rebalance logs (glu_linux_dr2_oracle-rebalance.log) on glustoretst03.net.dr.dk and glustoretst04.net.dr.dk to see what went wrong. Regards, Nithya On 4 May 2017 at 11:46, Jesper Led Lauridsen TS Infra server wrote: >

Re: [Gluster-users] [Gluster-devel] [Gluster-Maintainers] Release 3.11: Has been Branched (and pending feature notes)

We have one more blocker bug (opened today): https://bugzilla.redhat.com/show_bug.cgi?id=1448307 On 5 May 2017 at 15:31, Kaushal M wrote: > On Thu, May 4, 2017 at 6:40 PM, Kaushal M wrote: > > On Thu, May 4, 2017 at 4:38 PM, Niels de Vos

Re: [Gluster-users] [Gluster-Maintainers] [Gluster-devel] Release 3.11: Has been Branched (and pending feature notes)

On Thu, May 4, 2017 at 6:40 PM, Kaushal M wrote: > On Thu, May 4, 2017 at 4:38 PM, Niels de Vos wrote: >> On Thu, May 04, 2017 at 03:39:58PM +0530, Pranith Kumar Karampuri wrote: >>> On Wed, May 3, 2017 at 2:36 PM, Kaushal M wrote:

Re: [Gluster-users] disperse volume brick counts limits in RHES

It is the over all time, 8TB data disk healed 2x faster in 8+2 configuration. On Fri, May 5, 2017 at 10:00 AM, Pranith Kumar Karampuri wrote: > > > On Fri, May 5, 2017 at 11:42 AM, Serkan Çoban wrote: >> >> Healing gets slower as you increase m in m+n

[Gluster-users] Is my (low) performance expected?

Hi on a test setup of 2 nodes with replication over a 1Gbit link I'm seeing extremely low performance while deleting thousands of small files (in the order of a few hundred times slower than hitting the disks directly ~300 unlinks/min to give you a number). The setup is very basic -- I've just

Re: [Gluster-users] disperse volume brick counts limits in RHES

On Fri, May 5, 2017 at 11:42 AM, Serkan Çoban wrote: > Healing gets slower as you increase m in m+n configuration. > We are using 16+4 configuration without any problems other then heal speed. > I tested heal speed with 8+2 and 16+4 on 3.9.0 and see that heals on > 8+2 is

Re: [Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

thanks Krutika for the alternative. @*Prasanna @**Pranith* I was going thorough the mentioned blog post and saw that used tested environment was Fedora , but i am using RHEL based Oracle linux so does gluster-block compatible with RHEL as well? On Fri, May 5, 2017 at 12:03 PM, Krutika Dhananjay

Re: [Gluster-users] Elasticsearch facing CorruptIndexException exception with GlusterFs 3.10.1

Yeah, there are a couple of cache consistency issues with performance translators that are causing these exceptions. Some of them were fixed by 3.10.1. Some still remain. Alternatively you can give gluster-block + elasticsearch a try, which doesn't require solving all these caching issues. Here's

Re: [Gluster-users] total folder size increased

Sorry for the trouble, I clearly not in my right mind when I looked at df -h output. To my surprise there is no size increment. 2017-05-05 9:59 GMT+08:00 Umarzuki Mochlis : > Hi, > > I'm copying a web folder with total size of 109GB to a mounted volume > (locally). Now it is

Re: [Gluster-users] [Gluster-devel] Enabling shard on EC

Hi, Work is in progress for this (and going a bit slow at the moment because of other priorities). At the moment we support sharding only for VM image store use-case - most common large file + single writer use case we know of. Just curious, what is the use case where you want to use shard+EC?

Re: [Gluster-users] disperse volume brick counts limits in RHES

Healing gets slower as you increase m in m+n configuration. We are using 16+4 configuration without any problems other then heal speed. I tested heal speed with 8+2 and 16+4 on 3.9.0 and see that heals on 8+2 is faster by 2x. On Fri, May 5, 2017 at 9:04 AM, Ashish Pandey

Re: [Gluster-users] Advice needed for geo replication

Hi Felipe, All the observations you have made are correct. AFR is a synchronous replication where the client replicates the data which is limited by the speed by the slowest node (in your case HDD node). AFR is the replicating each brick and is part of single volume. At the end, you will have

Re: [Gluster-users] disperse volume brick counts limits in RHES

8+2 and 8+3 configurations are not the limitation but just suggestions. You can create 16+3 volume without any issue. Ashish - Original Message - From: "Alastair Neil" To: "gluster-users" Sent: Friday, May 5, 2017 2:23:32 AM