Re: [Gluster-users] memory leak in glusterd 3.7.x

[Zdenek Styblik]

On Sat, Aug 20, 2016 at 11:48 AM, Pranith Kumar Karampuri
 wrote:
> hi Zdenek,
> I recently found this issue and there has been a discussion on
> gluster-devel about how to fix this. It is a bit involved, so taking more
> time than I would like to fix, this bug has been there for almost 3-4 years
> I guess.
>
> You can find the discussion here:
> http://www.gluster.org/pipermail/gluster-devel/2016-July/050085.html
>

Hello Pranith,

thank you for reply and heads up. I've read only through mails with
subject "memory leak" and didn't know you, or somebody else, already
found this issue.

Best regards,
Z.

> On Fri, Aug 19, 2016 at 7:10 PM, Zdenek Styblik 
> wrote:
>>
>> Hello,
>>
>> we've found a memory leak in glusterd v3.7.x(currently at v3.7.14, but
>> we are users of v3.7.x from the beginning).
>> It seems, we've empirically verified, that continuous execution of %
>> gluster volume set   ; leads to memory leaks in
>> glusterd and OOM, although not necessarily OOM of glusterd itself.
>> Settings which were being set over and over again are
>> `nfs.addr-namelookup false` and `nfs.disable true`. There might have
>> been other settings, but I was able to find these in recent logs.
>> Unfortunately, we don't have capacity to debug this issue
>> further(statedumps are quite overwhelming :] ).
>> Repeated execution has been caused due to bug in Puppet module we're
>> using(and we were able to address this issue). Therefore, it's safe to
>> say that the number of affected users or like hood of somebody else
>> having these problem is probably low. It's still a memory leak and,
>> well, rather serious one if you happen to stumble upon it. Also, it
>> must be noted that this gets amplified if you have more than just
>> volume.
>>
>> If there is anything I can help with, let me know.
>>
>> Please, keep me on CC as I'm not subscribed to the mailing list.
>>
>> Best regards,
>> Zdenek Styblik
>> ___
>> Gluster-users mailing list
>> Gluster-users@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> --
> Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] memory leak in glusterd 3.7.x

[Pranith Kumar Karampuri]

On Mon, Aug 22, 2016 at 9:03 PM, Zdenek Styblik 
wrote:

> On Sat, Aug 20, 2016 at 11:48 AM, Pranith Kumar Karampuri
>  wrote:
> > hi Zdenek,
> > I recently found this issue and there has been a discussion on
> > gluster-devel about how to fix this. It is a bit involved, so taking more
> > time than I would like to fix, this bug has been there for almost 3-4
> years
> > I guess.
> >
> > You can find the discussion here:
> > http://www.gluster.org/pipermail/gluster-devel/2016-July/050085.html
> >
>
> Hello Pranith,
>
> thank you for reply and heads up. I've read only through mails with
> subject "memory leak" and didn't know you, or somebody else, already
> found this issue.
>

Hey,
   Please err on posting these kinds of mails in future as well :-). In
the worst case it will be something we already know. But in the best case
it wll be something new :-).


>
> Best regards,
> Z.
>
> > On Fri, Aug 19, 2016 at 7:10 PM, Zdenek Styblik <
> zdenek.styb...@showmax.com>
> > wrote:
> >>
> >> Hello,
> >>
> >> we've found a memory leak in glusterd v3.7.x(currently at v3.7.14, but
> >> we are users of v3.7.x from the beginning).
> >> It seems, we've empirically verified, that continuous execution of %
> >> gluster volume set   ; leads to memory leaks in
> >> glusterd and OOM, although not necessarily OOM of glusterd itself.
> >> Settings which were being set over and over again are
> >> `nfs.addr-namelookup false` and `nfs.disable true`. There might have
> >> been other settings, but I was able to find these in recent logs.
> >> Unfortunately, we don't have capacity to debug this issue
> >> further(statedumps are quite overwhelming :] ).
> >> Repeated execution has been caused due to bug in Puppet module we're
> >> using(and we were able to address this issue). Therefore, it's safe to
> >> say that the number of affected users or like hood of somebody else
> >> having these problem is probably low. It's still a memory leak and,
> >> well, rather serious one if you happen to stumble upon it. Also, it
> >> must be noted that this gets amplified if you have more than just
> >> volume.
> >>
> >> If there is anything I can help with, let me know.
> >>
> >> Please, keep me on CC as I'm not subscribed to the mailing list.
> >>
> >> Best regards,
> >> Zdenek Styblik
> >> ___
> >> Gluster-users mailing list
> >> Gluster-users@gluster.org
> >> http://www.gluster.org/mailman/listinfo/gluster-users
> >
> >
> >
> >
> > --
> > Pranith
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] memory leak in glusterd 3.7.x

[Pranith Kumar Karampuri]

hi Zdenek,
I recently found this issue and there has been a discussion on
gluster-devel about how to fix this. It is a bit involved, so taking more
time than I would like to fix, this bug has been there for almost 3-4 years
I guess.

You can find the discussion here:
http://www.gluster.org/pipermail/gluster-devel/2016-July/050085.html

On Fri, Aug 19, 2016 at 7:10 PM, Zdenek Styblik 
wrote:

> Hello,
>
> we've found a memory leak in glusterd v3.7.x(currently at v3.7.14, but
> we are users of v3.7.x from the beginning).
> It seems, we've empirically verified, that continuous execution of %
> gluster volume set   ; leads to memory leaks in
> glusterd and OOM, although not necessarily OOM of glusterd itself.
> Settings which were being set over and over again are
> `nfs.addr-namelookup false` and `nfs.disable true`. There might have
> been other settings, but I was able to find these in recent logs.
> Unfortunately, we don't have capacity to debug this issue
> further(statedumps are quite overwhelming :] ).
> Repeated execution has been caused due to bug in Puppet module we're
> using(and we were able to address this issue). Therefore, it's safe to
> say that the number of affected users or like hood of somebody else
> having these problem is probably low. It's still a memory leak and,
> well, rather serious one if you happen to stumble upon it. Also, it
> must be noted that this gets amplified if you have more than just
> volume.
>
> If there is anything I can help with, let me know.
>
> Please, keep me on CC as I'm not subscribed to the mailing list.
>
> Best regards,
> Zdenek Styblik
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] memory leak in glusterd 3.7.x

[Zdenek Styblik]

Hello,

we've found a memory leak in glusterd v3.7.x(currently at v3.7.14, but
we are users of v3.7.x from the beginning).
It seems, we've empirically verified, that continuous execution of %
gluster volume set   ; leads to memory leaks in
glusterd and OOM, although not necessarily OOM of glusterd itself.
Settings which were being set over and over again are
`nfs.addr-namelookup false` and `nfs.disable true`. There might have
been other settings, but I was able to find these in recent logs.
Unfortunately, we don't have capacity to debug this issue
further(statedumps are quite overwhelming :] ).
Repeated execution has been caused due to bug in Puppet module we're
using(and we were able to address this issue). Therefore, it's safe to
say that the number of affected users or like hood of somebody else
having these problem is probably low. It's still a memory leak and,
well, rather serious one if you happen to stumble upon it. Also, it
must be noted that this gets amplified if you have more than just
volume.

If there is anything I can help with, let me know.

Please, keep me on CC as I'm not subscribed to the mailing list.

Best regards,
Zdenek Styblik
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users