Re: [Gluster-users] High load on glusterfsd process
Hi Kotresh, As we know this problem occurs when BitRot start versioning of file of big size. Is there any possibility to disable this feature totally means remove the BitRot feature so that it will not do this even when it is disabled. Regards, Abhishek On Tue, Apr 25, 2017 at 12:47 PM, ABHISHEK PALIWAL wrote: > Thanks Kotresh. > > Let me discuss in my team and will let you know. > > Regards, > Abhishek > > On Tue, Apr 25, 2017 at 12:41 PM, Kotresh Hiremath Ravishankar < > khire...@redhat.com> wrote: > >> Hi Abhishek, >> >> As this is an enhancement it won't be back ported to 3.7/3.8/3.10 >> It would be only available from upcoming 3.11 release. >> >> But I did try applying it to 3.7.6. It has lot of conflicts. >> If it's important for you, you can upgrade to latest version. >> available and back port it. If it's impossible to upgrade to >> latest version, atleast 3.7.20 would do. It has minimal >> conflicts. I can help you out with that. >> >> Thanks and Regards, >> Kotresh H R >> >> - Original Message - >> > From: "ABHISHEK PALIWAL" >> > To: "Kotresh Hiremath Ravishankar" >> > Cc: "Pranith Kumar Karampuri" , "Gluster Devel" < >> gluster-de...@gluster.org>, "gluster-users" >> > >> > Sent: Tuesday, April 25, 2017 10:58:41 AM >> > Subject: Re: [Gluster-users] High load on glusterfsd process >> > >> > Hi Kotresh, >> > >> > Could you please update whether it is possible to get the patch or >> bakport >> > this patch on Gluster 3.7.6 version. >> > >> > Regards, >> > Abhishek >> > >> > On Mon, Apr 24, 2017 at 6:14 PM, ABHISHEK PALIWAL < >> abhishpali...@gmail.com> >> > wrote: >> > >> > > What is the way to take this patch on Gluster 3.7.6 or only way to >> upgrade >> > > the version? >> > > >> > > On Mon, Apr 24, 2017 at 3:22 PM, ABHISHEK PALIWAL < >> abhishpali...@gmail.com >> > > > wrote: >> > > >> > >> Hi Kotresh, >> > >> >> > >> I have seen the patch available on the link which you shared. It >> seems we >> > >> don't have some files in gluser 3.7.6 which you modified in the >> patch. >> > >> >> > >> Is there any possibility to provide the patch for Gluster 3.7.6? >> > >> >> > >> Regards, >> > >> Abhishek >> > >> >> > >> On Mon, Apr 24, 2017 at 3:07 PM, Kotresh Hiremath Ravishankar < >> > >> khire...@redhat.com> wrote: >> > >> >> > >>> Hi Abhishek, >> > >>> >> > >>> Bitrot requires versioning of files to be down on writes. >> > >>> This was being done irrespective of whether bitrot is >> > >>> enabled or not. This takes considerable CPU. With the >> > >>> fix https://review.gluster.org/#/c/14442/, it is made >> > >>> optional and is enabled only with bitrot. If bitrot >> > >>> is not enabled, then you won't see any setxattr/getxattrs >> > >>> related to bitrot. >> > >>> >> > >>> The fix would be available in 3.11. >> > >>> >> > >>> >> > >>> Thanks and Regards, >> > >>> Kotresh H R >> > >>> >> > >>> - Original Message - >> > >>> > From: "ABHISHEK PALIWAL" >> > >>> > To: "Pranith Kumar Karampuri" >> > >>> > Cc: "Gluster Devel" , "gluster-users" >> < >> > >>> gluster-users@gluster.org>, "Kotresh Hiremath >> > >>> > Ravishankar" >> > >>> > Sent: Monday, April 24, 2017 11:30:57 AM >> > >>> > Subject: Re: [Gluster-users] High load on glusterfsd process >> > >>> > >> > >>> > Hi Kotresh, >> > >>> > >> > >>> > Could you please update me on this? >> > >>> > >> > >>> > Regards, >> > >>> > Abhishek >> > >>> > >> > >>> > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < >> > >>> > pkara...@redhat.com> wrote: >> > >>> > >> > >>> > > +Kotre
Re: [Gluster-users] High load on glusterfsd process
Thanks Kotresh. Let me discuss in my team and will let you know. Regards, Abhishek On Tue, Apr 25, 2017 at 12:41 PM, Kotresh Hiremath Ravishankar < khire...@redhat.com> wrote: > Hi Abhishek, > > As this is an enhancement it won't be back ported to 3.7/3.8/3.10 > It would be only available from upcoming 3.11 release. > > But I did try applying it to 3.7.6. It has lot of conflicts. > If it's important for you, you can upgrade to latest version. > available and back port it. If it's impossible to upgrade to > latest version, atleast 3.7.20 would do. It has minimal > conflicts. I can help you out with that. > > Thanks and Regards, > Kotresh H R > > - Original Message - > > From: "ABHISHEK PALIWAL" > > To: "Kotresh Hiremath Ravishankar" > > Cc: "Pranith Kumar Karampuri" , "Gluster Devel" < > gluster-de...@gluster.org>, "gluster-users" > > > > Sent: Tuesday, April 25, 2017 10:58:41 AM > > Subject: Re: [Gluster-users] High load on glusterfsd process > > > > Hi Kotresh, > > > > Could you please update whether it is possible to get the patch or > bakport > > this patch on Gluster 3.7.6 version. > > > > Regards, > > Abhishek > > > > On Mon, Apr 24, 2017 at 6:14 PM, ABHISHEK PALIWAL < > abhishpali...@gmail.com> > > wrote: > > > > > What is the way to take this patch on Gluster 3.7.6 or only way to > upgrade > > > the version? > > > > > > On Mon, Apr 24, 2017 at 3:22 PM, ABHISHEK PALIWAL < > abhishpali...@gmail.com > > > > wrote: > > > > > >> Hi Kotresh, > > >> > > >> I have seen the patch available on the link which you shared. It > seems we > > >> don't have some files in gluser 3.7.6 which you modified in the patch. > > >> > > >> Is there any possibility to provide the patch for Gluster 3.7.6? > > >> > > >> Regards, > > >> Abhishek > > >> > > >> On Mon, Apr 24, 2017 at 3:07 PM, Kotresh Hiremath Ravishankar < > > >> khire...@redhat.com> wrote: > > >> > > >>> Hi Abhishek, > > >>> > > >>> Bitrot requires versioning of files to be down on writes. > > >>> This was being done irrespective of whether bitrot is > > >>> enabled or not. This takes considerable CPU. With the > > >>> fix https://review.gluster.org/#/c/14442/, it is made > > >>> optional and is enabled only with bitrot. If bitrot > > >>> is not enabled, then you won't see any setxattr/getxattrs > > >>> related to bitrot. > > >>> > > >>> The fix would be available in 3.11. > > >>> > > >>> > > >>> Thanks and Regards, > > >>> Kotresh H R > > >>> > > >>> - Original Message - > > >>> > From: "ABHISHEK PALIWAL" > > >>> > To: "Pranith Kumar Karampuri" > > >>> > Cc: "Gluster Devel" , "gluster-users" < > > >>> gluster-users@gluster.org>, "Kotresh Hiremath > > >>> > Ravishankar" > > >>> > Sent: Monday, April 24, 2017 11:30:57 AM > > >>> > Subject: Re: [Gluster-users] High load on glusterfsd process > > >>> > > > >>> > Hi Kotresh, > > >>> > > > >>> > Could you please update me on this? > > >>> > > > >>> > Regards, > > >>> > Abhishek > > >>> > > > >>> > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < > > >>> > pkara...@redhat.com> wrote: > > >>> > > > >>> > > +Kotresh who seems to have worked on the bug you mentioned. > > >>> > > > > >>> > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < > > >>> > > abhishpali...@gmail.com> wrote: > > >>> > > > > >>> > >> > > >>> > >> If the patch provided in that case will resolve my bug as well > then > > >>> > >> please provide the patch so that I will backport it on 3.7.6 > > >>> > >> > > >>> > >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < > > >>> > >> abhishpali...@gmail.com> wrote: > > >>> > >
Re: [Gluster-users] High load on glusterfsd process
Hi Abhishek, As this is an enhancement it won't be back ported to 3.7/3.8/3.10 It would be only available from upcoming 3.11 release. But I did try applying it to 3.7.6. It has lot of conflicts. If it's important for you, you can upgrade to latest version. available and back port it. If it's impossible to upgrade to latest version, atleast 3.7.20 would do. It has minimal conflicts. I can help you out with that. Thanks and Regards, Kotresh H R - Original Message - > From: "ABHISHEK PALIWAL" > To: "Kotresh Hiremath Ravishankar" > Cc: "Pranith Kumar Karampuri" , "Gluster Devel" > , "gluster-users" > > Sent: Tuesday, April 25, 2017 10:58:41 AM > Subject: Re: [Gluster-users] High load on glusterfsd process > > Hi Kotresh, > > Could you please update whether it is possible to get the patch or bakport > this patch on Gluster 3.7.6 version. > > Regards, > Abhishek > > On Mon, Apr 24, 2017 at 6:14 PM, ABHISHEK PALIWAL > wrote: > > > What is the way to take this patch on Gluster 3.7.6 or only way to upgrade > > the version? > > > > On Mon, Apr 24, 2017 at 3:22 PM, ABHISHEK PALIWAL > > wrote: > > > >> Hi Kotresh, > >> > >> I have seen the patch available on the link which you shared. It seems we > >> don't have some files in gluser 3.7.6 which you modified in the patch. > >> > >> Is there any possibility to provide the patch for Gluster 3.7.6? > >> > >> Regards, > >> Abhishek > >> > >> On Mon, Apr 24, 2017 at 3:07 PM, Kotresh Hiremath Ravishankar < > >> khire...@redhat.com> wrote: > >> > >>> Hi Abhishek, > >>> > >>> Bitrot requires versioning of files to be down on writes. > >>> This was being done irrespective of whether bitrot is > >>> enabled or not. This takes considerable CPU. With the > >>> fix https://review.gluster.org/#/c/14442/, it is made > >>> optional and is enabled only with bitrot. If bitrot > >>> is not enabled, then you won't see any setxattr/getxattrs > >>> related to bitrot. > >>> > >>> The fix would be available in 3.11. > >>> > >>> > >>> Thanks and Regards, > >>> Kotresh H R > >>> > >>> - Original Message - > >>> > From: "ABHISHEK PALIWAL" > >>> > To: "Pranith Kumar Karampuri" > >>> > Cc: "Gluster Devel" , "gluster-users" < > >>> gluster-users@gluster.org>, "Kotresh Hiremath > >>> > Ravishankar" > >>> > Sent: Monday, April 24, 2017 11:30:57 AM > >>> > Subject: Re: [Gluster-users] High load on glusterfsd process > >>> > > >>> > Hi Kotresh, > >>> > > >>> > Could you please update me on this? > >>> > > >>> > Regards, > >>> > Abhishek > >>> > > >>> > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < > >>> > pkara...@redhat.com> wrote: > >>> > > >>> > > +Kotresh who seems to have worked on the bug you mentioned. > >>> > > > >>> > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < > >>> > > abhishpali...@gmail.com> wrote: > >>> > > > >>> > >> > >>> > >> If the patch provided in that case will resolve my bug as well then > >>> > >> please provide the patch so that I will backport it on 3.7.6 > >>> > >> > >>> > >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < > >>> > >> abhishpali...@gmail.com> wrote: > >>> > >> > >>> > >>> Hi Team, > >>> > >>> > >>> > >>> I have noticed that there are so many glusterfsd threads are > >>> running in > >>> > >>> my system and we observed some of those thread consuming more cpu. > >>> I > >>> > >>> did “strace” on two such threads (before the problem disappeared by > >>> > >>> itself) > >>> > >>> and found that there is a continuous activity like below: > >>> > >>> > >>> > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 > >>> > >>> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-
Re: [Gluster-users] High load on glusterfsd process
Hi Kotresh, Could you please update whether it is possible to get the patch or bakport this patch on Gluster 3.7.6 version. Regards, Abhishek On Mon, Apr 24, 2017 at 6:14 PM, ABHISHEK PALIWAL wrote: > What is the way to take this patch on Gluster 3.7.6 or only way to upgrade > the version? > > On Mon, Apr 24, 2017 at 3:22 PM, ABHISHEK PALIWAL > wrote: > >> Hi Kotresh, >> >> I have seen the patch available on the link which you shared. It seems we >> don't have some files in gluser 3.7.6 which you modified in the patch. >> >> Is there any possibility to provide the patch for Gluster 3.7.6? >> >> Regards, >> Abhishek >> >> On Mon, Apr 24, 2017 at 3:07 PM, Kotresh Hiremath Ravishankar < >> khire...@redhat.com> wrote: >> >>> Hi Abhishek, >>> >>> Bitrot requires versioning of files to be down on writes. >>> This was being done irrespective of whether bitrot is >>> enabled or not. This takes considerable CPU. With the >>> fix https://review.gluster.org/#/c/14442/, it is made >>> optional and is enabled only with bitrot. If bitrot >>> is not enabled, then you won't see any setxattr/getxattrs >>> related to bitrot. >>> >>> The fix would be available in 3.11. >>> >>> >>> Thanks and Regards, >>> Kotresh H R >>> >>> - Original Message - >>> > From: "ABHISHEK PALIWAL" >>> > To: "Pranith Kumar Karampuri" >>> > Cc: "Gluster Devel" , "gluster-users" < >>> gluster-users@gluster.org>, "Kotresh Hiremath >>> > Ravishankar" >>> > Sent: Monday, April 24, 2017 11:30:57 AM >>> > Subject: Re: [Gluster-users] High load on glusterfsd process >>> > >>> > Hi Kotresh, >>> > >>> > Could you please update me on this? >>> > >>> > Regards, >>> > Abhishek >>> > >>> > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < >>> > pkara...@redhat.com> wrote: >>> > >>> > > +Kotresh who seems to have worked on the bug you mentioned. >>> > > >>> > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < >>> > > abhishpali...@gmail.com> wrote: >>> > > >>> > >> >>> > >> If the patch provided in that case will resolve my bug as well then >>> > >> please provide the patch so that I will backport it on 3.7.6 >>> > >> >>> > >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < >>> > >> abhishpali...@gmail.com> wrote: >>> > >> >>> > >>> Hi Team, >>> > >>> >>> > >>> I have noticed that there are so many glusterfsd threads are >>> running in >>> > >>> my system and we observed some of those thread consuming more cpu. >>> I >>> > >>> did “strace” on two such threads (before the problem disappeared by >>> > >>> itself) >>> > >>> and found that there is a continuous activity like below: >>> > >>> >>> > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >>> > >>> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170 >>> 126T113552+.log.gz", >>> > >>> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 >>> > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >>> > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_2 >>> 0170126T113552+.log.gz", >>> > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No >>> data >>> > >>> available) >>> > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >>> > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_2 >>> 0170126T113552+.log.gz", >>> > >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No >>> data >>> > >>> available) >>> > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >>> > >>> dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T >>> 180550+.log.gz", >>> > >>>
Re: [Gluster-users] High load on glusterfsd process
What is the way to take this patch on Gluster 3.7.6 or only way to upgrade the version? On Mon, Apr 24, 2017 at 3:22 PM, ABHISHEK PALIWAL wrote: > Hi Kotresh, > > I have seen the patch available on the link which you shared. It seems we > don't have some files in gluser 3.7.6 which you modified in the patch. > > Is there any possibility to provide the patch for Gluster 3.7.6? > > Regards, > Abhishek > > On Mon, Apr 24, 2017 at 3:07 PM, Kotresh Hiremath Ravishankar < > khire...@redhat.com> wrote: > >> Hi Abhishek, >> >> Bitrot requires versioning of files to be down on writes. >> This was being done irrespective of whether bitrot is >> enabled or not. This takes considerable CPU. With the >> fix https://review.gluster.org/#/c/14442/, it is made >> optional and is enabled only with bitrot. If bitrot >> is not enabled, then you won't see any setxattr/getxattrs >> related to bitrot. >> >> The fix would be available in 3.11. >> >> >> Thanks and Regards, >> Kotresh H R >> >> - Original Message - >> > From: "ABHISHEK PALIWAL" >> > To: "Pranith Kumar Karampuri" >> > Cc: "Gluster Devel" , "gluster-users" < >> gluster-users@gluster.org>, "Kotresh Hiremath >> > Ravishankar" >> > Sent: Monday, April 24, 2017 11:30:57 AM >> > Subject: Re: [Gluster-users] High load on glusterfsd process >> > >> > Hi Kotresh, >> > >> > Could you please update me on this? >> > >> > Regards, >> > Abhishek >> > >> > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < >> > pkara...@redhat.com> wrote: >> > >> > > +Kotresh who seems to have worked on the bug you mentioned. >> > > >> > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < >> > > abhishpali...@gmail.com> wrote: >> > > >> > >> >> > >> If the patch provided in that case will resolve my bug as well then >> > >> please provide the patch so that I will backport it on 3.7.6 >> > >> >> > >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < >> > >> abhishpali...@gmail.com> wrote: >> > >> >> > >>> Hi Team, >> > >>> >> > >>> I have noticed that there are so many glusterfsd threads are >> running in >> > >>> my system and we observed some of those thread consuming more cpu. I >> > >>> did “strace” on two such threads (before the problem disappeared by >> > >>> itself) >> > >>> and found that there is a continuous activity like below: >> > >>> >> > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >> > >>> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170 >> 126T113552+.log.gz", >> > >>> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 >> > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >> > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_ >> 20170126T113552+.log.gz", >> > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No >> data >> > >>> available) >> > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >> > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_ >> 20170126T113552+.log.gz", >> > >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No >> data >> > >>> available) >> > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >> > >>> dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T >> 180550+.log.gz", >> > >>> {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 >> > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >> > >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170 >> 123T180550+.log.gz", >> > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No >> data >> > >>> available) >> > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >> > >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170 >> 123T180550+.log.gz", >> > >>> &
Re: [Gluster-users] High load on glusterfsd process
Hi Kotresh, I have seen the patch available on the link which you shared. It seems we don't have some files in gluser 3.7.6 which you modified in the patch. Is there any possibility to provide the patch for Gluster 3.7.6? Regards, Abhishek On Mon, Apr 24, 2017 at 3:07 PM, Kotresh Hiremath Ravishankar < khire...@redhat.com> wrote: > Hi Abhishek, > > Bitrot requires versioning of files to be down on writes. > This was being done irrespective of whether bitrot is > enabled or not. This takes considerable CPU. With the > fix https://review.gluster.org/#/c/14442/, it is made > optional and is enabled only with bitrot. If bitrot > is not enabled, then you won't see any setxattr/getxattrs > related to bitrot. > > The fix would be available in 3.11. > > > Thanks and Regards, > Kotresh H R > > - Original Message - > > From: "ABHISHEK PALIWAL" > > To: "Pranith Kumar Karampuri" > > Cc: "Gluster Devel" , "gluster-users" < > gluster-users@gluster.org>, "Kotresh Hiremath > > Ravishankar" > > Sent: Monday, April 24, 2017 11:30:57 AM > > Subject: Re: [Gluster-users] High load on glusterfsd process > > > > Hi Kotresh, > > > > Could you please update me on this? > > > > Regards, > > Abhishek > > > > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < > > pkara...@redhat.com> wrote: > > > > > +Kotresh who seems to have worked on the bug you mentioned. > > > > > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < > > > abhishpali...@gmail.com> wrote: > > > > > >> > > >> If the patch provided in that case will resolve my bug as well then > > >> please provide the patch so that I will backport it on 3.7.6 > > >> > > >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < > > >> abhishpali...@gmail.com> wrote: > > >> > > >>> Hi Team, > > >>> > > >>> I have noticed that there are so many glusterfsd threads are running > in > > >>> my system and we observed some of those thread consuming more cpu. I > > >>> did “strace” on two such threads (before the problem disappeared by > > >>> itself) > > >>> and found that there is a continuous activity like below: > > >>> > > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 > > >>> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_ > 20170126T113552+.log.gz", > > >>> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 > > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482- > 425_20170126T113552+.log.gz", > > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No > data > > >>> available) > > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482- > 425_20170126T113552+.log.gz", > > >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No > data > > >>> available) > > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 > > >>> dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_ > 20170123T180550+.log.gz", > > >>> {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 > > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > > >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_ > 20170123T180550+.log.gz", > > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No > data > > >>> available) > > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > > >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_ > 20170123T180550+.log.gz", > > >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No > data > > >>> available) > > >>> > > >>> I have found the below existing issue which is very similar to my > > >>> scenario. > > >>> > > >>> https://bugzilla.redhat.com/show_bug.cgi?id=1298258 > > >>> > > >>> We are using the gluster-3.7.6 and it seems that the issue is fixed > in > > >>> 3.8.4 version. > > >>> > > >>> Could you please let me know why it showing the number of above logs > and > > >>> reason behind it as it is not explained in the above bug. > > >>> > > >>> Regards, > > >>> Abhishek > > >>> > > >>> -- > > >>> > > >>> > > >>> > > >>> > > >>> Regards > > >>> Abhishek Paliwal > > >>> > > >> > > >> > > >> > > >> -- > > >> > > >> > > >> > > >> > > >> Regards > > >> Abhishek Paliwal > > >> > > >> ___ > > >> Gluster-users mailing list > > >> Gluster-users@gluster.org > > >> http://lists.gluster.org/mailman/listinfo/gluster-users > > >> > > > > > > > > > > > > -- > > > Pranith > > > > > > > > > > > -- > > > > > > > > > > Regards > > Abhishek Paliwal > > > -- Regards Abhishek Paliwal ___ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] High load on glusterfsd process
Hi Abhishek, Bitrot requires versioning of files to be down on writes. This was being done irrespective of whether bitrot is enabled or not. This takes considerable CPU. With the fix https://review.gluster.org/#/c/14442/, it is made optional and is enabled only with bitrot. If bitrot is not enabled, then you won't see any setxattr/getxattrs related to bitrot. The fix would be available in 3.11. Thanks and Regards, Kotresh H R - Original Message - > From: "ABHISHEK PALIWAL" > To: "Pranith Kumar Karampuri" > Cc: "Gluster Devel" , "gluster-users" > , "Kotresh Hiremath > Ravishankar" > Sent: Monday, April 24, 2017 11:30:57 AM > Subject: Re: [Gluster-users] High load on glusterfsd process > > Hi Kotresh, > > Could you please update me on this? > > Regards, > Abhishek > > On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < > pkara...@redhat.com> wrote: > > > +Kotresh who seems to have worked on the bug you mentioned. > > > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < > > abhishpali...@gmail.com> wrote: > > > >> > >> If the patch provided in that case will resolve my bug as well then > >> please provide the patch so that I will backport it on 3.7.6 > >> > >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < > >> abhishpali...@gmail.com> wrote: > >> > >>> Hi Team, > >>> > >>> I have noticed that there are so many glusterfsd threads are running in > >>> my system and we observed some of those thread consuming more cpu. I > >>> did “strace” on two such threads (before the problem disappeared by > >>> itself) > >>> and found that there is a continuous activity like below: > >>> > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 > >>> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", > >>> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data > >>> available) > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", > >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data > >>> available) > >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 > >>> dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", > >>> {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", > >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data > >>> available) > >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 > >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", > >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data > >>> available) > >>> > >>> I have found the below existing issue which is very similar to my > >>> scenario. > >>> > >>> https://bugzilla.redhat.com/show_bug.cgi?id=1298258 > >>> > >>> We are using the gluster-3.7.6 and it seems that the issue is fixed in > >>> 3.8.4 version. > >>> > >>> Could you please let me know why it showing the number of above logs and > >>> reason behind it as it is not explained in the above bug. > >>> > >>> Regards, > >>> Abhishek > >>> > >>> -- > >>> > >>> > >>> > >>> > >>> Regards > >>> Abhishek Paliwal > >>> > >> > >> > >> > >> -- > >> > >> > >> > >> > >> Regards > >> Abhishek Paliwal > >> > >> ___ > >> Gluster-users mailing list > >> Gluster-users@gluster.org > >> http://lists.gluster.org/mailman/listinfo/gluster-users > >> > > > > > > > > -- > > Pranith > > > > > > -- > > > > > Regards > Abhishek Paliwal > ___ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] High load on glusterfsd process
Hi Kotresh, Could you please update me on this? Regards, Abhishek On Sat, Apr 22, 2017 at 12:31 PM, Pranith Kumar Karampuri < pkara...@redhat.com> wrote: > +Kotresh who seems to have worked on the bug you mentioned. > > On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL < > abhishpali...@gmail.com> wrote: > >> >> If the patch provided in that case will resolve my bug as well then >> please provide the patch so that I will backport it on 3.7.6 >> >> On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < >> abhishpali...@gmail.com> wrote: >> >>> Hi Team, >>> >>> I have noticed that there are so many glusterfsd threads are running in >>> my system and we observed some of those thread consuming more cpu. I >>> did “strace” on two such threads (before the problem disappeared by itself) >>> and found that there is a continuous activity like below: >>> >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >>> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", >>> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data >>> available) >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >>> f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data >>> available) >>> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >>> dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", >>> {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", >>> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data >>> available) >>> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92 >>> f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", >>> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data >>> available) >>> >>> I have found the below existing issue which is very similar to my >>> scenario. >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=1298258 >>> >>> We are using the gluster-3.7.6 and it seems that the issue is fixed in >>> 3.8.4 version. >>> >>> Could you please let me know why it showing the number of above logs and >>> reason behind it as it is not explained in the above bug. >>> >>> Regards, >>> Abhishek >>> >>> -- >>> >>> >>> >>> >>> Regards >>> Abhishek Paliwal >>> >> >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> >> ___ >> Gluster-users mailing list >> Gluster-users@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-users >> > > > > -- > Pranith > -- Regards Abhishek Paliwal ___ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] High load on glusterfsd process
+Kotresh who seems to have worked on the bug you mentioned. On Fri, Apr 21, 2017 at 12:21 PM, ABHISHEK PALIWAL wrote: > > If the patch provided in that case will resolve my bug as well then please > provide the patch so that I will backport it on 3.7.6 > > On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL < > abhishpali...@gmail.com> wrote: > >> Hi Team, >> >> I have noticed that there are so many glusterfsd threads are running in >> my system and we observed some of those thread consuming more cpu. I did >> “strace” on two such threads (before the problem disappeared by itself) and >> found that there is a continuous activity like below: >> >> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >> dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", >> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 >> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3- >> 92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482- >> 425_20170126T113552+.log.gz", "trusted.bit-rot.bad-file", >> 0x3fff81f58550, 255) = -1 ENODATA (No data available) >> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3- >> 92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482- >> 425_20170126T113552+.log.gz", "trusted.bit-rot.signature", >> 0x3fff81f58550, 255) = -1 ENODATA (No data available) >> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4 >> dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", >> {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 >> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3- >> 92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", >> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data >> available) >> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3- >> 92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", >> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data >> available) >> >> I have found the below existing issue which is very similar to my >> scenario. >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1298258 >> >> We are using the gluster-3.7.6 and it seems that the issue is fixed in >> 3.8.4 version. >> >> Could you please let me know why it showing the number of above logs and >> reason behind it as it is not explained in the above bug. >> >> Regards, >> Abhishek >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> > > > > -- > > > > > Regards > Abhishek Paliwal > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users > -- Pranith ___ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] High load on glusterfsd process
If the patch provided in that case will resolve my bug as well then please provide the patch so that I will backport it on 3.7.6 On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL wrote: > Hi Team, > > I have noticed that there are so many glusterfsd threads are running in my > system and we observed some of those thread consuming more cpu. I did > “strace” on two such threads (before the problem disappeared by itself) and > found that there is a continuous activity like below: > > lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8- > 4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", > {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 > lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f- > 246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", > "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data > available) > lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f- > 246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", > "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data > available) > lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8- > 4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", > {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 > lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f- > 246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", > "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data > available) > lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f- > 246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", > "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data > available) > > I have found the below existing issue which is very similar to my scenario. > > https://bugzilla.redhat.com/show_bug.cgi?id=1298258 > > We are using the gluster-3.7.6 and it seems that the issue is fixed in > 3.8.4 version. > > Could you please let me know why it showing the number of above logs and > reason behind it as it is not explained in the above bug. > > Regards, > Abhishek > > -- > > > > > Regards > Abhishek Paliwal > -- Regards Abhishek Paliwal ___ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] High load on glusterfsd process
Hi Team, I have noticed that there are so many glusterfsd threads are running in my system and we observed some of those thread consuming more cpu. I did “strace” on two such threads (before the problem disappeared by itself) and found that there is a continuous activity like below: lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0 lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data available) lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz", "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data available) lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", {st_mode=S_IFREG|0670, st_size=169, ...}) = 0 lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data available) lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz", "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data available) I have found the below existing issue which is very similar to my scenario. https://bugzilla.redhat.com/show_bug.cgi?id=1298258 We are using the gluster-3.7.6 and it seems that the issue is fixed in 3.8.4 version. Could you please let me know why it showing the number of above logs and reason behind it as it is not explained in the above bug. Regards, Abhishek -- Regards Abhishek Paliwal ___ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users