Re: [Gluster-users] glusterd crashing

[Kandalf ®]

Hi,
I don't have that problem with gluster crashing. I have write poor performance 
when linux or vmware esxi write to an raw image file like vmdk files.
Thanks!
 


 On Friday, October 16, 2015 3:08 PM, Gene Liverman  
wrote:
   

 Bug filed https://bugzilla.redhat.com/show_bug.cgi?id=1272436





--Gene LivermanSystems Integration ArchitectInformation Technology 
ServicesUniversity of West Georgiagliverma@westga.edu678.839.5492
ITS: Making Technology Work for You!



On Thu, Oct 8, 2015 at 1:48 PM, Vijay Bellur  wrote:

On Thursday 08 October 2015 11:01 PM, Gene Liverman wrote:

Happy to do so... what all info should go in the bug report?



Guidelines for logging a bug are available at [1]. Please try to provide 
relevant data requested in the Package Information, Cluster Information, Volume 
Information and Logs section of the guidelines page. Information pertaining to 
bricks, volume details & statedump can be skipped as it is the management 
daemon that is crashing here. Attaching the entire glusterd log would be 
helpful.

Regards,
Vijay

[1] 
http://www.gluster.org/community/documentation/index.php/Bug_reporting_guidelines




___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

  ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] glusterd crashing

[Gene Liverman]

Bug filed https://bugzilla.redhat.com/show_bug.cgi?id=1272436





--
*Gene Liverman*
Systems Integration Architect
Information Technology Services
University of West Georgia
glive...@westga.edu
678.839.5492

ITS: Making Technology Work for You!




On Thu, Oct 8, 2015 at 1:48 PM, Vijay Bellur  wrote:

> On Thursday 08 October 2015 11:01 PM, Gene Liverman wrote:
>
>> Happy to do so... what all info should go in the bug report?
>>
>>
> Guidelines for logging a bug are available at [1]. Please try to provide
> relevant data requested in the Package Information, Cluster Information,
> Volume Information and Logs section of the guidelines page. Information
> pertaining to bricks, volume details & statedump can be skipped as it is
> the management daemon that is crashing here. Attaching the entire glusterd
> log would be helpful.
>
> Regards,
> Vijay
>
> [1]
> http://www.gluster.org/community/documentation/index.php/Bug_reporting_guidelines
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] glusterd crashing

[Vijay Bellur]

On Thursday 08 October 2015 11:01 PM, Gene Liverman wrote:

Happy to do so... what all info should go in the bug report?



Guidelines for logging a bug are available at [1]. Please try to provide 
relevant data requested in the Package Information, Cluster Information, 
Volume Information and Logs section of the guidelines page. Information 
pertaining to bricks, volume details & statedump can be skipped as it is 
the management daemon that is crashing here. Attaching the entire 
glusterd log would be helpful.


Regards,
Vijay

[1] 
http://www.gluster.org/community/documentation/index.php/Bug_reporting_guidelines


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] glusterd crashing

[Gene Liverman]

Happy to do so... what all info should go in the bug report?





--
*Gene Liverman*
Systems Integration Architect
Information Technology Services
University of West Georgia
glive...@westga.edu
678.839.5492

ITS: Making Technology Work for You!




On Thu, Oct 8, 2015 at 1:04 PM, Vijay Bellur  wrote:

> On Wednesday 07 October 2015 09:20 PM, Atin Mukherjee wrote:
>
>> This looks like a glibc corruption to me. Which distribution platform
>> are you running Gluster on?
>>
>>
> A crash in glibc would mostly be due to memory corruption caused by the
> application. Can we please open a tracking bug if not done yet?
>
> Thanks,
> Vijay
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] glusterd crashing

[Vijay Bellur]

On Wednesday 07 October 2015 09:20 PM, Atin Mukherjee wrote:

This looks like a glibc corruption to me. Which distribution platform
are you running Gluster on?



A crash in glibc would mostly be due to memory corruption caused by the 
application. Can we please open a tracking bug if not done yet?


Thanks,
Vijay

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] glusterd crashing

[Atin Mukherjee]

n Wed, Oct 7, 2015 at 12:06 AM, Atin Mukherjee 
>>> wrote:
>>>
>>>>
>>>>
>>>> On 10/07/2015 09:34 AM, Atin Mukherjee wrote:
>>>> >
>>>> >
>>>> > On 10/06/2015 08:15 PM, Gene Liverman wrote:
>>>> >> Sorry for the delay... they joys of multiple proverbial fires at
>>>> once.
>>>> >> In /var/log/messages I found this for our most recent crash:
>>>> >>
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> pending frames:
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> patchset: git://git.gluster.com/glusterfs.git
>>>> >> <http://git.gluster.com/glusterfs.git>
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> signal received: 6
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> time
>>>> >> of crash:
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> 2015-10-03 04:26:21
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> configuration details:
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> argp 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> backtrace 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> dlfcn 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> libpthread 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> llistxattr 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> setfsid 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> spinlock 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> epoll.h 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> xattr.h 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> st_atim.tv_nsec 1
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> >> package-string: glusterfs 3.7.4
>>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>>> -
>>>> >>
>>>> >>
>>>> >> I have posted etc-glusterfs-glusterd.vol.log
>>>> >> to http://pastebin.com/Pzq1j5J3. I also put the core file and an
>>>> >> sosreport on my web server for you but don't want to leave them there
>>>> >> for long so I'd appreciate it if you'd let me know once you get them.
>>>> >> They are at the following url's:
>>>> >> http://www.westga.edu/~gliverma/tmp-files/core.36992
>>>> > Could you get the backtrace and share with us with the following
>>>> commands:
>>>> >
>>>> > $ gdb glusterd2 
>>>> > $ bt
>>>> Also "t a a bt" output in gdb might help.
>>>> >
>>>> >>
>>>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz
>>>> >>
>>>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz.md5
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> Thanks again for the help!
>>>> >> *Gene Liverman*
>>>> >> Systems Integration Architect
>>>> >> Information Technology Services
>>>> >> University of West Georgia
>>>> >> glive...@westga.edu <mailto:glive...@westga.edu>
>>>> >>
>>>> >> ITS: Making Technology Work for You!
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg >>> >> <mailto:gg...@redhat.com>> wrote:
>>>> >>
>>>> >> >> Pulling those logs now but ho

Re: [Gluster-users] glusterd crashing

[Gene Liverman]

gt; >> 2015-10-03 04:26:21
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> configuration details:
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> argp 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> backtrace 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> dlfcn 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> libpthread 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> llistxattr 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> setfsid 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> spinlock 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> epoll.h 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> xattr.h 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> st_atim.tv_nsec 1
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> >> package-string: glusterfs 3.7.4
>>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>>> -
>>> >>
>>> >>
>>> >> I have posted etc-glusterfs-glusterd.vol.log
>>> >> to http://pastebin.com/Pzq1j5J3. I also put the core file and an
>>> >> sosreport on my web server for you but don't want to leave them there
>>> >> for long so I'd appreciate it if you'd let me know once you get them.
>>> >> They are at the following url's:
>>> >> http://www.westga.edu/~gliverma/tmp-files/core.36992
>>> > Could you get the backtrace and share with us with the following
>>> commands:
>>> >
>>> > $ gdb glusterd2 
>>> > $ bt
>>> Also "t a a bt" output in gdb might help.
>>> >
>>> >>
>>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz
>>> >>
>>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz.md5
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> Thanks again for the help!
>>> >> *Gene Liverman*
>>> >> Systems Integration Architect
>>> >> Information Technology Services
>>> >> University of West Georgia
>>> >> glive...@westga.edu <mailto:glive...@westga.edu>
>>> >>
>>> >> ITS: Making Technology Work for You!
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg >> >> <mailto:gg...@redhat.com>> wrote:
>>> >>
>>> >> >> Pulling those logs now but how do I generate the core file you
>>> are asking
>>> >> for?
>>> >>
>>> >> When there is crash then core file automatically generated based
>>> on
>>> >> your *ulimit* set option. you can find location of core file in
>>> your
>>> >> root or current working directory or where ever you have set your
>>> >> core dump file location. core file gives you information regarding
>>> >> crash, where exactly crash happened.
>>> >> you can find appropriate core file by looking at crash time in
>>> >> glusterd log's by searching "crash" keyword. you can also paste
>>> few
>>> >> line's just above latest "crash" keyword in glusterd logs.
>>> >>
>>> >> Just for your curiosity if you willing to look where it crash then
>>> >> you can debug it by #gdb -c  glusterd
>>> >>
>>> >> Thank you...
>>> >>
>>> >> Regards,
>>> >> Gaurav
>>> >>
>>> >> - Original Message -
>>> >> From: "Gene Liverman" >> glive...@westga.edu>>
>>> >> To: "Gaurav Garg" mailto:gg...@redhat.com>>
>>> >> Cc: "gluster-users" >>

Re: [Gluster-users] glusterd crashing

[Atin Mukherjee]

:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> >> package-string: glusterfs 3.7.4
>> >> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> -
>> >>
>> >>
>> >> I have posted etc-glusterfs-glusterd.vol.log
>> >> to http://pastebin.com/Pzq1j5J3. I also put the core file and an
>> >> sosreport on my web server for you but don't want to leave them there
>> >> for long so I'd appreciate it if you'd let me know once you get them.
>> >> They are at the following url's:
>> >> http://www.westga.edu/~gliverma/tmp-files/core.36992
>> > Could you get the backtrace and share with us with the following
>> commands:
>> >
>> > $ gdb glusterd2 
>> > $ bt
>> Also "t a a bt" output in gdb might help.
>> >
>> >>
>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz
>> >>
>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz.md5
>> >>
>> >>
>> >>
>> >>
>> >> Thanks again for the help!
>> >> *Gene Liverman*
>> >> Systems Integration Architect
>> >> Information Technology Services
>> >> University of West Georgia
>> >> glive...@westga.edu <mailto:glive...@westga.edu>
>> >>
>> >> ITS: Making Technology Work for You!
>> >>
>> >>
>> >>
>> >>
>> >> On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg > >> <mailto:gg...@redhat.com>> wrote:
>> >>
>> >> >> Pulling those logs now but how do I generate the core file you
>> are asking
>> >> for?
>> >>
>> >> When there is crash then core file automatically generated based on
>> >> your *ulimit* set option. you can find location of core file in
>> your
>> >> root or current working directory or where ever you have set your
>> >> core dump file location. core file gives you information regarding
>> >> crash, where exactly crash happened.
>> >> you can find appropriate core file by looking at crash time in
>> >> glusterd log's by searching "crash" keyword. you can also paste few
>> >> line's just above latest "crash" keyword in glusterd logs.
>> >>
>> >> Just for your curiosity if you willing to look where it crash then
>> >> you can debug it by #gdb -c  glusterd
>> >>
>> >> Thank you...
>> >>
>> >> Regards,
>> >> Gaurav
>> >>
>> >> - Original Message -
>> >> From: "Gene Liverman" > glive...@westga.edu>>
>> >> To: "Gaurav Garg" mailto:gg...@redhat.com>>
>> >> Cc: "gluster-users" > >> <mailto:gluster-users@gluster.org>>
>> >> Sent: Friday, October 2, 2015 8:28:49 PM
>> >> Subject: Re: [Gluster-users] glusterd crashing
>> >>
>> >> Pulling those logs now but how do I generate the core file you are
>> >> asking
>> >> for?
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> *Gene Liverman*
>> >> Systems Integration Architect
>> >> Information Technology Services
>> >> University of West Georgia
>> >> glive...@westga.edu <mailto:glive...@westga.edu>
>> >> 678.839.5492 
>> >>
>> >> ITS: Making Technology Work for You!
>> >>
>> >>
>> >>
>> >>
>> >> On Fri, Oct 2, 2015 at 2:25 AM, Gaurav Garg > >> <mailto:gg...@redhat.com>> wrote:
>> >>
>> >> > Hi Gene,
>> >> >
>> >> > you have paste glustershd log. we asked you to paste glusterd
>> log.
>> >> > glusterd and glustershd both are different process. with this
>> >> information
>> >> > we can't find out why your glusterd crashed. could you paste
>> >> *glusterd*
>> >> > logs
>> (/var/log/glusterfs/usr-local-etc-glusterfs-glusterd.vol.log*) in
>> >> > pastebin (not in this mail thread) and give the link of

Re: [Gluster-users] glusterd crashing

[Gene Liverman]

ervices
> >> University of West Georgia
> >> glive...@westga.edu <mailto:glive...@westga.edu>
> >>
> >> ITS: Making Technology Work for You!
> >>
> >>
> >>
> >>
> >> On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg  >> <mailto:gg...@redhat.com>> wrote:
> >>
> >> >> Pulling those logs now but how do I generate the core file you
> are asking
> >> for?
> >>
> >> When there is crash then core file automatically generated based on
> >> your *ulimit* set option. you can find location of core file in your
> >> root or current working directory or where ever you have set your
> >> core dump file location. core file gives you information regarding
> >> crash, where exactly crash happened.
> >> you can find appropriate core file by looking at crash time in
> >> glusterd log's by searching "crash" keyword. you can also paste few
> >> line's just above latest "crash" keyword in glusterd logs.
> >>
> >> Just for your curiosity if you willing to look where it crash then
> >> you can debug it by #gdb -c  glusterd
> >>
> >> Thank you...
> >>
> >> Regards,
> >> Gaurav
> >>
> >> - Original Message -
> >> From: "Gene Liverman"  glive...@westga.edu>>
> >> To: "Gaurav Garg" mailto:gg...@redhat.com>>
> >> Cc: "gluster-users"  >> <mailto:gluster-users@gluster.org>>
> >> Sent: Friday, October 2, 2015 8:28:49 PM
> >> Subject: Re: [Gluster-users] glusterd crashing
> >>
> >> Pulling those logs now but how do I generate the core file you are
> >> asking
> >> for?
> >>
> >>
> >>
> >>
> >>
> >> --
> >> *Gene Liverman*
> >> Systems Integration Architect
> >> Information Technology Services
> >> University of West Georgia
> >> glive...@westga.edu <mailto:glive...@westga.edu>
> >> 678.839.5492 
> >>
> >> ITS: Making Technology Work for You!
> >>
> >>
> >>
> >>
> >> On Fri, Oct 2, 2015 at 2:25 AM, Gaurav Garg  >> <mailto:gg...@redhat.com>> wrote:
> >>
> >> > Hi Gene,
> >> >
> >> > you have paste glustershd log. we asked you to paste glusterd log.
> >> > glusterd and glustershd both are different process. with this
> >> information
> >> > we can't find out why your glusterd crashed. could you paste
> >> *glusterd*
> >> > logs
> (/var/log/glusterfs/usr-local-etc-glusterfs-glusterd.vol.log*) in
> >> > pastebin (not in this mail thread) and give the link of pastebin
> >> in this
> >> > mail thread. Can you also attach core file or you can paste
> >> backtrace of
> >> > that core dump file.
> >> > It will be great if you give us sos report of the node where the
> crash
> >> > happen.
> >> >
> >> > Thanx,
> >> >
> >> > ~Gaurav
> >> >
> >> > - Original Message -
> >> > From: "Gene Liverman"  >> <mailto:glive...@westga.edu>>
> >> > To: "gluster-users"  >> <mailto:gluster-users@gluster.org>>
> >> > Sent: Friday, October 2, 2015 4:47:00 AM
> >> > Subject: Re: [Gluster-users] glusterd crashing
> >> >
> >> > Sorry for the delay. Here is what's installed:
> >> > # rpm -qa | grep gluster
> >> > glusterfs-geo-replication-3.7.4-2.el6.x86_64
> >> > glusterfs-client-xlators-3.7.4-2.el6.x86_64
> >> > glusterfs-3.7.4-2.el6.x86_64
> >> > glusterfs-libs-3.7.4-2.el6.x86_64
> >> > glusterfs-api-3.7.4-2.el6.x86_64
> >> > glusterfs-fuse-3.7.4-2.el6.x86_64
> >> > glusterfs-server-3.7.4-2.el6.x86_64
> >> > glusterfs-cli-3.7.4-2.el6.x86_64
> >> >
> >> > The cmd_history.log file is attached.
> >> > In gluster.log I have filtered out a bunch of lines like the one
> >> below due
> >> > to make them more readable. I had a node down for multiple days
> due

Re: [Gluster-users] glusterd crashing

[Atin Mukherjee]

On 10/07/2015 09:34 AM, Atin Mukherjee wrote:
> 
> 
> On 10/06/2015 08:15 PM, Gene Liverman wrote:
>> Sorry for the delay... they joys of multiple proverbial fires at once.
>> In /var/log/messages I found this for our most recent crash:
>>
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> pending frames:
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> patchset: git://git.gluster.com/glusterfs.git
>> <http://git.gluster.com/glusterfs.git>
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> signal received: 6
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: time
>> of crash:
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> 2015-10-03 04:26:21
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> configuration details:
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: argp 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> backtrace 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: dlfcn 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> libpthread 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> llistxattr 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: setfsid 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> spinlock 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: epoll.h 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: xattr.h 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> st_atim.tv_nsec 1
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
>> package-string: glusterfs 3.7.4
>> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: -
>>
>>
>> I have posted etc-glusterfs-glusterd.vol.log
>> to http://pastebin.com/Pzq1j5J3. I also put the core file and an
>> sosreport on my web server for you but don't want to leave them there
>> for long so I'd appreciate it if you'd let me know once you get them.
>> They are at the following url's:
>> http://www.westga.edu/~gliverma/tmp-files/core.36992
> Could you get the backtrace and share with us with the following commands:
> 
> $ gdb glusterd2 
> $ bt
Also "t a a bt" output in gdb might help.
> 
>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz
>> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz.md5
>>
>>
>>
>>
>> Thanks again for the help!
>> *Gene Liverman*
>> Systems Integration Architect
>> Information Technology Services
>> University of West Georgia
>> glive...@westga.edu <mailto:glive...@westga.edu>
>>
>> ITS: Making Technology Work for You!
>>
>>
>>
>>
>> On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg > <mailto:gg...@redhat.com>> wrote:
>>
>> >> Pulling those logs now but how do I generate the core file you are 
>> asking
>> for?
>>
>> When there is crash then core file automatically generated based on
>> your *ulimit* set option. you can find location of core file in your
>> root or current working directory or where ever you have set your
>> core dump file location. core file gives you information regarding
>> crash, where exactly crash happened.
>> you can find appropriate core file by looking at crash time in
>> glusterd log's by searching "crash" keyword. you can also paste few
>> line's just above latest "crash" keyword in glusterd logs.
>>
>> Just for your curiosity if you willing to look where it crash then
>> you can debug it by #gdb -c  glusterd
>>
>> Thank you...
>>
>> Regards,
>> Gaurav
>>
>> - Original Message -
>> From: "Gene Liverman" mailto:glive...@westga.edu>>
>> To: "Gaurav Garg" mailto:gg...@redhat.com>>
>> Cc: "gluster-users" > <mailto:gluster-users@gluster.org>>
>> Sent: Friday, October 2, 2015 8:28:49 PM
>> Subject: Re: [Gluster-users] glusterd crashing
>>
>> Pulling those logs now but how do I generate the core file you are
>> asking
>> for?
>>
>>
>>
>>
>>
>> --
>

Re: [Gluster-users] glusterd crashing

[Atin Mukherjee]

On 10/06/2015 08:15 PM, Gene Liverman wrote:
> Sorry for the delay... they joys of multiple proverbial fires at once.
> In /var/log/messages I found this for our most recent crash:
> 
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> pending frames:
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> patchset: git://git.gluster.com/glusterfs.git
> <http://git.gluster.com/glusterfs.git>
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> signal received: 6
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: time
> of crash:
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> 2015-10-03 04:26:21
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> configuration details:
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: argp 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> backtrace 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: dlfcn 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> libpthread 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> llistxattr 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: setfsid 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> spinlock 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: epoll.h 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: xattr.h 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> st_atim.tv_nsec 1
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
> package-string: glusterfs 3.7.4
> Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: -
> 
> 
> I have posted etc-glusterfs-glusterd.vol.log
> to http://pastebin.com/Pzq1j5J3. I also put the core file and an
> sosreport on my web server for you but don't want to leave them there
> for long so I'd appreciate it if you'd let me know once you get them.
> They are at the following url's:
> http://www.westga.edu/~gliverma/tmp-files/core.36992
Could you get the backtrace and share with us with the following commands:

$ gdb glusterd2 
$ bt

> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz
> http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz.md5
> 
> 
> 
> 
> Thanks again for the help!
> *Gene Liverman*
> Systems Integration Architect
> Information Technology Services
> University of West Georgia
> glive...@westga.edu <mailto:glive...@westga.edu>
> 
> ITS: Making Technology Work for You!
> 
> 
> 
> 
> On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg  <mailto:gg...@redhat.com>> wrote:
> 
> >> Pulling those logs now but how do I generate the core file you are 
> asking
> for?
> 
> When there is crash then core file automatically generated based on
> your *ulimit* set option. you can find location of core file in your
> root or current working directory or where ever you have set your
> core dump file location. core file gives you information regarding
> crash, where exactly crash happened.
> you can find appropriate core file by looking at crash time in
> glusterd log's by searching "crash" keyword. you can also paste few
> line's just above latest "crash" keyword in glusterd logs.
> 
> Just for your curiosity if you willing to look where it crash then
> you can debug it by #gdb -c  glusterd
> 
> Thank you...
> 
> Regards,
> Gaurav
> 
> - Original Message -
> From: "Gene Liverman" mailto:glive...@westga.edu>>
> To: "Gaurav Garg" mailto:gg...@redhat.com>>
> Cc: "gluster-users"  <mailto:gluster-users@gluster.org>>
> Sent: Friday, October 2, 2015 8:28:49 PM
> Subject: Re: [Gluster-users] glusterd crashing
> 
> Pulling those logs now but how do I generate the core file you are
> asking
> for?
> 
> 
> 
> 
> 
> --
> *Gene Liverman*
> Systems Integration Architect
> Information Technology Services
> University of West Georgia
> glive...@westga.edu <mailto:glive...@westga.edu>
> 678.839.5492 
> 
> ITS: Making Technology Work for You!
> 
> 
> 
> 
> On Fri, Oct 2, 2015 at 2:25 AM, Gaurav Garg  <mailto:gg...@redhat.com>> wrote:
> 
> > Hi Gene,
> >
> > you have paste glustershd log. we asked you to paste glusterd 

Re: [Gluster-users] glusterd crashing

[Gene Liverman]

Sorry for the delay... they joys of multiple proverbial fires at once. In
/var/log/messages I found this for our most recent crash:

Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: pending
frames:
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
patchset: git://git.gluster.com/glusterfs.git
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: signal
received: 6
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: time of
crash:
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
2015-10-03 04:26:21
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
configuration details:
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: argp 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
backtrace 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: dlfcn 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
libpthread 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
llistxattr 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: setfsid 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: spinlock
1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: epoll.h 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: xattr.h 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
st_atim.tv_nsec 1
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]:
package-string: glusterfs 3.7.4
Oct  3 00:26:21 eapps-gluster01 etc-glusterfs-glusterd.vol[36992]: -


I have posted etc-glusterfs-glusterd.vol.log to http://pastebin.com/Pzq1j5J3.
I also put the core file and an sosreport on my web server for you but
don't want to leave them there for long so I'd appreciate it if you'd let
me know once you get them. They are at the following url's:
http://www.westga.edu/~gliverma/tmp-files/core.36992
http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz
http://www.westga.edu/~gliverma/tmp-files/sosreport-gliverman.gluster-crashing-20151006101239.tar.xz.md5




Thanks again for the help!
*Gene Liverman*
Systems Integration Architect
Information Technology Services
University of West Georgia
glive...@westga.edu

ITS: Making Technology Work for You!




On Fri, Oct 2, 2015 at 11:18 AM, Gaurav Garg  wrote:

> >> Pulling those logs now but how do I generate the core file you are
> asking
> for?
>
> When there is crash then core file automatically generated based on your
> *ulimit* set option. you can find location of core file in your root or
> current working directory or where ever you have set your core dump file
> location. core file gives you information regarding crash, where exactly
> crash happened.
> you can find appropriate core file by looking at crash time in glusterd
> log's by searching "crash" keyword. you can also paste few line's just
> above latest "crash" keyword in glusterd logs.
>
> Just for your curiosity if you willing to look where it crash then you can
> debug it by #gdb -c  glusterd
>
> Thank you...
>
> Regards,
> Gaurav
>
> - Original Message -
> From: "Gene Liverman" 
> To: "Gaurav Garg" 
> Cc: "gluster-users" 
> Sent: Friday, October 2, 2015 8:28:49 PM
> Subject: Re: [Gluster-users] glusterd crashing
>
> Pulling those logs now but how do I generate the core file you are asking
> for?
>
>
>
>
>
> --
> *Gene Liverman*
> Systems Integration Architect
> Information Technology Services
> University of West Georgia
> glive...@westga.edu
> 678.839.5492
>
> ITS: Making Technology Work for You!
>
>
>
>
> On Fri, Oct 2, 2015 at 2:25 AM, Gaurav Garg  wrote:
>
> > Hi Gene,
> >
> > you have paste glustershd log. we asked you to paste glusterd log.
> > glusterd and glustershd both are different process. with this information
> > we can't find out why your glusterd crashed. could you paste *glusterd*
> > logs (/var/log/glusterfs/usr-local-etc-glusterfs-glusterd.vol.log*) in
> > pastebin (not in this mail thread) and give the link of pastebin in this
> > mail thread. Can you also attach core file or you can paste backtrace of
> > that core dump file.
> > It will be great if you give us sos report of the node where the crash
> > happen.
> >
> > Thanx,
> >
> > ~Gaurav
> >
> > - Original Message -
> > From: "Gene Liverman" 
> > To: "gluster-users" 
> > Sent: Friday, October 2, 2015 4:47:00 AM
> > Subject: Re: [Gluster-users] glusterd crashing
> >
> > Sorry for the delay. Here is what's installed:
> >

Re: [Gluster-users] glusterd crashing

[Gaurav Garg]

>> Pulling those logs now but how do I generate the core file you are asking
for?

When there is crash then core file automatically generated based on your 
*ulimit* set option. you can find location of core file in your root or current 
working directory or where ever you have set your core dump file location. core 
file gives you information regarding crash, where exactly crash happened.
you can find appropriate core file by looking at crash time in glusterd log's 
by searching "crash" keyword. you can also paste few line's just above latest 
"crash" keyword in glusterd logs.

Just for your curiosity if you willing to look where it crash then you can 
debug it by #gdb -c  glusterd

Thank you...

Regards,
Gaurav  

- Original Message -
From: "Gene Liverman" 
To: "Gaurav Garg" 
Cc: "gluster-users" 
Sent: Friday, October 2, 2015 8:28:49 PM
Subject: Re: [Gluster-users] glusterd crashing

Pulling those logs now but how do I generate the core file you are asking
for?





--
*Gene Liverman*
Systems Integration Architect
Information Technology Services
University of West Georgia
glive...@westga.edu
678.839.5492

ITS: Making Technology Work for You!




On Fri, Oct 2, 2015 at 2:25 AM, Gaurav Garg  wrote:

> Hi Gene,
>
> you have paste glustershd log. we asked you to paste glusterd log.
> glusterd and glustershd both are different process. with this information
> we can't find out why your glusterd crashed. could you paste *glusterd*
> logs (/var/log/glusterfs/usr-local-etc-glusterfs-glusterd.vol.log*) in
> pastebin (not in this mail thread) and give the link of pastebin in this
> mail thread. Can you also attach core file or you can paste backtrace of
> that core dump file.
> It will be great if you give us sos report of the node where the crash
> happen.
>
> Thanx,
>
> ~Gaurav
>
> - Original Message -
> From: "Gene Liverman" 
> To: "gluster-users" 
> Sent: Friday, October 2, 2015 4:47:00 AM
> Subject: Re: [Gluster-users] glusterd crashing
>
> Sorry for the delay. Here is what's installed:
> # rpm -qa | grep gluster
> glusterfs-geo-replication-3.7.4-2.el6.x86_64
> glusterfs-client-xlators-3.7.4-2.el6.x86_64
> glusterfs-3.7.4-2.el6.x86_64
> glusterfs-libs-3.7.4-2.el6.x86_64
> glusterfs-api-3.7.4-2.el6.x86_64
> glusterfs-fuse-3.7.4-2.el6.x86_64
> glusterfs-server-3.7.4-2.el6.x86_64
> glusterfs-cli-3.7.4-2.el6.x86_64
>
> The cmd_history.log file is attached.
> In gluster.log I have filtered out a bunch of lines like the one below due
> to make them more readable. I had a node down for multiple days due to
> maintenance and another one went down due to a hardware failure during that
> time too.
> [2015-10-01 00:16:09.643631] W [MSGID: 114031]
> [client-rpc-fops.c:2971:client3_3_lookup_cbk] 0-gv0-client-0: remote
> operation failed. Path: 
> (31f17f8c-6c96-4440-88c0-f813b3c8d364) [No such file or directory]
>
> I also filtered out a boat load of self heal lines like these two:
> [2015-10-01 15:14:14.851015] I [MSGID: 108026]
> [afr-self-heal-metadata.c:56:__afr_selfheal_metadata_do] 0-gv0-replicate-0:
> performing metadata selfheal on f78a47db-a359-430d-a655-1d217eb848c3
> [2015-10-01 15:14:14.856392] I [MSGID: 108026]
> [afr-self-heal-common.c:651:afr_log_selfheal] 0-gv0-replicate-0: Completed
> metadata selfheal on f78a47db-a359-430d-a655-1d217eb848c3. source=0 sinks=1
>
>
> [root@eapps-gluster01 glusterfs]# cat glustershd.log |grep -v 'remote
> operation failed' |grep -v 'self-heal'
> [2015-09-27 08:46:56.893125] E [rpc-clnt.c:201:call_bail] 0-glusterfs:
> bailing out frame type(GlusterFS Handshake) op(GETSPEC(2)) xid = 0x6 sent =
> 2015-09-27 08:16:51.742731. timeout = 1800 for 127.0.0.1:24007
> [2015-09-28 12:54:17.524924] W [socket.c:588:__socket_rwv] 0-glusterfs:
> readv on 127.0.0.1:24007 failed (Connection reset by peer)
> [2015-09-28 12:54:27.844374] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk]
> 0-glusterfs: No change in volfile, continuing
> [2015-09-28 12:57:03.485027] W [socket.c:588:__socket_rwv] 0-gv0-client-2:
> readv on 160.10.31.227:24007 failed (Connection reset by peer)
> [2015-09-28 12:57:05.872973] E [socket.c:2278:socket_connect_finish]
> 0-gv0-client-2: connection to 160.10.31.227:24007 failed (Connection
> refused)
> [2015-09-28 12:57:38.490578] W [socket.c:588:__socket_rwv] 0-glusterfs:
> readv on 127.0.0.1:24007 failed (No data available)
> [2015-09-28 12:57:49.054475] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk]
> 0-glusterfs: No change in volfile, continuing
> [2015-09-28 13:01:12.062960] W [glusterfsd.c:1219:cleanup_and_exit]
> (-->/lib64/libpthread.so.0() [0x3c65e07a51]
> -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d

Re: [Gluster-users] glusterd crashing

[Gene Liverman]

Pulling those logs now but how do I generate the core file you are asking
for?





--
*Gene Liverman*
Systems Integration Architect
Information Technology Services
University of West Georgia
glive...@westga.edu
678.839.5492

ITS: Making Technology Work for You!




On Fri, Oct 2, 2015 at 2:25 AM, Gaurav Garg  wrote:

> Hi Gene,
>
> you have paste glustershd log. we asked you to paste glusterd log.
> glusterd and glustershd both are different process. with this information
> we can't find out why your glusterd crashed. could you paste *glusterd*
> logs (/var/log/glusterfs/usr-local-etc-glusterfs-glusterd.vol.log*) in
> pastebin (not in this mail thread) and give the link of pastebin in this
> mail thread. Can you also attach core file or you can paste backtrace of
> that core dump file.
> It will be great if you give us sos report of the node where the crash
> happen.
>
> Thanx,
>
> ~Gaurav
>
> - Original Message -
> From: "Gene Liverman" 
> To: "gluster-users" 
> Sent: Friday, October 2, 2015 4:47:00 AM
> Subject: Re: [Gluster-users] glusterd crashing
>
> Sorry for the delay. Here is what's installed:
> # rpm -qa | grep gluster
> glusterfs-geo-replication-3.7.4-2.el6.x86_64
> glusterfs-client-xlators-3.7.4-2.el6.x86_64
> glusterfs-3.7.4-2.el6.x86_64
> glusterfs-libs-3.7.4-2.el6.x86_64
> glusterfs-api-3.7.4-2.el6.x86_64
> glusterfs-fuse-3.7.4-2.el6.x86_64
> glusterfs-server-3.7.4-2.el6.x86_64
> glusterfs-cli-3.7.4-2.el6.x86_64
>
> The cmd_history.log file is attached.
> In gluster.log I have filtered out a bunch of lines like the one below due
> to make them more readable. I had a node down for multiple days due to
> maintenance and another one went down due to a hardware failure during that
> time too.
> [2015-10-01 00:16:09.643631] W [MSGID: 114031]
> [client-rpc-fops.c:2971:client3_3_lookup_cbk] 0-gv0-client-0: remote
> operation failed. Path: 
> (31f17f8c-6c96-4440-88c0-f813b3c8d364) [No such file or directory]
>
> I also filtered out a boat load of self heal lines like these two:
> [2015-10-01 15:14:14.851015] I [MSGID: 108026]
> [afr-self-heal-metadata.c:56:__afr_selfheal_metadata_do] 0-gv0-replicate-0:
> performing metadata selfheal on f78a47db-a359-430d-a655-1d217eb848c3
> [2015-10-01 15:14:14.856392] I [MSGID: 108026]
> [afr-self-heal-common.c:651:afr_log_selfheal] 0-gv0-replicate-0: Completed
> metadata selfheal on f78a47db-a359-430d-a655-1d217eb848c3. source=0 sinks=1
>
>
> [root@eapps-gluster01 glusterfs]# cat glustershd.log |grep -v 'remote
> operation failed' |grep -v 'self-heal'
> [2015-09-27 08:46:56.893125] E [rpc-clnt.c:201:call_bail] 0-glusterfs:
> bailing out frame type(GlusterFS Handshake) op(GETSPEC(2)) xid = 0x6 sent =
> 2015-09-27 08:16:51.742731. timeout = 1800 for 127.0.0.1:24007
> [2015-09-28 12:54:17.524924] W [socket.c:588:__socket_rwv] 0-glusterfs:
> readv on 127.0.0.1:24007 failed (Connection reset by peer)
> [2015-09-28 12:54:27.844374] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk]
> 0-glusterfs: No change in volfile, continuing
> [2015-09-28 12:57:03.485027] W [socket.c:588:__socket_rwv] 0-gv0-client-2:
> readv on 160.10.31.227:24007 failed (Connection reset by peer)
> [2015-09-28 12:57:05.872973] E [socket.c:2278:socket_connect_finish]
> 0-gv0-client-2: connection to 160.10.31.227:24007 failed (Connection
> refused)
> [2015-09-28 12:57:38.490578] W [socket.c:588:__socket_rwv] 0-glusterfs:
> readv on 127.0.0.1:24007 failed (No data available)
> [2015-09-28 12:57:49.054475] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk]
> 0-glusterfs: No change in volfile, continuing
> [2015-09-28 13:01:12.062960] W [glusterfsd.c:1219:cleanup_and_exit]
> (-->/lib64/libpthread.so.0() [0x3c65e07a51]
> -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xcd) [0x405e4d]
> -->/usr/sbin/glusterfs(cleanup_and_exit+0x65) [0x4059b5] ) 0-: received
> signum (15), shutting down
> [2015-09-28 13:01:12.981945] I [MSGID: 100030] [glusterfsd.c:2301:main]
> 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4
> (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/glustershd -p
> /var/lib/glusterd/glustershd/run/glustershd.pid -l
> /var/log/glusterfs/glustershd.log -S
> /var/run/gluster/9a9819e90404187e84e67b01614bbe10.socket --xlator-option
> *replicate*.node-uuid=416d712a-06fc-4b3c-a92f-8c82145626ff)
> [2015-09-28 13:01:13.009171] I [MSGID: 101190]
> [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread
> with index 1
> [2015-09-28 13:01:13.092483] I [graph.c:269:gf_add_cmdline_options]
> 0-gv0-replicate-0: adding option 'node-uuid' for volume 'gv0-replicate-0'
> with value '416d712a-06fc-4b3c-a92f-8c82145626ff'
&g

Re: [Gluster-users] glusterd crashing

[Gaurav Garg]

Hi Gene,

Could you paste or attach core file/glusterd log file/cmd history to find out 
actual RCA of the crash. What steps you performed for this crash.

>> How can I troubleshoot this?

If you want to troubleshoot this then you can look into the glusterd log file, 
core file.

Thank you..

Regards,
Gaurav

- Original Message -
From: "Gene Liverman" 
To: gluster-users@gluster.org
Sent: Thursday, October 1, 2015 7:59:47 AM
Subject: [Gluster-users] glusterd crashing

In the last few days I've started having issues with my glusterd service 
crashing. When it goes down it seems to do so on all nodes in my replicated 
volume. How can I troubleshoot this? I'm on a mix of CentOS 6 and RHEL 6. 
Thanks! 



Gene Liverman 
Systems Integration Architect 
Information Technology Services 
University of West Georgia 
glive...@westga.edu 


Sent from Outlook on my iPhone 


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] glusterd crashing

[Atin Mukherjee]

Could you attach the core mentioning the Gluster version?

-Atin
Sent from one plus one
On Oct 1, 2015 7:59 AM, "Gene Liverman"  wrote:

> In the last few days I've started having issues with my glusterd service
> crashing. When it goes down it seems to do so on all nodes in my replicated
> volume. How can I troubleshoot this? I'm on a mix of CentOS 6 and RHEL 6.
> Thanks!
>
>
>
> Gene Liverman
> Systems Integration Architect
> Information Technology Services
> University of West Georgia
> glive...@westga.edu
>
>
> Sent from Outlook  on my iPhone
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] glusterd crashing

[Gene Liverman]

In the last few days I've started having issues with my glusterd service 
crashing. When it goes down it seems to do so on all nodes in my replicated 
volume. How can I troubleshoot this? I'm on a mix of CentOS 6 and RHEL 6. 
Thanks!



Gene Liverman 
Systems Integration Architect 
Information Technology Services 
University of West Georgia 
glive...@westga.edu 


Sent from Outlook on my iPhone___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users