date:20160212

Re: [Gluster-devel] NSR: Suggestions for a new name

2016-02-12 Thread Joseph Fernandes

My vote for Consensus Based Replication (CBR)

Also because HONDA CBR ;)
http://www.honda2wheelersindia.com/cbr1000rr/

~Joe
 

- Original Message -
From: "Avra Sengupta" 
To: gluster-devel@gluster.org
Sent: Friday, February 12, 2016 2:05:41 PM
Subject: Re: [Gluster-devel] NSR: Suggestions for a new name

Well, we got quite a few suggestions. So I went ahead and created a 
doodle poll. Please find the link below for the poll, and vote for the 
name you think will be the best.

http://doodle.com/poll/h7gfdhswrbsxxiaa

Regards,
Avra

On 01/21/2016 12:21 PM, Avra Sengupta wrote:
> On 01/21/2016 12:20 PM, Atin Mukherjee wrote:
>> Etherpad link please?
> Oops My Bad. Here it is 
> https://public.pad.fsfe.org/p/NSR_name_suggestions
>>
>> On 01/21/2016 12:19 PM, Avra Sengupta wrote:
>>> Thanks for the suggestion Pranith. To make things interesting, we have
>>> created an etherpad where people can put their suggestions. Somewhere
>>> around mid of feb, we will look at all the suggestions we have got, 
>>> have
>>> a community vote and zero in on one. The suggester of the winning name
>>> gets a goody.
>>>
>>> Feel free to add more than one entry.
>>>
>>> Regards,
>>> Avra
>>>
>>> On 01/21/2016 10:08 AM, Pranith Kumar Karampuri wrote:

 On 01/19/2016 08:00 PM, Avra Sengupta wrote:
> Hi,
>
> The leader election based replication has been called NSR or "New
> Style Replication" for a while now. We would like to have a new name
> for the same that's less generic. It can be something like "Leader
> Driven Replication" or something more specific that would make sense
> a few years down the line too.
>
> We would love to hear more suggestions from the community. Thanks
 If I had a chance to name AFR (Automatic File Replication) I would
 have named it Automatic Data replication. Feel free to use it if you
 like it.

 Pranith
> Regards,
> Avra
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NSR: Suggestions for a new name

2016-02-12 Thread Avra Sengupta

Well, we got quite a few suggestions. So I went ahead and created a 
doodle poll. Please find the link below for the poll, and vote for the 
name you think will be the best.


http://doodle.com/poll/h7gfdhswrbsxxiaa

Regards,
Avra

On 01/21/2016 12:21 PM, Avra Sengupta wrote:

On 01/21/2016 12:20 PM, Atin Mukherjee wrote:

Etherpad link please?
Oops My Bad. Here it is 
https://public.pad.fsfe.org/p/NSR_name_suggestions


On 01/21/2016 12:19 PM, Avra Sengupta wrote:

Thanks for the suggestion Pranith. To make things interesting, we have
created an etherpad where people can put their suggestions. Somewhere
around mid of feb, we will look at all the suggestions we have got, 
have

a community vote and zero in on one. The suggester of the winning name
gets a goody.

Feel free to add more than one entry.

Regards,
Avra

On 01/21/2016 10:08 AM, Pranith Kumar Karampuri wrote:


On 01/19/2016 08:00 PM, Avra Sengupta wrote:

Hi,

The leader election based replication has been called NSR or "New
Style Replication" for a while now. We would like to have a new name
for the same that's less generic. It can be something like "Leader
Driven Replication" or something more specific that would make sense
a few years down the line too.

We would love to hear more suggestions from the community. Thanks

If I had a chance to name AFR (Automatic File Replication) I would
have named it Automatic Data replication. Feel free to use it if you
like it.

Pranith

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] libgfapi libvirt memory leak version 3.7.8

2016-02-12 Thread Piotr Rybicki


Hi

W dniu 2016-02-12 o 07:04, Soumya Koduri pisze:

Hi Piotr,

Could you apply below gfAPI patch and check the valgrind output -

http://review.gluster.org/13125


tried both patches, on client and my 2 bricks. Even recompiled qemu. No 
change - stil leaks (Although few bytes less).


running valgrind on:

qemu-img info gluster://SERVER_IP:0/pool/FILE.img

==4549== LEAK SUMMARY:
==4549==definitely lost: 19,441 bytes in 96 blocks
==4549==indirectly lost: 2,478,511 bytes in 177 blocks
==4549==  possibly lost: 240,600 bytes in 7 blocks
==4549==still reachable: 3,271,130 bytes in 2,931 blocks
==4549== suppressed: 0 bytes in 0 blocks

valgrind full log:
http://195.191.233.1/qemu-img.log
http://195.191.233.1/qemu-img.log.bz2

Best regards
Piotr Rybicki



On 02/11/2016 09:40 PM, Piotr Rybicki wrote:

Hi All

I have to report, that there is a mem leak latest version of gluster

gluster: 3.7.8
libvirt 1.3.1

mem leak exists when starting domain (virsh start DOMAIN) which acesses
drivie via libgfapi (although leak is much smaller than with gluster
3.5.X).

I believe libvirt itself uses libgfapi only to check existence of a disk
(via libgfapi). Libvirt calls glfs_ini and glfs_fini when doing this
check.

When using drive via file (gluster fuse mount), there is no mem leak
when starting domain.

my drive definition (libgfapi):

 
   
   
  # connection is still
via tcp. Defining 'tcp' here doesn't make any difference.
   
   
   
   
 

I've at first reported to libvirt developers, but they blame gluster.

valgrind details (libgfapi):

# valgrind --leak-check=full --show-reachable=yes
--child-silent-after-fork=yes libvirtd --listen 2> libvirt-gfapi.log

On the other console:
virsh start DOMAIN
...wait...
virsh shutdown DOMAIN
...wait and stop valgrind/libvirtd

valgrind log:

==5767== LEAK SUMMARY:
==5767==definitely lost: 19,666 bytes in 96 blocks
==5767==indirectly lost: 21,194 bytes in 123 blocks
==5767==  possibly lost: 2,699,140 bytes in 68 blocks
==5767==still reachable: 986,951 bytes in 15,038 blocks
==5767== suppressed: 0 bytes in 0 blocks
==5767==
==5767== For counts of detected and suppressed errors, rerun with: -v
==5767== ERROR SUMMARY: 96 errors from 96 contexts (suppressed: 0 from 0)

full log:
http://195.191.233.1/libvirt-gfapi.log
http://195.191.233.1/libvirt-gfapi.log.bz2

Best regards
Piotr Rybicki

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NSR: Suggestions for a new name

2016-02-12 Thread Niels de Vos

On Fri, Feb 12, 2016 at 02:05:41PM +0530, Avra Sengupta wrote:
> Well, we got quite a few suggestions. So I went ahead and created a doodle
> poll. Please find the link below for the poll, and vote for the name you
> think will be the best.
> 
> http://doodle.com/poll/h7gfdhswrbsxxiaa

Thanks!
I would have loved an option to select multiple choises. Fedora uses
Range Voting [1] for their polls, but that is probably not something
Doodle can do.

Niels

1. https://en.wikipedia.org/wiki/Range_voting

> 
> Regards,
> Avra
> 
> On 01/21/2016 12:21 PM, Avra Sengupta wrote:
> >On 01/21/2016 12:20 PM, Atin Mukherjee wrote:
> >>Etherpad link please?
> >Oops My Bad. Here it is https://public.pad.fsfe.org/p/NSR_name_suggestions
> >>
> >>On 01/21/2016 12:19 PM, Avra Sengupta wrote:
> >>>Thanks for the suggestion Pranith. To make things interesting, we have
> >>>created an etherpad where people can put their suggestions. Somewhere
> >>>around mid of feb, we will look at all the suggestions we have got,
> >>>have
> >>>a community vote and zero in on one. The suggester of the winning name
> >>>gets a goody.
> >>>
> >>>Feel free to add more than one entry.
> >>>
> >>>Regards,
> >>>Avra
> >>>
> >>>On 01/21/2016 10:08 AM, Pranith Kumar Karampuri wrote:
> 
> On 01/19/2016 08:00 PM, Avra Sengupta wrote:
> >Hi,
> >
> >The leader election based replication has been called NSR or "New
> >Style Replication" for a while now. We would like to have a new name
> >for the same that's less generic. It can be something like "Leader
> >Driven Replication" or something more specific that would make sense
> >a few years down the line too.
> >
> >We would love to hear more suggestions from the community. Thanks
> If I had a chance to name AFR (Automatic File Replication) I would
> have named it Automatic Data replication. Feel free to use it if you
> like it.
> 
> Pranith
> >Regards,
> >Avra
> >___
> >Gluster-devel mailing list
> >Gluster-devel@gluster.org
> >http://www.gluster.org/mailman/listinfo/gluster-devel
> >>>___
> >>>Gluster-devel mailing list
> >>>Gluster-devel@gluster.org
> >>>http://www.gluster.org/mailman/listinfo/gluster-devel
> >
> >___
> >Gluster-devel mailing list
> >Gluster-devel@gluster.org
> >http://www.gluster.org/mailman/listinfo/gluster-devel
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Quota list not reflecting disk usage

2016-02-12 Thread Steve Dainard

What would happen if I:
- Did not disable quotas
- Did not stop the volume (140T volume takes at least 3-4 days to do
any find operations, which is too much downtime)
- Find and remove all xattrs:
trusted.glusterfs.quota.242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3.contri on
the /brick/volumename/modules
- set the dirty bit on /brick/volumename/modules

As far as an upgrade to 3.7, I'm not comfortable with running the
newest release - which version is RHGS based on? I typically like to
follow supported product version if I can, so I know most of the kinks
are worked out :)

On Wed, Feb 10, 2016 at 11:02 PM, Manikandan Selvaganesh
 wrote:
> Hi Steve,
>
> We suspect the mismatching in accounting is probably because of the
> xattr's being not cleaned up properly. Please ensure you do the following
> steps and make sure the xattr's are cleaned up properly before quota
> is enabled for the next time.
>
> 1) stop the volume
> 2) on each brick in the backend do
>Find and remove all the xattrs and make sure they are not present
>  # find /module | xargs getfattr -d -m . -e hex | grep quota | 
> grep -E 'contri|size'
>  # setxattr -x xattrname 
>
> 3) set dirty on /
>  # setxattr -n trusted.glusterfs.quota.dirty -v 0x3100
>By setting dirty value on root as 1(0x3100), the contri will be calculated 
> again
>and the proper contri will be crawled and updated again.
>
> 4) Start volume and from a fuse mount
># stat /mountpath
>
> If you have ever performed a rename, then there is a possibility of two 
> contributions
> getting created for a single entry.
>
> We have fixed quite some rename issues and have refactored the marker 
> approach. Also
> as I have mentioned already we have also done Versioning of xattr's which 
> solves the
> issue you are facing in 3.7. It would be really helpful in a production 
> environment if
> you could upgrade to 3.7
>
> --
> Thanks & Regards,
> Manikandan Selvaganesh.
>
> - Original Message -
> From: "Steve Dainard" 
> To: "Manikandan Selvaganesh" 
> Cc: "Vijaikumar Mallikarjuna" , "Gluster Devel" 
> , "gluster-us...@gluster.org List" 
> 
> Sent: Thursday, February 11, 2016 1:48:19 AM
> Subject: Re: [Gluster-users] Quota list not reflecting disk usage
>
> So after waiting out the process of disabling quotas, waiting for the
> xattrs to be cleaned up, re-enabling quotas and waiting for the
> xattr's to be created, then applying quotas I'm running into the same
> issue.
>
> Yesterday at ~2pm one of the quotas was listed as:
> /modules|100.0GB|18.3GB|81.7GB
>
> I initiated a copy from that glusterfs fuse mount to another fuse
> mount for a different volume, and now I'm seeing:
> /modules|100.0GB|27.4GB|72.6GB
>
> So an increase of 9GB usage.
>
> There were no writes at all to this directory during or after the cp.
>
> I did a bit of digging through the /modules directory on one of the
> gluster nodes and created this spreadsheet:
> https://docs.google.com/spreadsheets/d/1l_6ze68TCOcx6LEh9MFwmqPZ9bM-70CUlSM_8tpQ654/edit?usp=sharing
>
> The /modules/R/3.2.2 directory quota value doesn't come close to
> matching the du value.
>
> Funny bit, there are TWO quota contribution attributes:
> # getfattr -d -m quota -e hex 3.2.2
> # file: 3.2.2
> trusted.glusterfs.quota.242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3.contri=0x09af6000
> trusted.glusterfs.quota.c890be20-1bb9-4aec-a8d0-eacab0446f16.contri=0x13fda800
> trusted.glusterfs.quota.dirty=0x3000
> trusted.glusterfs.quota.size=0x13fda800
>
> For reference, another directory /modules/R/2.14.2 has only one
> contribution attribute:
> # getfattr -d -m quota -e hex 2.14.2
> # file: 2.14.2
> trusted.glusterfs.quota.c890be20-1bb9-4aec-a8d0-eacab0446f16.contri=0x00692800
> trusted.glusterfs.quota.dirty=0x3000
> trusted.glusterfs.quota.size=0x00692800
>
> Questions:
> 1. Why wasn't the
> trusted.glusterfs.quota.242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3.contri=0x09af6000
> cleaned up?
> 2A. How can I remove old attributes from the fs, and then force a
> re-calculation of contributions for the quota path /modules once I've
> done this on all gluster nodes?
> 2B. Or am I stuck yet again removing quotas completely, waiting for
> the automated setfattr to remove the quotas for
> c890be20-1bb9-4aec-a8d0-eacab0446f16 ID, manually removing attrs for
> 242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3, re-enabling quotas, waiting for
> xattrs to be generated, then enabling limits?
> 3. Shouldn't there be a command to re-trigger quota accounting on a
> directory that confirms the attrs are set correctly and checks that
> the contribution attr actually match disk usage?
>
> On Tue, Feb 2, 2016 at 3:00 AM, Manikandan Selvaganesh
>  wrote:
>> Hi Steve,
>>
>> As you have mentioned, if you are using a glusterfs version lesser than 3.7,
>> then you are doing it

Re: [Gluster-devel] [Gluster-users] Quota list not reflecting disk usage

2016-02-12 Thread Steve Dainard

So after waiting out the process of disabling quotas, waiting for the
xattrs to be cleaned up, re-enabling quotas and waiting for the
xattr's to be created, then applying quotas I'm running into the same
issue.

Yesterday at ~2pm one of the quotas was listed as:
/modules|100.0GB|18.3GB|81.7GB

I initiated a copy from that glusterfs fuse mount to another fuse
mount for a different volume, and now I'm seeing:
/modules|100.0GB|27.4GB|72.6GB

So an increase of 9GB usage.

There were no writes at all to this directory during or after the cp.

I did a bit of digging through the /modules directory on one of the
gluster nodes and created this spreadsheet:
https://docs.google.com/spreadsheets/d/1l_6ze68TCOcx6LEh9MFwmqPZ9bM-70CUlSM_8tpQ654/edit?usp=sharing

The /modules/R/3.2.2 directory quota value doesn't come close to
matching the du value.

Funny bit, there are TWO quota contribution attributes:
# getfattr -d -m quota -e hex 3.2.2
# file: 3.2.2
trusted.glusterfs.quota.242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3.contri=0x09af6000
trusted.glusterfs.quota.c890be20-1bb9-4aec-a8d0-eacab0446f16.contri=0x13fda800
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.size=0x13fda800

For reference, another directory /modules/R/2.14.2 has only one
contribution attribute:
# getfattr -d -m quota -e hex 2.14.2
# file: 2.14.2
trusted.glusterfs.quota.c890be20-1bb9-4aec-a8d0-eacab0446f16.contri=0x00692800
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.size=0x00692800

Questions:
1. Why wasn't the
trusted.glusterfs.quota.242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3.contri=0x09af6000
cleaned up?
2A. How can I remove old attributes from the fs, and then force a
re-calculation of contributions for the quota path /modules once I've
done this on all gluster nodes?
2B. Or am I stuck yet again removing quotas completely, waiting for
the automated setfattr to remove the quotas for
c890be20-1bb9-4aec-a8d0-eacab0446f16 ID, manually removing attrs for
242dcfd9-6aea-4cb8-beb2-c0ed91ad70d3, re-enabling quotas, waiting for
xattrs to be generated, then enabling limits?
3. Shouldn't there be a command to re-trigger quota accounting on a
directory that confirms the attrs are set correctly and checks that
the contribution attr actually match disk usage?

On Tue, Feb 2, 2016 at 3:00 AM, Manikandan Selvaganesh
 wrote:
> Hi Steve,
>
> As you have mentioned, if you are using a glusterfs version lesser than 3.7,
> then you are doing it right. We are sorry to say but unfortunately that's the 
> only
> way(manually going and cleaning up the xattr's before enabling quota or wait 
> for
> the process to complete itself, which would take quite some time depending 
> upon the
> files) that can be done so as not to mess up quota enforcing/accounting. 
> Also, we could
> not find anything that could help us with the logs too. Thanks for the
> point. We are in the process of writing blogs and documenting clearly about 
> quota and
> it's internal working. There is an initial blog[1] which we have written. 
> More blogs will
> follow.
>
> With glusterfs-3.7, we have introduced something called "Quota versioning".
> So whenever you enable quota, we are suffixing a number(1..N) with the quota 
> xattr's,
> say you enable quota for the first time and the xattr will be like,
> "trusted.glusterfs.quota.size.". So all the quota 
> related xattr's
> will have the number suffixed to the xattr. With the versioning patch[2], 
> when you disable and
> enable quota again for the next time, it will be 
> "trusted.glusterfs.quota.size.2"(Similarly
> for other quota related xattr's). So quota accounting can happen 
> independently depending on
> the suffix and the cleanup process can go on independently which solves the 
> issue that you
> have.
>
> [1] https://manikandanselvaganesh.wordpress.com/
>
> [2] http://review.gluster.org/12386
>
> --
> Thanks & Regards,
> Manikandan Selvaganesh.
>
> - Original Message -
> From: "Vijaikumar Mallikarjuna" 
> To: "Steve Dainard" 
> Cc: "Manikandan Selvaganesh" 
> Sent: Tuesday, February 2, 2016 10:12:51 AM
> Subject: Re: [Gluster-users] Quota list not reflecting disk usage
>
> Hi Steve,
>
> Sorry for the delay. Mani and myself was busy with something else at work,
> we will update you on this by eod.
>
> Many quota issues has been fixed in 3.7, also version numbers are added to
> quota xattrs, so when quota is disabled we don't need to cleanup the xattrs.
>
> Thanks,
> Vijay
>
>
>
>
>
> On Tue, Feb 2, 2016 at 12:26 AM, Steve Dainard  wrote:
>
>> I haven't heard anything back on this thread so here's where I've landed:
>>
>> It appears that the quota xattr's are not being cleared when quota's
>> are disabled, so when they are disabled and re-enabled the value for
>> size is added to the previous size, making it appear that the 'Used'
>> space is significantly greater than it

Re: [Gluster-devel] libgfapi libvirt memory leak version 3.7.8

2016-02-12 Thread Piotr Rybicki




W dniu 2016-02-11 o 16:02, Piotr Rybicki pisze:

Hi All

I have to report, that there is a mem leak latest version of gluster

gluster: 3.7.8
libvirt 1.3.1

mem leak exists when starting domain (virsh start DOMAIN) which acesses
drivie via libgfapi (although leak is much smaller than with gluster
3.5.X).

I believe libvirt itself uses libgfapi only to check existence of a disk
(via libgfapi). Libvirt calls glfs_ini and glfs_fini when doing this check.

When using drive via file (gluster fuse mount), there is no mem leak
when starting domain.

my drive definition (libgfapi):

 
   
   
  # connection is still
via tcp. Defining 'tcp' here doesn't make any difference.
   
   
   
   
 

I've at first reported to libvirt developers, but they blame gluster.

valgrind details (libgfapi):

# valgrind --leak-check=full --show-reachable=yes
--child-silent-after-fork=yes libvirtd --listen 2> libvirt-gfapi.log

On the other console:
virsh start DOMAIN
...wait...
virsh shutdown DOMAIN
...wait and stop valgrind/libvirtd

valgrind log:

==5767== LEAK SUMMARY:
==5767==definitely lost: 19,666 bytes in 96 blocks
==5767==indirectly lost: 21,194 bytes in 123 blocks
==5767==  possibly lost: 2,699,140 bytes in 68 blocks
==5767==still reachable: 986,951 bytes in 15,038 blocks
==5767== suppressed: 0 bytes in 0 blocks
==5767==
==5767== For counts of detected and suppressed errors, rerun with: -v
==5767== ERROR SUMMARY: 96 errors from 96 contexts (suppressed: 0 from 0)

full log:
http://195.191.233.1/libvirt-gfapi.log
http://195.191.233.1/libvirt-gfapi.log.bz2

Best regards
Piotr Rybicki


Even simpler case to see memleak is to valgrind on:

qemu-img info gluster://SERVER_IP:0/pool/FILE.img

==6100== LEAK SUMMARY:
==6100==definitely lost: 19,846 bytes in 98 blocks
==6100==indirectly lost: 2,479,205 bytes in 182 blocks
==6100==  possibly lost: 240,600 bytes in 7 blocks
==6100==still reachable: 3,271,130 bytes in 2,931 blocks
==6100== suppressed: 0 bytes in 0 blocks

Best regards
Piotr Rybicki
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] GlusterFS-3.7.6-2 packages for Debian Wheezy now available

2016-02-12 Thread Ronny Adsetts

Kaleb Keithley wrote on 04/02/2016 06:40:
> 
> If you're a Debian Wheezy user please give the new packages a try.

Hi Kaleb,

Apologies for the delay in getting back to you. I tried the upgrade on one node 
last week and it failed but I hadn't had the time to try it again without the 
feeling of panic around my neck :-).

So, I did the upgrade again on one node only but the node does not restart 
without error.

Excerpt of etc-glusterfs-glusterd.vol.log follows. Other than the errors, the 
thing that sticks out to me is in the management volume definition where it 
says "option transport-type rdma" as we're not using rdma. This may of course 
be a red herring.

I've now uninstalled the 3.7.6 packages and reinstalled 3.6.8.

If you need any further information please do let me know. I can try the 
upgrade again if there are changes you'd like me to make.

Thanks for your work on this so far.

Ronny


[2016-02-10 09:24:42.479807] W [glusterfsd.c:1211:cleanup_and_exit] (--> 0-: 
received signum (15), shutting down
[2016-02-10 09:27:14.031377] I [MSGID: 100030] [glusterfsd.c:2318:main] 
0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.6 (args: 
/usr/sbin/glusterd -p /var/run/glusterd.pid)
[2016-02-10 09:27:14.035512] I [MSGID: 106478] [glusterd.c:1350:init] 
0-management: Maximum allowed open file descriptors set to 65536
[2016-02-10 09:27:14.035554] I [MSGID: 106479] [glusterd.c:1399:init] 
0-management: Using /var/lib/glusterd as working directory
[2016-02-10 09:27:14.040817] W [MSGID: 103071] 
[rdma.c:4592:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event channel 
creation failed [No such device]
[2016-02-10 09:27:14.040848] W [MSGID: 103055] [rdma.c:4899:init] 
0-rdma.management: Failed to initialize IB Device
[2016-02-10 09:27:14.040860] W [rpc-transport.c:359:rpc_transport_load] 
0-rpc-transport: 'rdma' initialization failed
[2016-02-10 09:27:14.040921] W [rpcsvc.c:1597:rpcsvc_transport_create] 
0-rpc-service: cannot create listener, initing the transport failed
[2016-02-10 09:27:14.040937] E [MSGID: 106243] [glusterd.c:1623:init] 
0-management: creation of 1 listeners failed, continuing with succeeded 
transport
[2016-02-10 09:27:15.725220] I [MSGID: 106513] 
[glusterd-store.c:2047:glusterd_restore_op_version] 0-glusterd: retrieved 
op-version: 1
[2016-02-10 09:27:15.900057] I [MSGID: 106498] 
[glusterd-handler.c:3579:glusterd_friend_add_from_peerinfo] 0-management: 
connect returned 0
[2016-02-10 09:27:15.900138] I [rpc-clnt.c:984:rpc_clnt_connection_init] 
0-management: setting frame-timeout to 600
[2016-02-10 09:27:15.900735] W [socket.c:869:__socket_keepalive] 0-socket: 
failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
[2016-02-10 09:27:15.900749] E [socket.c:2965:socket_connect] 0-management: 
Failed to set keep-alive: Invalid argument
[2016-02-10 09:27:15.900922] I [MSGID: 106194] 
[glusterd-store.c:3487:glusterd_store_retrieve_missed_snaps_list] 0-management: 
No missed snaps list.
[2016-02-10 09:27:15.901746] I [MSGID: 106544] 
[glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID: 
79083345-b45a-466b-97f3-612ebfac7fe9
Final graph:
+--+
  1: volume management
  2: type mgmt/glusterd
  3: option rpc-auth.auth-glusterfs on
  4: option rpc-auth.auth-unix on
  5: option rpc-auth.auth-null on
  6: option rpc-auth-allow-insecure on
  7: option transport.socket.listen-backlog 128
  8: option ping-timeout 30
  9: option transport.socket.read-fail-log off
 10: option transport.socket.keepalive-interval 2
 11: option transport.socket.keepalive-time 10
 12: option transport-type rdma
 13: option working-directory /var/lib/glusterd
 14: end-volume
 15:
+--+
[2016-02-10 09:27:15.903840] I [MSGID: 101190] 
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with 
index 2
[2016-02-10 09:27:15.903897] I [MSGID: 101190] 
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with 
index 2
[2016-02-10 09:27:15.903945] I [MSGID: 101190] 
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with 
index 1
[2016-02-10 09:27:15.906366] W [socket.c:588:__socket_rwv] 0-management: readv 
on 172.18.40.17:24007 failed (Connection reset by peer)
[2016-02-10 09:27:15.906734] E [rpc-clnt.c:362:saved_frames_unwind] (--> 
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x1e7)[0x7f157cb46a57]
 (--> 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1be)[0x7f157c90d1be]
 (--> 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f157c90d2ce]
 (--> 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x88)[0x7f157c90ec58]
 (--> 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x1d0)[0x7f157c90f2a0] 
) 0-management: forced unwinding frame

Re: [Gluster-devel] [Gluster-users] GlusterFS-3.7.6-2 packages for Debian Wheezy now available

2016-02-12 Thread Ronny Adsetts

Kaushal M wrote on 10/02/2016 11:44:
[...]
>>>
>>> Excerpt of etc-glusterfs-glusterd.vol.log follows. Other than the errors, 
>>> the
>>> thing that sticks out to me is in the management volume definition where it
>>> says "option transport-type rdma" as we're not using rdma. This may of
>>> course be a red herring.
> 
> This is odd. Can you check the file `/etc/glusterfs/glusterd.vol` and
> verify it it has `transport-type rdma` as well? This file is supposed
> to be shipped by the `glusterfs-server` package and should the option
> should default to `tcp` or atleast `tcp,rdma`.

gotham:~# cat /etc/glusterfs/glusterd.vol
volume management
type mgmt/glusterd
option working-directory /var/lib/glusterd
option transport-type socket,rdma
option transport.socket.keepalive-time 10
option transport.socket.keepalive-interval 2
option transport.socket.read-fail-log off
option ping-timeout 30
#   option base-port 49152
end-volume

I don't recall ever editing this file *but* there was an ancient install of 
gluster on this server where I'd been playing around with things...

Comparing to the two other installs of gluster I have (on debian squeeze, 
gluster 3.2.7 for the moment...) with untouched files where we have this:

radsetts@monolith:~$ cat /etc/glusterfs/glusterd.vol
volume management
type mgmt/glusterd
option working-directory /etc/glusterd
option transport-type socket,rdma
option transport.socket.keepalive-time 10
option transport.socket.keepalive-interval 2
end-volume

Thanks.

Ronny


>>> [2016-02-10 09:24:42.479807] W [glusterfsd.c:1211:cleanup_and_exit] (--> 0-:
>>> received signum (15), shutting down
>>> [2016-02-10 09:27:14.031377] I [MSGID: 100030] [glusterfsd.c:2318:main]
>>> 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.6
>>> (args: /usr/sbin/glusterd -p /var/run/glusterd.pid)
>>> [2016-02-10 09:27:14.035512] I [MSGID: 106478] [glusterd.c:1350:init]
>>> 0-management: Maximum allowed open file descriptors set to 65536
>>> [2016-02-10 09:27:14.035554] I [MSGID: 106479] [glusterd.c:1399:init]
>>> 0-management: Using /var/lib/glusterd as working directory
>>> [2016-02-10 09:27:14.040817] W [MSGID: 103071]
>>> [rdma.c:4592:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event
>>> channel creation failed [No such device]
>>> [2016-02-10 09:27:14.040848] W [MSGID: 103055] [rdma.c:4899:init]
>>> 0-rdma.management: Failed to initialize IB Device
>>> [2016-02-10 09:27:14.040860] W [rpc-transport.c:359:rpc_transport_load]
>>> 0-rpc-transport: 'rdma' initialization failed
>>> [2016-02-10 09:27:14.040921] W [rpcsvc.c:1597:rpcsvc_transport_create]
>>> 0-rpc-service: cannot create listener, initing the transport failed
>>> [2016-02-10 09:27:14.040937] E [MSGID: 106243] [glusterd.c:1623:init]
>>> 0-management: creation of 1 listeners failed, continuing with succeeded
>>> transport
>>> [2016-02-10 09:27:15.725220] I [MSGID: 106513]
>>> [glusterd-store.c:2047:glusterd_restore_op_version] 0-glusterd: retrieved
>>> op-version: 1
>>> [2016-02-10 09:27:15.900057] I [MSGID: 106498]
>>> [glusterd-handler.c:3579:glusterd_friend_add_from_peerinfo] 0-management:
>>> connect returned 0
>>> [2016-02-10 09:27:15.900138] I [rpc-clnt.c:984:rpc_clnt_connection_init]
>>> 0-management: setting frame-timeout to 600
>>> [2016-02-10 09:27:15.900735] W [socket.c:869:__socket_keepalive] 0-socket:
>>> failed to set TCP_USER_TIMEOUT -1000 on socket 13, Invalid argument
>>> [2016-02-10 09:27:15.900749] E [socket.c:2965:socket_connect] 0-management:
>>> Failed to set keep-alive: Invalid argument
>>> [2016-02-10 09:27:15.900922] I [MSGID: 106194]
>>> [glusterd-store.c:3487:glusterd_store_retrieve_missed_snaps_list]
>>> 0-management: No missed snaps list.
>>> [2016-02-10 09:27:15.901746] I [MSGID: 106544]
>>> [glusterd.c:159:glusterd_uuid_init] 0-management: retrieved UUID:
>>> 79083345-b45a-466b-97f3-612ebfac7fe9
>>> Final graph:
>>> +--+
>>>   1: volume management
>>>   2: type mgmt/glusterd
>>>   3: option rpc-auth.auth-glusterfs on
>>>   4: option rpc-auth.auth-unix on
>>>   5: option rpc-auth.auth-null on
>>>   6: option rpc-auth-allow-insecure on
>>>   7: option transport.socket.listen-backlog 128
>>>   8: option ping-timeout 30
>>>   9: option transport.socket.read-fail-log off
>>>  10: option transport.socket.keepalive-interval 2
>>>  11: option transport.socket.keepalive-time 10
>>>  12: option transport-type rdma
>>>  13: option working-directory /var/lib/glusterd
>>>  14: end-volume
>>>  15:
>>> +--+
>>> [2016-02-10 09:27:15.903840] I [MSGID: 101190]
>>> [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with
>>> index 2
>>> [2016-02-10 09:27:15.903897] I [MSGID: 101190]
>>> [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started

Re: [Gluster-devel] [Gluster-users] Gluster samba panic?

2016-02-12 Thread Diego Remolina

I actually had this problem with CentOS 7 and glusterfs 3.7.x

I downgraded to 3.6.x and the crashes stopped.

See https://bugzilla.redhat.com/show_bug.cgi?id=1234877

It may be the same issue.

I am still in the old samba-vfs-glusterfs-4.1.12-23.el7_1.x86_64 and
glusterfs-3.6.6-1.el7.x86_64 on that machine and see no crashes.

Diego

On Wed, Feb 10, 2016 at 2:19 PM, Glomski, Patrick
 wrote:
> I am running an SL7.2 machine with the samba version 4.2.4 hosted at
> download.gluster.org. I use the samba glusterfs virtual filesystem to
> connect to a 3.6.6 gluster array. Periodically (every hour or so when in
> use) smbd will panic, crash, and restart. Users don't seem to be complaining
> about it, but the fact that it keeps crashing is worrisome. Does anyone know
> if this issue has a bug or is fixed in a newer version of the glusterfs vfs
> plugin? I've attached an excerpt from /var/log/messages and a core dump.
>
> # cat /etc/redhat-release; echo "samba: $(smbd --version)"; echo "vfs: $(rpm
> -qa | grep vfs-glusterfs)"
>
> Scientific Linux release 7.2 (Nitrogen)
> samba: Version 4.2.4
> vfs: samba-vfs-glusterfs-4.2.4-6.el7.centos.x86_64
>
> /var/log/messages panic
> ###
> Feb 10 12:38:11 sb1 smbd[4880]: [2016/02/10 12:38:11.329296,  0]
> ../lib/util/fault.c:78(fault_report)
> Feb 10 12:38:11 sb1 smbd[4880]:
> ===
> Feb 10 12:38:11 sb1 smbd[4880]: [2016/02/10 12:38:11.329342,  0]
> ../lib/util/fault.c:79(fault_report)
> Feb 10 12:38:11 sb1 smbd[4880]:  INTERNAL ERROR: Signal 6 in pid 4880
> (4.2.4)
> Feb 10 12:38:11 sb1 smbd[4880]:  Please read the Trouble-Shooting section of
> the Samba HOWTO
> Feb 10 12:38:11 sb1 smbd[4880]: [2016/02/10 12:38:11.329365,  0]
> ../lib/util/fault.c:81(fault_report)
> Feb 10 12:38:11 sb1 smbd[4880]:
> ===
> Feb 10 12:38:11 sb1 smbd[4880]: [2016/02/10 12:38:11.329390,  0]
> ../source3/lib/util.c:788(smb_panic_s3)
> Feb 10 12:38:11 sb1 smbd[4880]:  PANIC (pid 4880): internal error
> Feb 10 12:38:11 sb1 smbd[4880]: [2016/02/10 12:38:11.329633,  0]
> ../source3/lib/util.c:899(log_stack_trace)
> Feb 10 12:38:11 sb1 smbd[4880]:  BACKTRACE: 12 stack frames:
> Feb 10 12:38:11 sb1 smbd[4880]:   #0
> /lib64/libsmbconf.so.0(log_stack_trace+0x1a) [0x7f24d03ced5a]
> Feb 10 12:38:11 sb1 smbd[4880]:   #1
> /lib64/libsmbconf.so.0(smb_panic_s3+0x20) [0x7f24d03cee30]
> Feb 10 12:38:11 sb1 smbd[4880]:   #2
> /lib64/libsamba-util.so.0(smb_panic+0x2f) [0x7f24d221e87f]
> Feb 10 12:38:11 sb1 smbd[4880]:   #3 /lib64/libsamba-util.so.0(+0x1aa96)
> [0x7f24d221ea96]
> Feb 10 12:38:11 sb1 smbd[4880]:   #4 /lib64/libpthread.so.0(+0xf100)
> [0x7f24d2447100]
> Feb 10 12:38:11 sb1 smbd[4880]:   #5 /lib64/libc.so.6(gsignal+0x37)
> [0x7f24cea7d5f7]
> Feb 10 12:38:11 sb1 smbd[4880]:   #6 /lib64/libc.so.6(abort+0x148)
> [0x7f24cea7ece8]
> Feb 10 12:38:11 sb1 smbd[4880]:   #7 /lib64/libc.so.6(+0x75317)
> [0x7f24ceabd317]
> Feb 10 12:38:11 sb1 smbd[4880]:   #8 /lib64/libc.so.6(+0x7cfe1)
> [0x7f24ceac4fe1]
> Feb 10 12:38:11 sb1 smbd[4880]:   #9
> /lib64/libglusterfs.so.0(gf_timer_proc+0x251) [0x7f24b75cc961]
> Feb 10 12:38:11 sb1 smbd[4880]:   #10 /lib64/libpthread.so.0(+0x7dc5)
> [0x7f24d243fdc5]
> Feb 10 12:38:11 sb1 smbd[4880]:   #11 /lib64/libc.so.6(clone+0x6d)
> [0x7f24ceb3e21d]
> Feb 10 12:38:11 sb1 smbd[4880]: [2016/02/10 12:38:11.330887,  0]
> ../source3/lib/dumpcore.c:318(dump_core)
> Feb 10 12:38:11 sb1 smbd[4880]:  dumping core in /var/log/samba/cores/smbd
> Feb 10 12:38:11 sb1 smbd[4880]:
> Feb 10 12:38:11 sb1 abrt-server: Package 'samba' isn't signed with proper
> key
> Feb 10 12:38:11 sb1 abrt-server: 'post-create' on
> '/var/spool/abrt/ccpp-2016-02-10-12:38:11-4880' exited with 1
> Feb 10 12:38:11 sb1 abrt-server: Deleting problem directory
> '/var/spool/abrt/ccpp-2016-02-10-12:38:11-4880'
>
> Thanks,
> Patrick
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] GlusterFS-3.7.6-2 packages for Debian Wheezy now available

2016-02-12 Thread Ronny Adsetts

Kaleb Keithley wrote on 10/02/2016 11:28:
> 
> Please attach the logs to https://bugzilla.redhat.com/show_bug.cgi?id=1304348
> (or mail them to Kaushal and/or me.

Log files have been attached to the ticket.

Ronny
-- 
Ronny Adsetts
Technical Director
Amazing Internet Ltd, London
t: +44 20 8977 8943
w: www.amazinginternet.com

Registered office: 85 Waldegrave Park, Twickenham, TW1 4TJ
Registered in England. Company No. 4042957




signature.asc
Description: OpenPGP digital signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] changelog bug

2016-02-12 Thread Peponnet, Cyril (Nokia - CA)

Btw,

Issuing:

gluster vol set usr_global diagnostics.brick-log-level CRITICAL

Crash my bricks :/

https://gist.github.com/CyrilPeponnet/11954cbca725d4b8da7a

--
Cyril Peponnet

On Feb 9, 2016, at 8:56 AM, Vijay Bellur  wrote:

On 02/08/2016 01:14 AM, Kotresh Hiremath Ravishankar wrote:
Hi,

This bug is already tracked BZ 1221629
I will start working on this and will update once it is fixed.


Cyril (in CC) also reported a similar crash with changelog in 3.6.5:

https://gist.github.com/CyrilPeponnet/b67b360f186f31d34d8f

The crash seems to be consistently reproducible in Cyril's setup. Can we 
address this soon?

Thanks,
Vijay


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] libgfapi libvirt memory leak version 3.7.8

2016-02-12 Thread Piotr Rybicki


Hi All

I have to report, that there is a mem leak latest version of gluster

gluster: 3.7.8
libvirt 1.3.1

mem leak exists when starting domain (virsh start DOMAIN) which acesses 
drivie via libgfapi (although leak is much smaller than with gluster 3.5.X).


I believe libvirt itself uses libgfapi only to check existence of a disk 
(via libgfapi). Libvirt calls glfs_ini and glfs_fini when doing this check.


When using drive via file (gluster fuse mount), there is no mem leak 
when starting domain.


my drive definition (libgfapi):


  
  
 # connection is still 
via tcp. Defining 'tcp' here doesn't make any difference.

  
  
  
  function='0x0'/>



I've at first reported to libvirt developers, but they blame gluster.

valgrind details (libgfapi):

# valgrind --leak-check=full --show-reachable=yes 
--child-silent-after-fork=yes libvirtd --listen 2> libvirt-gfapi.log


On the other console:
virsh start DOMAIN
...wait...
virsh shutdown DOMAIN
...wait and stop valgrind/libvirtd

valgrind log:

==5767== LEAK SUMMARY:
==5767==definitely lost: 19,666 bytes in 96 blocks
==5767==indirectly lost: 21,194 bytes in 123 blocks
==5767==  possibly lost: 2,699,140 bytes in 68 blocks
==5767==still reachable: 986,951 bytes in 15,038 blocks
==5767== suppressed: 0 bytes in 0 blocks
==5767==
==5767== For counts of detected and suppressed errors, rerun with: -v
==5767== ERROR SUMMARY: 96 errors from 96 contexts (suppressed: 0 from 0)

full log:
http://195.191.233.1/libvirt-gfapi.log
http://195.191.233.1/libvirt-gfapi.log.bz2

Best regards
Piotr Rybicki
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Downtime for Gerrit and Jenkins - 12th February

2016-02-12 Thread Kaushal M

On Thu, Feb 11, 2016 at 5:28 AM, Vijay Bellur  wrote:
> Hi All,
>
> We will be migrating our Gerrit and (possibly) Jenkins services from the
> current hosted infrastructure on Friday, 12th February. To accomplish the
> migration, we will have a downtime for these services starting at 0900 UTC.
> We expect the migration to be complete by 1700 UTC on the same day. If for
> any reason the migration does not complete by then, we will be reverting
> back to the existing infrastructure and attempt migration again after a
> subsequent announcement.
>
> During the migration window submission of patches for review and continuous
> integration will be affected. We expect to have a more robust infrastructure
> for these services once the migration is complete. If you have any questions
> about this migration, please send out a note to us on the gluster-infra
> mailing list.
>
> Thanks,
> Gluster Infrastructure team

An update on what happened.

We migrated Gerrit successfully. The migration happened really quickly
(< 1hr IIRC) and without much problems. https://review.gluster.org is
now hosted in a Red Hat DC and live right now. We've checked and it
appears to be working correctly. If anyone finds any issue, please
contact gluster-infra.

Migration of Jenkins on the other hand was unsuccessful. Copying out
the disk image took longer than expected, and it didn't complete
within the expected time. So for now we've restarted the old instance.
We've verified that it's able to connect to gerrit and is able to
trigger jobs, so it will continue work as before.

We're in the process of coming up with a better migration plan for
Jenkins, on that will hopefully take less downtime. We'll share the
new plan once we finalize on it and the dates.

Thanks,
Gluster Infra team

> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Feature: Automagic lock-revocation for features/locks xlator (v3.7.x)

2016-02-12 Thread Richard Wareing

Hey,

Sorry for the late reply but I missed this e-mail.  With respect to identifying 
locking domains, we use the identical logic that GlusterFS itself uses to 
identify the domains; which is just a simple string comparison if I'm not 
mistaken.   System processes (SHD/Rebalance) locking domains are treated 
identical to any other, this is specifically critical to things like DHT 
healing as this locking domain is used both in userland and by SHDs (you cannot 
disable DHT healing).  To illustrate this, consider the case where a SHD holds 
a lock to do a DHT heal but can't because of GFID split-braina user comes 
along a hammers that directory attempting to get a lockyou can pretty much 
kiss your cluster good-bye after that :).

With this in mind, we explicitly choose not to respect system process 
(SHD/rebalance) locks any more than a user lock request as they can be just as 
likely (if not more so) to cause a system to fall over vs. a user (see example 
above).  Although this might seem unwise at first, I'd put forth that having 
clusters fall over catastrophically pushes far worse decisions on operators 
such as re-kicking random bricks or entire clusters in desperate attempts at 
freeing locks (the CLI is often unable to free the locks in our experience) or 
stopping run away memory consumption due to frames piling up on the bricks.  To 
date, we haven't even observed a single instance of data corruption (and we've 
been looking for it!) due to this feature.

We've even used it on clusters where they were on the verge of falling over and 
we enable revocation and the entire system stabilizes almost instantly (it's 
really like magic when you see it :) ).

Hope this helps!

Richard

From: raghavendra...@gmail.com [raghavendra...@gmail.com] on behalf of 
Raghavendra G [raghaven...@gluster.com]
Sent: Tuesday, January 26, 2016 9:49 PM
To: Raghavendra Gowdappa
Cc: Richard Wareing; Gluster Devel
Subject: Re: [Gluster-devel] Feature: Automagic lock-revocation for 
features/locks xlator (v3.7.x)

On Mon, Jan 25, 2016 at 10:39 AM, Raghavendra Gowdappa 
> wrote:

- Original Message -
> From: "Richard Wareing" >
> To: "Pranith Kumar Karampuri" 
> >
> Cc: gluster-devel@gluster.org
> Sent: Monday, January 25, 2016 8:17:11 AM
> Subject: Re: [Gluster-devel] Feature: Automagic lock-revocation for 
> features/locks xlator (v3.7.x)
>
> Yup per domain would be useful, the patch itself currently honors domains as
> well. So locks in a different domains will not be touched during revocation.
>
> In our cases we actually prefer to pull the plug on SHD/DHT domains to ensure
> clients do not hang, this is important for DHT self heals which cannot be
> disabled via any option, we've found in most cases once we reap the lock
> another properly behaving client comes along and completes the DHT heal
> properly.

Flushing waiting locks of DHT can affect application continuity too. Though 
locks requested by rebalance process can be flushed to certain extent without 
applications noticing any failures, there is no guarantee that locks requested 
in DHT_LAYOUT_HEAL_DOMAIN and DHT_FILE_MIGRATE_DOMAIN, are issued by only 
rebalance process.

I missed this point in my previous mail. Now I remember that we can use 
frame->root->pid (being negative) to identify internal processes. Was this the 
approach you followed to identify locks from rebalance process?

These two domains are used for locks to synchronize among and between rebalance 
process(es) and client(s). So, there is equal probability that these locks 
might be requests from clients and hence application can see some file 
operations failing.

In case of pulling plug on DHT_LAYOUT_HEAL_DOMAIN, dentry operations that 
depend on layout can fail. These operations can include create, link, unlink, 
symlink, mknod, mkdir, rename for files/directory within the directory on which 
lock request is failed.

In case of pulling plug on DHT_FILE_MIGRATE_DOMAIN, rename of immediate 
subdirectories/files can fail.

>
> Richard
>
>
> Sent from my iPhone
>
> On Jan 24, 2016, at 6:42 PM, Pranith Kumar Karampuri < 
> pkara...@redhat.com >
> wrote:
>
>
>
>
>
>
> On 01/25/2016 02:17 AM, Richard Wareing wrote:
>
>
>
> Hello all,
>
> Just gave a talk at SCaLE 14x today and I mentioned our new locks revocation
> feature which has had a significant impact on our GFS cluster reliability.
> As such I wanted to share the patch with the community, so here's the
> bugzilla report:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1301401
>
> =
> Summary:
> Mis-behaving brick clients (gNFSd, FUSE, gfAPI) can cause cluster instability
> and eventual complete unavailability due to failures in releasing
> entry/inode locks in a timely manner.
>
> Classic

Re: [Gluster-devel] Throttling xlator on the bricks

2016-02-12 Thread Richard Wareing

Hey Ravi,

I'll ping Shreyas about this today.  There's also a patch we'll need for 
multi-threaded SHD to fix the least-pri queuing.  The PID of the process wasn't 
tagged correctly via the call frame in my original patch.  The patch below 
fixes this (for 3.6.3), I didn't see multi-threaded self heal on github/master 
yet so let me know what branch you need this patch on and I can come up with a 
clean patch.

Richard


=


diff --git a/xlators/cluster/afr/src/afr-self-heald.c 
b/xlators/cluster/afr/src/afr-self-heald.c
index 028010d..b0f6248 100644
--- a/xlators/cluster/afr/src/afr-self-heald.c
+++ b/xlators/cluster/afr/src/afr-self-heald.c
@@ -532,6 +532,9 @@ afr_mt_process_entries_done (int ret, call_frame_t 
*sync_frame,
 pthread_cond_signal (_data->task_done);
 }
 pthread_mutex_unlock (_data->lock);
+
+if (task_ctx->frame)
+AFR_STACK_DESTROY (task_ctx->frame);
 GF_FREE (task_ctx);
 return 0;
 }
@@ -787,6 +790,7 @@ _afr_mt_create_process_entries_task (xlator_t *this,
 int   ret = -1;
 afr_mt_process_entries_task_ctx_t *task_ctx;
 afr_mt_data_t *mt_data;
+call_frame_t  *frame = NULL;

 mt_data = >mt_data;

@@ -799,6 +803,8 @@ _afr_mt_create_process_entries_task (xlator_t *this,
 if (!task_ctx)
 goto err;

+task_ctx->frame = afr_frame_create (this);
+
 INIT_LIST_HEAD (_ctx->list);
 task_ctx->readdir_xl = this;
 task_ctx->healer = healer;
@@ -812,7 +818,7 @@ _afr_mt_create_process_entries_task (xlator_t *this,
 // This returns immediately, and afr_mt_process_entries_done will
 // be called when the task is completed e.g. our queue is empty
 ret = synctask_new (this->ctx->env, afr_mt_process_entries_task,
-afr_mt_process_entries_done, NULL,
+afr_mt_process_entries_done, task_ctx->frame,
 (void *)task_ctx);

 if (!ret) {
diff --git a/xlators/cluster/afr/src/afr-self-heald.h 
b/xlators/cluster/afr/src/afr-self-heald.h
index 817e712..1588fc8 100644
--- a/xlators/cluster/afr/src/afr-self-heald.h
+++ b/xlators/cluster/afr/src/afr-self-heald.h
@@ -74,6 +74,7 @@ typedef struct afr_mt_process_entries_task_ctx_ {
 subvol_healer_t *healer;
 xlator_t*readdir_xl;
 inode_t *idx_inode;  /* inode ref for xattrop dir */
+call_frame_t*frame;
 unsigned intentries_healed;
 unsigned intentries_processed;
 unsigned intalready_healed;


Richard

From: Ravishankar N [ravishan...@redhat.com]
Sent: Sunday, February 07, 2016 11:15 PM
To: Shreyas Siravara
Cc: Richard Wareing; Vijay Bellur; Gluster Devel
Subject: Re: [Gluster-devel] Throttling xlator on the bricks

Hello,

On 01/29/2016 06:51 AM, Shreyas Siravara wrote:
> So the way our throttling works is (intentionally) very simplistic.
>
> (1) When someone mounts an NFS share, we tag the frame with a 32 bit hash of 
> the export name they were authorized to mount.
> (2) io-stats keeps track of the "current rate" of fops we're seeing for that 
> particular mount, using a sampling of fops and a moving average over a short 
> period of time.
> (3) Based on whether the share violated its allowed rate (which is defined in 
> a config file), we tag the FOP as "least-pri". Of course this makes the 
> assumption that all NFS endpoints are receiving roughly the same # of FOPs. 
> The rate defined in the config file is a *per* NFS endpoint number. So if 
> your cluster has 10 NFS endpoints, and you've pre-computed that it can do 
> roughly 1000 FOPs per second, the rate in the config file would be 100.
> (4) IO-Threads then shoves the FOP into the least-pri queue, rather than its 
> default. The value is honored all the way down to the bricks.
>
> The code is actually complete, and I'll put it up for review after we iron 
> out a few minor issues.

Did you get a chance to send the patch? Just wanted to run some tests
and see if this is all we need at the moment to regulate shd traffic,
especially with Richard's multi-threaded heal patch
https://urldefense.proofpoint.com/v2/url?u=http-3A__review.gluster.org_-23_c_13329_=CwIC-g=5VD0RTtNlTh3ycd41b3MUw=qJ8Lp7ySfpQklq3QZr44Iw=B873EiTlTeUXIjEcoutZ6Py5KL0bwXIVroPbpwaKD8s=fo86UTOQWXf0nQZvvauqIIhlwoZHpRlQMNfQd7Ubu7g=
  being revived and made ready for 3.8.

-Ravi

>
>> On Jan 27, 2016, at 9:48 PM, Ravishankar N  wrote:
>>
>> On 01/26/2016 08:41 AM, Richard Wareing wrote:
>>> In any event, it might be worth having Shreyas detail his throttling 
>>> feature (that can throttle any directory hierarchy no less) to illustrate 
>>> how a simpler design can achieve similar results to these more complicated 
>>> (and it

[Gluster-devel] 3.6.8 crashing a lot in production

2016-02-12 Thread Joe Julian

I have multiple bricks crashing in production. Any help would be greatly 
appreciated.


The crash log is in this bug report: 
https://bugzilla.redhat.com/show_bug.cgi?id=1307146


Looks like it's crashing in pl_inodelk_client_cleanup
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.6.8 crashing a lot in production

2016-02-12 Thread Krutika Dhananjay

Taking a look. Give me some time. 

-Krutika 

- Original Message -

> From: "Joe Julian" 
> To: "Krutika Dhananjay" , gluster-devel@gluster.org
> Sent: Saturday, February 13, 2016 6:02:13 AM
> Subject: Fwd: [Gluster-devel] 3.6.8 crashing a lot in production

> Could this be a regression from http://review.gluster.org/7981 ?

>  Forwarded Message  Subject:  [Gluster-devel] 3.6.8 crashing
> a lot in production
> Date: Fri, 12 Feb 2016 16:20:59 -0800
> From: Joe Julian 

> To:   gluster-us...@gluster.org , gluster-devel@gluster.org

> I have multiple bricks crashing in production. Any help would be greatly
> appreciated.

> The crash log is in this bug report:
> https://bugzilla.redhat.com/show_bug.cgi?id=1307146 Looks like it's crashing
> in pl_inodelk_client_cleanup
> ___
> Gluster-devel mailing list Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Fwd: 3.6.8 crashing a lot in production

2016-02-12 Thread Joe Julian


Could this be a regression from http://review.gluster.org/7981 ?

 Forwarded Message 
Subject:[Gluster-devel] 3.6.8 crashing a lot in production
Date:   Fri, 12 Feb 2016 16:20:59 -0800
From:   Joe Julian 
To: gluster-us...@gluster.org, gluster-devel@gluster.org



I have multiple bricks crashing in production. Any help would be greatly
appreciated.

The crash log is in this bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1307146

Looks like it's crashing in pl_inodelk_client_cleanup
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] 3.6.8 glusterfsd processes not responding

2016-02-12 Thread Joe Julian


I've also got several glusterfsd processes that have stopped responding.

A backtrace from a live core, strace, and state dump follow:

Thread 10 (LWP 31587):
#0  0x7f81d384289c in __lll_lock_wait () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81d383e065 in _L_lock_858 () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#2  0x7f81d383deba in pthread_mutex_lock () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#3  0x7f81ce600ff8 in pl_inodelk_client_cleanup ()
   from /usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/features/locks.so
#4  0x7f81ce5fe84a in ?? () from 
/usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/features/locks.so
#5  0x7f81d3ef573d in gf_client_disconnect () from 
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0
#6  0x7f81cd74e270 in server_connection_cleanup ()
   from /usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/protocol/server.so
#7  0x7f81cd7486ec in server_rpc_notify ()
   from /usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/protocol/server.so
#8  0x7f81d3c70f1b in rpcsvc_handle_disconnect () from 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0
#9  0x7f81d3c710b0 in rpcsvc_notify () from 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0
#10 0x7f81d3c74257 in rpc_transport_notify () from 
/usr/lib/x86_64-linux-gnu/libgfrpc.so.0
#11 0x7f81cf4d4077 in ?? () from 
/usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/rpc-transport/socket.so
#12 0x7f81d3ef793b in ?? () from /usr/lib/x86_64-linux-gnu/libglusterfs.so.0
#13 0x7f81d4348f71 in main ()

Thread 9 (LWP 3385):
#0  0x7f81d353408d in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x7f81d3533f2c in sleep () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x7f81cee50c38 in ?? () from 
/usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/storage/posix.so
#3  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#4  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x in ?? ()

Thread 8 (LWP 20656):
#0  0x7f81d38400fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81ce3ea032 in iot_worker ()
   from 
/usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/performance/io-threads.so
#2  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#3  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x in ?? ()

Thread 7 (LWP 31881):
#0  0x7f81d383fd84 in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81cee50f3b in posix_fsyncer_pick ()
   from /usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/storage/posix.so
#2  0x7f81cee51155 in posix_fsyncer ()
   from /usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/storage/posix.so
#3  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#4  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x in ?? ()

Thread 6 (LWP 31880):
#0  0x7f81d38400fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81cee4de5a in ?? () from 
/usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/storage/posix.so
---Type  to continue, or q  to quit---
#2  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#3  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x in ?? ()

Thread 5 (LWP 31842):
#0  0x7f81d383fd84 in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81cdfd77fb in index_worker ()
   from /usr/lib/x86_64-linux-gnu/glusterfs/3.6.8/xlator/features/index.so
#2  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#3  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x in ?? ()

Thread 4 (LWP 31591):
#0  0x7f81d38400fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81d3eddfe3 in syncenv_task () from 
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0
#2  0x7f81d3ede440 in syncenv_processor () from 
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0
#3  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#4  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x in ?? ()

Thread 3 (LWP 31590):
#0  0x7f81d38400fe in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7f81d3eddfe3 in syncenv_task () from 
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0
#2  0x7f81d3ede440 in syncenv_processor () from 
/usr/lib/x86_64-linux-gnu/libglusterfs.so.0
#3  0x7f81d383be9a in start_thread () from 
/lib/x86_64-linux-gnu/libpthread.so.0
#4  0x7f81d35683fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x in ?? ()

Thread 2 (LWP 31589):
#0  0x7f81d38439f7 in do_sigwait () from

Re: [Gluster-devel] Throttling xlator on the bricks

2016-02-12 Thread Pranith Kumar Karampuri




On 02/13/2016 12:13 AM, Richard Wareing wrote:

Hey Ravi,

I'll ping Shreyas about this today.  There's also a patch we'll need for 
multi-threaded SHD to fix the least-pri queuing.  The PID of the process wasn't 
tagged correctly via the call frame in my original patch.  The patch below 
fixes this (for 3.6.3), I didn't see multi-threaded self heal on github/master 
yet so let me know what branch you need this patch on and I can come up with a 
clean patch.


Hi Richard,
 I reviewed the patch and found that the same needs to be done 
even for ec. So I am thinking if I can split it out as two different 
patches, 1 patch in syncop-utils which builds the functionality of 
parallelization. Another patch which uses this in afr, ec. Do you mind 
if I give it a go? I can complete it by end of Wednesday.


Pranith


Richard


=


diff --git a/xlators/cluster/afr/src/afr-self-heald.c 
b/xlators/cluster/afr/src/afr-self-heald.c
index 028010d..b0f6248 100644
--- a/xlators/cluster/afr/src/afr-self-heald.c
+++ b/xlators/cluster/afr/src/afr-self-heald.c
@@ -532,6 +532,9 @@ afr_mt_process_entries_done (int ret, call_frame_t 
*sync_frame,
  pthread_cond_signal (_data->task_done);
  }
  pthread_mutex_unlock (_data->lock);
+
+if (task_ctx->frame)
+AFR_STACK_DESTROY (task_ctx->frame);
  GF_FREE (task_ctx);
  return 0;
  }
@@ -787,6 +790,7 @@ _afr_mt_create_process_entries_task (xlator_t *this,
  int   ret = -1;
  afr_mt_process_entries_task_ctx_t *task_ctx;
  afr_mt_data_t *mt_data;
+call_frame_t  *frame = NULL;

  mt_data = >mt_data;

@@ -799,6 +803,8 @@ _afr_mt_create_process_entries_task (xlator_t *this,
  if (!task_ctx)
  goto err;

+task_ctx->frame = afr_frame_create (this);
+
  INIT_LIST_HEAD (_ctx->list);
  task_ctx->readdir_xl = this;
  task_ctx->healer = healer;
@@ -812,7 +818,7 @@ _afr_mt_create_process_entries_task (xlator_t *this,
  // This returns immediately, and afr_mt_process_entries_done will
  // be called when the task is completed e.g. our queue is empty
  ret = synctask_new (this->ctx->env, afr_mt_process_entries_task,
-afr_mt_process_entries_done, NULL,
+afr_mt_process_entries_done, task_ctx->frame,
  (void *)task_ctx);

  if (!ret) {
diff --git a/xlators/cluster/afr/src/afr-self-heald.h 
b/xlators/cluster/afr/src/afr-self-heald.h
index 817e712..1588fc8 100644
--- a/xlators/cluster/afr/src/afr-self-heald.h
+++ b/xlators/cluster/afr/src/afr-self-heald.h
@@ -74,6 +74,7 @@ typedef struct afr_mt_process_entries_task_ctx_ {
  subvol_healer_t *healer;
  xlator_t*readdir_xl;
  inode_t *idx_inode;  /* inode ref for xattrop dir */
+call_frame_t*frame;
  unsigned intentries_healed;
  unsigned intentries_processed;
  unsigned intalready_healed;


Richard

From: Ravishankar N [ravishan...@redhat.com]
Sent: Sunday, February 07, 2016 11:15 PM
To: Shreyas Siravara
Cc: Richard Wareing; Vijay Bellur; Gluster Devel
Subject: Re: [Gluster-devel] Throttling xlator on the bricks

Hello,

On 01/29/2016 06:51 AM, Shreyas Siravara wrote:

So the way our throttling works is (intentionally) very simplistic.

(1) When someone mounts an NFS share, we tag the frame with a 32 bit hash of 
the export name they were authorized to mount.
(2) io-stats keeps track of the "current rate" of fops we're seeing for that 
particular mount, using a sampling of fops and a moving average over a short period of 
time.
(3) Based on whether the share violated its allowed rate (which is defined in a config 
file), we tag the FOP as "least-pri". Of course this makes the assumption that 
all NFS endpoints are receiving roughly the same # of FOPs. The rate defined in the 
config file is a *per* NFS endpoint number. So if your cluster has 10 NFS endpoints, and 
you've pre-computed that it can do roughly 1000 FOPs per second, the rate in the config 
file would be 100.
(4) IO-Threads then shoves the FOP into the least-pri queue, rather than its 
default. The value is honored all the way down to the bricks.

The code is actually complete, and I'll put it up for review after we iron out 
a few minor issues.

Did you get a chance to send the patch? Just wanted to run some tests
and see if this is all we need at the moment to regulate shd traffic,
especially with Richard's multi-threaded heal patch
https://urldefense.proofpoint.com/v2/url?u=http-3A__review.gluster.org_-23_c_13329_=CwIC-g=5VD0RTtNlTh3ycd41b3MUw=qJ8Lp7ySfpQklq3QZr44Iw=B873EiTlTeUXIjEcoutZ6Py5KL0bwXIVroPbpwaKD8s=fo86UTOQWXf0nQZvvauqIIhlwoZHpRlQMNfQd7Ubu7g=
  being revived

Re: [Gluster-devel] NSR: Suggestions for a new name

Re: [Gluster-devel] NSR: Suggestions for a new name

Re: [Gluster-devel] libgfapi libvirt memory leak version 3.7.8

Re: [Gluster-devel] NSR: Suggestions for a new name

Re: [Gluster-devel] [Gluster-users] Quota list not reflecting disk usage

Re: [Gluster-devel] [Gluster-users] Quota list not reflecting disk usage

Re: [Gluster-devel] libgfapi libvirt memory leak version 3.7.8

Re: [Gluster-devel] [Gluster-users] GlusterFS-3.7.6-2 packages for Debian Wheezy now available

Re: [Gluster-devel] [Gluster-users] GlusterFS-3.7.6-2 packages for Debian Wheezy now available

Re: [Gluster-devel] [Gluster-users] Gluster samba panic?

Re: [Gluster-devel] [Gluster-users] GlusterFS-3.7.6-2 packages for Debian Wheezy now available

Re: [Gluster-devel] changelog bug

[Gluster-devel] libgfapi libvirt memory leak version 3.7.8

Re: [Gluster-devel] Downtime for Gerrit and Jenkins - 12th February

Re: [Gluster-devel] Feature: Automagic lock-revocation for features/locks xlator (v3.7.x)

Re: [Gluster-devel] Throttling xlator on the bricks

[Gluster-devel] 3.6.8 crashing a lot in production

Re: [Gluster-devel] 3.6.8 crashing a lot in production

[Gluster-devel] Fwd: 3.6.8 crashing a lot in production

[Gluster-devel] 3.6.8 glusterfsd processes not responding

Re: [Gluster-devel] Throttling xlator on the bricks

21 matches

Site Navigation

Mail list logo

Footer information