Re: [Gluster-devel] Integrating liburcu source into the glusterfs source tree

[Niels de Vos]

On Mon, Feb 02, 2015 at 09:10:36AM +0530, Kaushal M wrote:
> Thanks for this information Kaleb.
> 
> I'll check the changes I've done till now with the older versions of the
> libraries. I think I'm going to need at least the 0.8.* release of liburcu,
> as some new apis were introduced in it, which I'm using. I'll post the
> outcome of my tests back here.
> 
> For a start, I collected the various versions of liburcu (userspace-rcu in
> some) available in the different distros. This list is based on the distros
> for which we provide community built packages and test on.
> 
> - Fedora 21 - 0.8.1 (0.8.5 in testing but stuck due to some breakages)
> - Fedora 20 - 0.7.7 (0.8.5 in testing but stuck due to some breakages)
> - EL7 - 0.7.9
> - EL6 - 0.7.7
> - Debian Wheezy - 0.6.7
> - Debian Jessie - 0.8.5 (in testing)
> - Ubuntu Precise - 0.6.7
> - Ubuntu Trusty - 0.7.12
> - Ubuntu Utopic - 0.8.4
> - Netbsd - 0.8.6
> - Freebsd - 0.7.7
> 
> The list doesn't look too good.

I do not like including libraries in the glusterfs sources. Currently
there are several bits under contrib/ that do not see any maintenance.
Who will be maintaining (fixing potential bugs, backporting newer
versions, ...) for linurcu? Note that we support *many* different
distributions, architectures and master+3 releases.

It would be *so* much more efficient to use the distributions version of
liburcu. Maybe it is possible to write some wrapper functions for the
older versions of the library, and place those wrappers in contrib/
instead?

Alternatively, we could maintain packages for liburcu in our
repositories on download.gluster.org for distributions that do not have
a recent enough version. You will need to find a relyable packager that
agrees to take on this task.

Niels

> 
> ~kaushal
> 
> 
> On Fri, Jan 30, 2015 at 6:30 PM, Kaleb KEITHLEY  wrote:
> 
> > Hi,
> >
> > Just FYI, what you propose is called bundling in Fedora packaging
> > parlance, and Fedora's packaging guidelines forbid bundling. It is possible
> > to get an exception granted, but it's not safe to presume that an exception
> > will be granted.
> >
> > (For downstream this is a non-issue, but here on gluster-devel we're not
> > concerned with downstream.)
> >
> > You either need to convince the maintainers of liburcu to update to the
> > newer versions everywhere, or you need to implement using the oldest
> > version on the platforms you intend to support. And if this is too onerous,
> > e.g. to use what's in (RH)EL5, then it's another argument in favor of
> > dropping support for (RH)EL5. (The other argument is that python on RHEL5
> > is too old for geo-rep.)
> >
> > In short, those of use who package gluster in Fedora would be, however
> > reluctantly, required to vote against doing this. I recommend contacting
> > the liburcu maintainers in Fedora/EPEL and see if you can't convince them
> > to update to the newest version.
> >
> > Regards,
> >
> > --
> >
> > Kaleb
> >
> > /30/2015 12:09 PM, Deepak Shetty wrote:
> >
> >>
> >>
> >> On Thu, Jan 29, 2015 at 5:05 PM, Kaushal M  >> > wrote:
> >>
> >> Hi all,
> >> I had started a thread previously on the efforts we are undertaking
> >> to improve thread synchronization in GlusterD [1]. I had mentioned
> >> that we will be using RCU for synchronization and the userspace RCU
> >> library (liburcu) [2] for implementation.
> >>
> >> I am now in a almost in a position to submit changes to Gerrit for
> >> review. But, I have an obstacle of making liburcu available on the
> >> jenkins slaves.
> >>
> >> I have begun development using the 0.8.6 version of liburcu, which
> >> is the latest stable release. EPEL has liburcu packages for CentOS 6
> >> and 7, but they are the of the older 0.7.* versions. Fedora has
> >> packages more recent packages, but they are still older, 0.8.1. [3].
> >>
> >> Considering the above situation with binary packages, I'm
> >> considering adding liburcu into the GlusterFS tree as a part of
> >> /contrib. This will be similar in vein to the argp-standalone library.
> >>
> >> liburcu is licensed under LGPL-v2.1, so I don't think there is going
> >> to be any problem including it. But IANAL, so I would like to know
> >> of if this would if this is okay from a legal perspective.
> >>
> >> I'll add the liburcu source to our tree and push the change for
> >> review. I'm not really familiar with autotools, so I'll need some
> >> help integrating it into our build system. I'll update the list when
> >> I have pushed the change for review.
> >>
> >>
> >> How do you intend to add, as a git submodule or ?
> >> I had worked on GNU autotools in the past, but frankly don't remember
> >> much of it. If any help is needed I can try, or can get someone to help
> >> from my ex-company :)
> >>
> >> thanx,
> >> deepak
> >>
> >>
> >>
> >> ___
> >> Gluster-devel mailing lis

Re: [Gluster-devel] Integrating liburcu source into the glusterfs source tree

[Kaushal M]

I'm currently testing my changes with liburcu-0.7.* . It is missing just
one API which I'm using from 0.8. I've written a local implementation of
just that one function, and am currently in process of testing this. If
this test is successful, then our problems will be minimized.

This only leaves out Debian Wheezy and Ubuntu Precise with liburcu-0.6.* .
I tried testing this out, but as liburcu-0.6 doesn't apparently provide
pkg-config configurations, I stopped as soon as I started.

~kaushal


On Mon, Feb 2, 2015 at 2:37 PM, Niels de Vos  wrote:

> On Mon, Feb 02, 2015 at 09:10:36AM +0530, Kaushal M wrote:
> > Thanks for this information Kaleb.
> >
> > I'll check the changes I've done till now with the older versions of the
> > libraries. I think I'm going to need at least the 0.8.* release of
> liburcu,
> > as some new apis were introduced in it, which I'm using. I'll post the
> > outcome of my tests back here.
> >
> > For a start, I collected the various versions of liburcu (userspace-rcu
> in
> > some) available in the different distros. This list is based on the
> distros
> > for which we provide community built packages and test on.
> >
> > - Fedora 21 - 0.8.1 (0.8.5 in testing but stuck due to some breakages)
> > - Fedora 20 - 0.7.7 (0.8.5 in testing but stuck due to some breakages)
> > - EL7 - 0.7.9
> > - EL6 - 0.7.7
> > - Debian Wheezy - 0.6.7
> > - Debian Jessie - 0.8.5 (in testing)
> > - Ubuntu Precise - 0.6.7
> > - Ubuntu Trusty - 0.7.12
> > - Ubuntu Utopic - 0.8.4
> > - Netbsd - 0.8.6
> > - Freebsd - 0.7.7
> >
> > The list doesn't look too good.
>
> I do not like including libraries in the glusterfs sources. Currently
> there are several bits under contrib/ that do not see any maintenance.
> Who will be maintaining (fixing potential bugs, backporting newer
> versions, ...) for linurcu? Note that we support *many* different
> distributions, architectures and master+3 releases.
>
> It would be *so* much more efficient to use the distributions version of
> liburcu. Maybe it is possible to write some wrapper functions for the
> older versions of the library, and place those wrappers in contrib/
> instead?
>
> Alternatively, we could maintain packages for liburcu in our
> repositories on download.gluster.org for distributions that do not have
> a recent enough version. You will need to find a relyable packager that
> agrees to take on this task.
>
> Niels
>
> >
> > ~kaushal
> >
> >
> > On Fri, Jan 30, 2015 at 6:30 PM, Kaleb KEITHLEY 
> wrote:
> >
> > > Hi,
> > >
> > > Just FYI, what you propose is called bundling in Fedora packaging
> > > parlance, and Fedora's packaging guidelines forbid bundling. It is
> possible
> > > to get an exception granted, but it's not safe to presume that an
> exception
> > > will be granted.
> > >
> > > (For downstream this is a non-issue, but here on gluster-devel we're
> not
> > > concerned with downstream.)
> > >
> > > You either need to convince the maintainers of liburcu to update to the
> > > newer versions everywhere, or you need to implement using the oldest
> > > version on the platforms you intend to support. And if this is too
> onerous,
> > > e.g. to use what's in (RH)EL5, then it's another argument in favor of
> > > dropping support for (RH)EL5. (The other argument is that python on
> RHEL5
> > > is too old for geo-rep.)
> > >
> > > In short, those of use who package gluster in Fedora would be, however
> > > reluctantly, required to vote against doing this. I recommend
> contacting
> > > the liburcu maintainers in Fedora/EPEL and see if you can't convince
> them
> > > to update to the newest version.
> > >
> > > Regards,
> > >
> > > --
> > >
> > > Kaleb
> > >
> > > /30/2015 12:09 PM, Deepak Shetty wrote:
> > >
> > >>
> > >>
> > >> On Thu, Jan 29, 2015 at 5:05 PM, Kaushal M  > >> > wrote:
> > >>
> > >> Hi all,
> > >> I had started a thread previously on the efforts we are
> undertaking
> > >> to improve thread synchronization in GlusterD [1]. I had mentioned
> > >> that we will be using RCU for synchronization and the userspace
> RCU
> > >> library (liburcu) [2] for implementation.
> > >>
> > >> I am now in a almost in a position to submit changes to Gerrit for
> > >> review. But, I have an obstacle of making liburcu available on the
> > >> jenkins slaves.
> > >>
> > >> I have begun development using the 0.8.6 version of liburcu, which
> > >> is the latest stable release. EPEL has liburcu packages for
> CentOS 6
> > >> and 7, but they are the of the older 0.7.* versions. Fedora has
> > >> packages more recent packages, but they are still older, 0.8.1.
> [3].
> > >>
> > >> Considering the above situation with binary packages, I'm
> > >> considering adding liburcu into the GlusterFS tree as a part of
> > >> /contrib. This will be similar in vein to the argp-standalone
> library.
> > >>
> > >> liburcu is licensed under LGPL-v2.1, so I don't think there is
> going
> > >>  

Re: [Gluster-devel] Integrating liburcu source into the glusterfs source tree

[Kaushal M]

The local implementation is working well for liburcu-0.7. I'll be
continuing with this approach as it is going to cover most of the systems
we target. We can decide how we'll handle Debian Wheezy and Ubuntu Precise
later.

~kaushal

On Mon, Feb 2, 2015 at 3:00 PM, Kaushal M  wrote:

> I'm currently testing my changes with liburcu-0.7.* . It is missing just
> one API which I'm using from 0.8. I've written a local implementation of
> just that one function, and am currently in process of testing this. If
> this test is successful, then our problems will be minimized.
>
> This only leaves out Debian Wheezy and Ubuntu Precise with liburcu-0.6.* .
> I tried testing this out, but as liburcu-0.6 doesn't apparently provide
> pkg-config configurations, I stopped as soon as I started.
>
> ~kaushal
>
>
> On Mon, Feb 2, 2015 at 2:37 PM, Niels de Vos  wrote:
>
>> On Mon, Feb 02, 2015 at 09:10:36AM +0530, Kaushal M wrote:
>> > Thanks for this information Kaleb.
>> >
>> > I'll check the changes I've done till now with the older versions of the
>> > libraries. I think I'm going to need at least the 0.8.* release of
>> liburcu,
>> > as some new apis were introduced in it, which I'm using. I'll post the
>> > outcome of my tests back here.
>> >
>> > For a start, I collected the various versions of liburcu (userspace-rcu
>> in
>> > some) available in the different distros. This list is based on the
>> distros
>> > for which we provide community built packages and test on.
>> >
>> > - Fedora 21 - 0.8.1 (0.8.5 in testing but stuck due to some breakages)
>> > - Fedora 20 - 0.7.7 (0.8.5 in testing but stuck due to some breakages)
>> > - EL7 - 0.7.9
>> > - EL6 - 0.7.7
>> > - Debian Wheezy - 0.6.7
>> > - Debian Jessie - 0.8.5 (in testing)
>> > - Ubuntu Precise - 0.6.7
>> > - Ubuntu Trusty - 0.7.12
>> > - Ubuntu Utopic - 0.8.4
>> > - Netbsd - 0.8.6
>> > - Freebsd - 0.7.7
>> >
>> > The list doesn't look too good.
>>
>> I do not like including libraries in the glusterfs sources. Currently
>> there are several bits under contrib/ that do not see any maintenance.
>> Who will be maintaining (fixing potential bugs, backporting newer
>> versions, ...) for linurcu? Note that we support *many* different
>> distributions, architectures and master+3 releases.
>>
>> It would be *so* much more efficient to use the distributions version of
>> liburcu. Maybe it is possible to write some wrapper functions for the
>> older versions of the library, and place those wrappers in contrib/
>> instead?
>>
>> Alternatively, we could maintain packages for liburcu in our
>> repositories on download.gluster.org for distributions that do not have
>> a recent enough version. You will need to find a relyable packager that
>> agrees to take on this task.
>>
>> Niels
>>
>> >
>> > ~kaushal
>> >
>> >
>> > On Fri, Jan 30, 2015 at 6:30 PM, Kaleb KEITHLEY 
>> wrote:
>> >
>> > > Hi,
>> > >
>> > > Just FYI, what you propose is called bundling in Fedora packaging
>> > > parlance, and Fedora's packaging guidelines forbid bundling. It is
>> possible
>> > > to get an exception granted, but it's not safe to presume that an
>> exception
>> > > will be granted.
>> > >
>> > > (For downstream this is a non-issue, but here on gluster-devel we're
>> not
>> > > concerned with downstream.)
>> > >
>> > > You either need to convince the maintainers of liburcu to update to
>> the
>> > > newer versions everywhere, or you need to implement using the oldest
>> > > version on the platforms you intend to support. And if this is too
>> onerous,
>> > > e.g. to use what's in (RH)EL5, then it's another argument in favor of
>> > > dropping support for (RH)EL5. (The other argument is that python on
>> RHEL5
>> > > is too old for geo-rep.)
>> > >
>> > > In short, those of use who package gluster in Fedora would be, however
>> > > reluctantly, required to vote against doing this. I recommend
>> contacting
>> > > the liburcu maintainers in Fedora/EPEL and see if you can't convince
>> them
>> > > to update to the newest version.
>> > >
>> > > Regards,
>> > >
>> > > --
>> > >
>> > > Kaleb
>> > >
>> > > /30/2015 12:09 PM, Deepak Shetty wrote:
>> > >
>> > >>
>> > >>
>> > >> On Thu, Jan 29, 2015 at 5:05 PM, Kaushal M > > >> > wrote:
>> > >>
>> > >> Hi all,
>> > >> I had started a thread previously on the efforts we are
>> undertaking
>> > >> to improve thread synchronization in GlusterD [1]. I had
>> mentioned
>> > >> that we will be using RCU for synchronization and the userspace
>> RCU
>> > >> library (liburcu) [2] for implementation.
>> > >>
>> > >> I am now in a almost in a position to submit changes to Gerrit
>> for
>> > >> review. But, I have an obstacle of making liburcu available on
>> the
>> > >> jenkins slaves.
>> > >>
>> > >> I have begun development using the 0.8.6 version of liburcu,
>> which
>> > >> is the latest stable release. EPEL has liburcu packages for
>> CentOS 6
>> > >> and 7, but they are the of the older 0.7.* v

Re: [Gluster-devel] Integrating liburcu source into the glusterfs source tree

[Niels de Vos]

On Mon, Feb 02, 2015 at 03:00:00PM +0530, Kaushal M wrote:
> I'm currently testing my changes with liburcu-0.7.* . It is missing just
> one API which I'm using from 0.8. I've written a local implementation of
> just that one function, and am currently in process of testing this. If
> this test is successful, then our problems will be minimized.

Okay, sounds good!

> This only leaves out Debian Wheezy and Ubuntu Precise with liburcu-0.6.* .
> I tried testing this out, but as liburcu-0.6 doesn't apparently provide
> pkg-config configurations, I stopped as soon as I started.

Yeah, well, I guess it is acceptible to drop support for very old
distributions with new GlusterFS releases. Older versions of GlusterFS
will still be without liburcu and will continue to receive regular
bugfix updates.

Thanks,
Niels

> ~kaushal
> 
> 
> On Mon, Feb 2, 2015 at 2:37 PM, Niels de Vos  wrote:
> 
> > On Mon, Feb 02, 2015 at 09:10:36AM +0530, Kaushal M wrote:
> > > Thanks for this information Kaleb.
> > >
> > > I'll check the changes I've done till now with the older versions of the
> > > libraries. I think I'm going to need at least the 0.8.* release of
> > liburcu,
> > > as some new apis were introduced in it, which I'm using. I'll post the
> > > outcome of my tests back here.
> > >
> > > For a start, I collected the various versions of liburcu (userspace-rcu
> > in
> > > some) available in the different distros. This list is based on the
> > distros
> > > for which we provide community built packages and test on.
> > >
> > > - Fedora 21 - 0.8.1 (0.8.5 in testing but stuck due to some breakages)
> > > - Fedora 20 - 0.7.7 (0.8.5 in testing but stuck due to some breakages)
> > > - EL7 - 0.7.9
> > > - EL6 - 0.7.7
> > > - Debian Wheezy - 0.6.7
> > > - Debian Jessie - 0.8.5 (in testing)
> > > - Ubuntu Precise - 0.6.7
> > > - Ubuntu Trusty - 0.7.12
> > > - Ubuntu Utopic - 0.8.4
> > > - Netbsd - 0.8.6
> > > - Freebsd - 0.7.7
> > >
> > > The list doesn't look too good.
> >
> > I do not like including libraries in the glusterfs sources. Currently
> > there are several bits under contrib/ that do not see any maintenance.
> > Who will be maintaining (fixing potential bugs, backporting newer
> > versions, ...) for linurcu? Note that we support *many* different
> > distributions, architectures and master+3 releases.
> >
> > It would be *so* much more efficient to use the distributions version of
> > liburcu. Maybe it is possible to write some wrapper functions for the
> > older versions of the library, and place those wrappers in contrib/
> > instead?
> >
> > Alternatively, we could maintain packages for liburcu in our
> > repositories on download.gluster.org for distributions that do not have
> > a recent enough version. You will need to find a relyable packager that
> > agrees to take on this task.
> >
> > Niels
> >
> > >
> > > ~kaushal
> > >
> > >
> > > On Fri, Jan 30, 2015 at 6:30 PM, Kaleb KEITHLEY 
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > Just FYI, what you propose is called bundling in Fedora packaging
> > > > parlance, and Fedora's packaging guidelines forbid bundling. It is
> > possible
> > > > to get an exception granted, but it's not safe to presume that an
> > exception
> > > > will be granted.
> > > >
> > > > (For downstream this is a non-issue, but here on gluster-devel we're
> > not
> > > > concerned with downstream.)
> > > >
> > > > You either need to convince the maintainers of liburcu to update to the
> > > > newer versions everywhere, or you need to implement using the oldest
> > > > version on the platforms you intend to support. And if this is too
> > onerous,
> > > > e.g. to use what's in (RH)EL5, then it's another argument in favor of
> > > > dropping support for (RH)EL5. (The other argument is that python on
> > RHEL5
> > > > is too old for geo-rep.)
> > > >
> > > > In short, those of use who package gluster in Fedora would be, however
> > > > reluctantly, required to vote against doing this. I recommend
> > contacting
> > > > the liburcu maintainers in Fedora/EPEL and see if you can't convince
> > them
> > > > to update to the newest version.
> > > >
> > > > Regards,
> > > >
> > > > --
> > > >
> > > > Kaleb
> > > >
> > > > /30/2015 12:09 PM, Deepak Shetty wrote:
> > > >
> > > >>
> > > >>
> > > >> On Thu, Jan 29, 2015 at 5:05 PM, Kaushal M  > > >> > wrote:
> > > >>
> > > >> Hi all,
> > > >> I had started a thread previously on the efforts we are
> > undertaking
> > > >> to improve thread synchronization in GlusterD [1]. I had mentioned
> > > >> that we will be using RCU for synchronization and the userspace
> > RCU
> > > >> library (liburcu) [2] for implementation.
> > > >>
> > > >> I am now in a almost in a position to submit changes to Gerrit for
> > > >> review. But, I have an obstacle of making liburcu available on the
> > > >> jenkins slaves.
> > > >>
> > > >> I have begun development using the 0.8.6 version of liburcu, which
> > > >> i

Re: [Gluster-devel] Improving Geo-replication Status and Checkpoints

[Aravinda]

Thanks Sahina, replied inline.

--
regards
Aravinda

On 02/02/2015 12:55 PM, Sahina Bose wrote:


On 01/28/2015 04:07 PM, Aravinda wrote:

Background
--
We have `status` and `status detail` commands for GlusterFS 
geo-replication, This mail is to fix the existing issues in these 
command outputs. Let us know if we need any other columns which helps 
users to get meaningful status.


Existing output
---
Status command output
MASTER NODE - Master node hostname/IP
MASTER VOL - Master volume name
MASTER BRICK - Master brick path
SLAVE - Slave host and Volume name(HOST::VOL format)
STATUS - Stable/Faulty/Active/Passive/Stopped/Not Started
CHECKPOINT STATUS - Details about Checkpoint completion
CRAWL STATUS - Hybrid/History/Changelog

Status detail -
MASTER NODE - Master node hostname/IP
MASTER VOL - Master volume name
MASTER BRICK - Master brick path
SLAVE - Slave host and Volume name(HOST::VOL format)
STATUS - Stable/Faulty/Active/Passive/Stopped/Not Started
CHECKPOINT STATUS - Details about Checkpoint completion
CRAWL STATUS - Hybrid/History/Changelog
FILES SYNCD - Number of Files Synced
FILES PENDING - Number of Files Pending
BYTES PENDING - Bytes pending
DELETES PENDING - Number of Deletes Pending
FILES SKIPPED - Number of Files skipped


Issues with existing status and status detail:
--

1. Active/Passive and Stable/faulty status is mixed up - Same column 
is used to show both active/passive status as well as Stable/faulty 
status. If Active node goes faulty then by looking at the status it 
is difficult to understand Active node is faulty or the passive one.
2. Info about last synced time, unless we set checkpoint it is 
difficult to understand till what time data is synced to slave. For 
example, if a admin want's to know all the files synced which are 
created 15 mins ago, it is not possible without setting checkpoint.

3. Wrong values in metrics.
4. When multiple bricks present in same node. Status shows Faulty 
when one of the worker is faulty in that node.


Changes:

1. Active nodes will be prefixed with * to identify it is a active 
node.(In xml output active tag will be introduced with values 0 or 1)
2. New column will show the last synced time, which minimizes the use 
of checkpoint feature. Checkpoint status will be shown only in status 
detail.
3. Checkpoint Status is removed, Separate Checkpoint command will be 
added to gluster cli(We can introduce multiple Checkpoint feature 
with this change)
4. Status values will be "Not 
Started/Initializing/Started/Faulty/Stopped". Stable is changed to 
"Started"
5. Slave User column will be introduced to show to which user geo-rep 
session is established.(Useful in Non root geo-rep)
6. Bytes pending column will be removed. It is not possible to 
identify the delta without simulating sync. For example, we are using 
rsync to sync data from master to slave, If we need to know how much 
data to be transferred then we have to run the rsync command with 
--dry-run flag before running actual command. With tar-ssh we have to 
stat all the files which are identified to be synced to calculate the 
total bytes to be synced. Both are costly operations which degrades 
the geo-rep performance.(In Future we can include these columns)
7. Files pending, Synced, deletes pending are only session 
information of the worker, these numbers will not match with the 
number of files present in Filesystem. If worker restarts, counter 
will reset to zero. When worker restarts, it logs previous session 
stats before resetting it.
8. Files Skipped is persistent status across sessions, Shows exact 
count of number of files skipped(Can get list of GFIDs skipped from 
log file)

9. "Deletes Pending" column can be removed?


Is there any way to know if there are errors syncing any of the files? 
Which column would that reflect in?

"Skipped" Column shows number of files failed to sync to Slave.


Is the last synced time - the least of the synced time across the nodes?
Status output will have one entry for each brick, so we are planning to 
display last synced time from that brick.





Example output

MASTER NODE  MASTER VOL  MASTER BRICK  SLAVE USER 
SLAVE STATUSLAST SYNCED   CRAWL
 

* fedoravm1  gvm /gfs/b1   root fedoravm3::gvs 
Started   2014-05-10 03:07 pm   Changelog
  fedoravm2  gvm /gfs/b2   root fedoravm4::gvs 
Started   2014-05-10 03:07 pm   Changelog


New Status columns

ACTIVE_PASSIVE - * if Active else none.
MASTER NODE - Master node hostname/IP
MASTER VOL - Master volume name
MASTER BRICK - Master brick path
SLAVE USER - Slave user to which geo-rep is established.
SLAVE - Slave host and Volume name(HOST::VOL format)
STATUS - S

Re: [Gluster-devel] [Gluster-infra] Possibility to move our Jenkins setup to the CentOS Jenkins infra

[Niels de Vos]

On Sat, Jan 31, 2015 at 07:43:39AM -0500, Justin Clift wrote:
> Hi all,
> 
> One of the things which has been a fair drag for the developer part of
> the Gluster Community, is maintaining our own Jenkins infrastructure.
> 
> When chatting with the CentOS guys (pre-FOSDEM) the other day, there is
> the possibility we could use their Jenkins infrastructure instead.
> 
> They have a *bunch* of physical nodes (100+), and several other projects
> are using it as well (Ceph, and others).
> 
> This would potentially make things easier for us, and they're not limited
> to running things on CentOS either.  They already have FreeBSD nodes,
> and don't foresee problems with getting NetBSD up and running.
> 
> At the moment, I'm kind of having the opinion this could be a good
> way forward, and we should probably try out our smoke/regression scripting
> on their setup to see if it's workable.
> 
> And if it turns out to be, then great, and we migrate all of our jenkins
> there then decommission ours (including the master. :)
> 
> What do people reckon?

Sounds good to me in general. Is there a document that describes their
Jenkins setupi (like provisioning of the slaves)? Note that we do not
(yet) have a way of automated multi-host tests in Jenkins, but that
should be looked into one day.  Also, is it possible for Gluster
Community members to log into a Jenkins slave and troubleshoot specific
failures?

Thanks,
Niels


pgpKxkJHtu_71.pgp
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] GFID to Path Conversion if Historical Changelogs available

[Aravinda]

Hi,

While discussing about the possible options to convert from GFID to 
path, Venky suggested to search in Historical GlusterFS Changelogs and 
get the information about Parent GFID and basename of the file. This 
method works well for FOPs except Rename and Hard links.


Approach:
-
Will consider b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 as GFID for all 
illustrations and file name as file1


1. If .glusterfs/b6/7b/b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 is Symlink, 
Readlink and Convert to relative path and Print(Easy conversion, done :))
2. If not Symlink, Stat on file 
.glusterfs/b6/7b/b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 and get 
st_ctime(For example ctime is 1422694569)
3. Search for CHANGELOG.1422694565, CHANGELOG.1422694566.. 
CHANGELOG.1422694590. Break when you get actual Changelog file, else 
break after MAX try.

4. If we didn't get valid Changelog file for that GFID, log GFID in stderr.
5. If Changelog file is available, Search in Changelog file for the 
ENTRY pattern. If found extract Parent GFID and basename. Convert Parent 
GFID to Path using readlink and add basename.
6. Get xattr trusted.gfid on the path we got and compare with input 
GFID. If both are same then we are Good, else log in stderr. (File may 
be changed after Create, may be Rename)
7. Collect all GFIDs from stderr and convert to Path using find . 
-samefile 


I created a Script using Python to find Path from GFID. Works really 
well if Changelog is enabled before file/dir Creation.


https://github.com/aravindavk/gfid_to_path

Usage:
--
Download the Python Script(gfid_to_path) from 
https://github.com/aravindavk/gfid_to_path and place it in any bin 
directory which is available in PATH


chmod +x /usr/local/bin/gfid_to_path

To convert single GFID to Path
--
echo  | gfid_to_path 

echo b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 | gfid_to_path 
/exports/bricks/b1


To convert list of GFIDs into Path,
--
gfid_to_path  

gfid_to_path /exports/bricks/b1 ~/collected_gfids.txt

To record Converted paths in ~/good.txt and failures in ~/bad.txt
-
gfid_to_path   1> GOOD_FILE 2> BAD_FILE

gfid_to_path /exports/bricks/b1 ~/collected_gfids.txt 1> ~/good.txt 
2> ~/bad.txt



Source is available in Github.
git clone https://github.com/aravindavk/gfid_to_path.git

C & S Welcome.

--
regards
Aravinda
http://aravindavk.in
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GFID to Path Conversion if Historical Changelogs available

[Prashanth Pai]

Nice :) Do mention your script in the upstream doc here:
https://github.com/gluster/glusterfs/blob/master/doc/debugging/gfid-to-path.md

Regards,
 -Prashanth Pai

- Original Message -
From: "Aravinda" 
To: "Gluster Devel" 
Sent: Monday, February 2, 2015 5:20:56 PM
Subject: [Gluster-devel] GFID to Path Conversion if Historical Changelogs   
available

Hi,

While discussing about the possible options to convert from GFID to 
path, Venky suggested to search in Historical GlusterFS Changelogs and 
get the information about Parent GFID and basename of the file. This 
method works well for FOPs except Rename and Hard links.

Approach:
-
Will consider b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 as GFID for all 
illustrations and file name as file1

1. If .glusterfs/b6/7b/b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 is Symlink, 
Readlink and Convert to relative path and Print(Easy conversion, done :))
2. If not Symlink, Stat on file 
.glusterfs/b6/7b/b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 and get 
st_ctime(For example ctime is 1422694569)
3. Search for CHANGELOG.1422694565, CHANGELOG.1422694566.. 
CHANGELOG.1422694590. Break when you get actual Changelog file, else 
break after MAX try.
4. If we didn't get valid Changelog file for that GFID, log GFID in stderr.
5. If Changelog file is available, Search in Changelog file for the 
ENTRY pattern. If found extract Parent GFID and basename. Convert Parent 
GFID to Path using readlink and add basename.
6. Get xattr trusted.gfid on the path we got and compare with input 
GFID. If both are same then we are Good, else log in stderr. (File may 
be changed after Create, may be Rename)
7. Collect all GFIDs from stderr and convert to Path using find . 
-samefile 

I created a Script using Python to find Path from GFID. Works really 
well if Changelog is enabled before file/dir Creation.

https://github.com/aravindavk/gfid_to_path

Usage:
--
Download the Python Script(gfid_to_path) from 
https://github.com/aravindavk/gfid_to_path and place it in any bin 
directory which is available in PATH

 chmod +x /usr/local/bin/gfid_to_path

To convert single GFID to Path
--
 echo  | gfid_to_path 

 echo b67b95a4-8bdc-44d5-aedd-b9b66fb0af95 | gfid_to_path 
/exports/bricks/b1

To convert list of GFIDs into Path,
--
 gfid_to_path  

 gfid_to_path /exports/bricks/b1 ~/collected_gfids.txt

To record Converted paths in ~/good.txt and failures in ~/bad.txt
-
 gfid_to_path   1> GOOD_FILE 2> BAD_FILE

 gfid_to_path /exports/bricks/b1 ~/collected_gfids.txt 1> ~/good.txt 
2> ~/bad.txt


Source is available in Github.
 git clone https://github.com/aravindavk/gfid_to_path.git

C & S Welcome.

--
regards
Aravinda
http://aravindavk.in
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] submitted but merge pending issue seems to be back :-(

[Pranith Kumar Karampuri]

I see the following two patches in that state:
http://review.gluster.com/9456
http://review.gluster.com/9409

Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] submitted but merge pending issue seems to be back :-(

[Venky Shankar]

... and review.g.o does not allow to "publish and submit".

On Mon, Feb 2, 2015 at 5:47 PM, Pranith Kumar Karampuri
 wrote:
> I see the following two patches in that state:
> http://review.gluster.com/9456
> http://review.gluster.com/9409
>
> Pranith
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] submitted but merge pending issue seems to be back :-(

[Niels de Vos]

On Mon, Feb 02, 2015 at 05:47:31PM +0530, Pranith Kumar Karampuri wrote:
> I see the following two patches in that state:
> http://review.gluster.com/9456
> http://review.gluster.com/9409

http://review.gluster.com/#/q/status:submitted,n,z has a list of all of
them :-/

http://review.gluster.com/9518 seems to be stuck in a loop or something?

Please have a look at
https://bugzilla.redhat.com/show_bug.cgi?id=1146985#c1 for fixing the
issue.

Who said a Gerrit update was overdue?
Niels


pgpM69aSItBbk.pgp
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] RFC: d_off encoding at client/protocol layer

[Dan Lambright]

Hello,

We have a prototype of this working on the tiering forge site, and noticed that 
in this scheme, each client translator needs to "know" the total number of 
#bricks in the volume. We can compute that number when the graph is created, 
and on a graph switch. But in some sense, a downside of this is it runs against 
a gluster design principal that a translator is only "aware" of the translators 
it is attached to. 

I can imagine an alternative: the d_off could set aside a fixed number of bits 
for the sub volume. The number would be as many needed for the maximum number 
of sub volumes we support. So for example,  if we support 2048 sub volumes, 11 
bits would be set aside in the d_off. This would mean the client translator 
would not need to know the #bricks.

The disadvantage to this approach is the more bits we use for the sub volume, 
the fewer remaining are available for the offset handed to us from the file 
system- and this can lead to a greater probability of losing track of which 
file in a directory we left off on. 

Overall, it seems the arguments I've heard in support of keeping with the 
current approach (dynamic # bits set aside for the sub volume) are stronger, 
given in most deployments the number of sub volumes occupies a much smaller 
number of bits.

Dan

- Original Message -
> From: "Shyam" 
> To: "Gluster Devel" 
> Cc: "Dan Lambright" 
> Sent: Monday, January 26, 2015 8:59:14 PM
> Subject: RFC: d_off encoding at client/protocol layer
> 
> Hi,
> 
> Some parts of this topic has been discussed in the recent past here [1]
> 
> The current mechanism of each xlator encoding the subvol in the lower or
> higher bits has its pitfalls as discussed in the threads and in this
> review, here [2]
> 
> Here is a solution design from the one of the comments posted on this by
> Avati here, [3], as in,
> 
> "One example approach (not necessarily the best): Make every xlator
> knows the total number of leaf xlators (protocol/clients), and also the
> number of all leaf xlators from each of its subvolumes. This way, the
> protocol/client xlators (alone) do the encoding, by knowing its global
> brick# and total #of bricks. The cluster xlators blindly forward the
> readdir_cbk without any further transformations of the d_offs, and also
> route the next readdir(old_doff) request to the appropriate subvolume
> based on the weighted graph (of counts of protocol/clients in the
> subtrees) till it reaches the right protocol/client to resume the
> enumeration."
> 
> So the current proposed scheme that is being worked on is as follows,
> - encode the d_off with the client/protocol ID, which is generated as
> its leaf position/number
> - no further encoding in any other xlator
> - on receiving further readdir requests with the d_off, consult the,
> graph/or immediate children, on ID encoded in the d_off, and send the
> request down that subvol path
> 
> IOW, given a d_off and a common routine, pass the d_off with this (i.e
> current xlator) to get a subvol that the d_off belongs to. This routine
> would decode the d_off for the leaf ID as encoded in the client/protocol
> layer, and match its subvol relative to this and send that for further
> processing. (it may consult the graph or store the range of IDs that any
> subvol has w.r.t client/protocol and deliver the result appropriately).
> 
> Given the current situation of ext4 and xfs, and continuing with the ID
> encoding scheme, this seems to be the best manner of preventing multiple
> encoding of subvol stomping on each other, and also preserving (in a
> sense) further loss of bits. This scheme would also give AFR/EC the
> ability to load balance readdir requests across its subvols better, than
> have a static subvol to send to for a longer duration.
> 
> Thoughts/comments?
> 
> Shyam
> 
> [1] https://www.mail-archive.com/gluster-devel@gluster.org/msg02834.html
> [2] review.gluster.org/#/c/8201/4/xlators/cluster/afr/src/afr-dir-read.c
> [3] https://www.mail-archive.com/gluster-devel@gluster.org/msg02847.html
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] 3.6.2 volume heal

[David F. Robinson]

I upgraded one of my bricks from 3.6.1 to 3.6.2 and I can no longer do a 
'gluster volume heal homegfs info'.  It hangs and never returns any 
information.
I was trying to ensure that gfs01a had finished healing before upgrading 
the other machines (gfs01b, gfs02a, gfs02b) in my configuration (see 
below).

'gluster volume homegfs statistics' still works fine.

Do I need to upgrade my other bricks to get the 'gluster volume heal 
homegfs info' working?  Or, should I fix this issue before upgrading my 
other machines?



Volume Name: homegfs
Type: Distributed-Replicate
Volume ID: 1e32672a-f1b7-4b58-ba94-58c085e59071
Status: Started
Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: gfsib01a.corvidtec.com:/data/brick01a/homegfs
Brick2: gfsib01b.corvidtec.com:/data/brick01b/homegfs
Brick3: gfsib01a.corvidtec.com:/data/brick02a/homegfs
Brick4: gfsib01b.corvidtec.com:/data/brick02b/homegfs
Brick5: gfsib02a.corvidtec.com:/data/brick01a/homegfs
Brick6: gfsib02b.corvidtec.com:/data/brick01b/homegfs
Brick7: gfsib02a.corvidtec.com:/data/brick02a/homegfs
Brick8: gfsib02b.corvidtec.com:/data/brick02b/homegfs
Options Reconfigured:
performance.io-thread-count: 32
performance.cache-size: 128MB
performance.write-behind-window-size: 128MB
server.allow-insecure: on
network.ping-timeout: 10
storage.owner-gid: 100
geo-replication.indexing: off
geo-replication.ignore-pid-check: on
changelog.changelog: on
changelog.fsync-interval: 3
changelog.rollover-time: 15
server.manage-gids: on

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] RFC: d_off encoding at client/protocol layer

[Jeff Darcy]

> IOW, given a d_off and a common routine, pass the d_off with this (i.e
> current xlator) to get a subvol that the d_off belongs to. This routine
> would decode the d_off for the leaf ID as encoded in the client/protocol
> layer, and match its subvol relative to this and send that for further
> processing. (it may consult the graph or store the range of IDs that any
> subvol has w.r.t client/protocol and deliver the result appropriately).

What happens to this scheme when bricks are repeatedly added/removed?
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] submitted but merge pending issue seems to be back :-(

[Vijay Bellur]

On 02/02/2015 01:37 PM, Niels de Vos wrote:

On Mon, Feb 02, 2015 at 05:47:31PM +0530, Pranith Kumar Karampuri wrote:

I see the following two patches in that state:
http://review.gluster.com/9456
http://review.gluster.com/9409


http://review.gluster.com/#/q/status:submitted,n,z has a list of all of
them :-/

http://review.gluster.com/9518 seems to be stuck in a loop or something?

Please have a look at
https://bugzilla.redhat.com/show_bug.cgi?id=1146985#c1 for fixing the
issue.



This issue seemed to be a different one. Noticed the following error in 
the logs:


org.eclipse.jgit.errors.ObjectWritingException: Unable to create new 
object:


Upon checking, I saw there were some permission problems on the 
glusterfs.git repository. I performed a recursive chown of glusterfs.git 
to review:gerrit2. It seems to be fixed now as a patch got merged after 
the chown.


I had to abandon a few patches before doing the recursive chown. 
Apologies for that. Please re-submit all abandoned patches and we'll 
ensure that they get merged on priority.


Thanks,
Vijay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] submitted but merge pending issue seems to be back :-(

[Vijay Bellur]

On 02/02/2015 09:14 PM, Vijay Bellur wrote:

On 02/02/2015 01:37 PM, Niels de Vos wrote:

On Mon, Feb 02, 2015 at 05:47:31PM +0530, Pranith Kumar Karampuri wrote:

I see the following two patches in that state:
http://review.gluster.com/9456
http://review.gluster.com/9409


http://review.gluster.com/#/q/status:submitted,n,z has a list of all of
them :-/

http://review.gluster.com/9518 seems to be stuck in a loop or something?

Please have a look at
https://bugzilla.redhat.com/show_bug.cgi?id=1146985#c1 for fixing the
issue.



This issue seemed to be a different one. Noticed the following error in
the logs:

org.eclipse.jgit.errors.ObjectWritingException: Unable to create new
object:

Upon checking, I saw there were some permission problems on the
glusterfs.git repository. I performed a recursive chown of glusterfs.git
to review:gerrit2. It seems to be fixed now as a patch got merged after
the chown.

I had to abandon a few patches before doing the recursive chown.
Apologies for that. Please re-submit all abandoned patches and we'll
ensure that they get merged on priority.



Update: I revived all patches that had been abandoned due to the gerrit 
problem and have merged them.


Can owners of these patches please verify that all is in order now?

Thanks,
Vijay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Possibility to move our Jenkins setup to the CentOS Jenkins infra

[Kaleb KEITHLEY]

Hell yeah.

On 01/31/2015 01:43 PM, Justin Clift wrote:

Hi all,

One of the things which has been a fair drag for the developer part of
the Gluster Community, is maintaining our own Jenkins infrastructure.

When chatting with the CentOS guys (pre-FOSDEM) the other day, there is
the possibility we could use their Jenkins infrastructure instead.

They have a *bunch* of physical nodes (100+), and several other projects
are using it as well (Ceph, and others).

This would potentially make things easier for us, and they're not limited
to running things on CentOS either.  They already have FreeBSD nodes,
and don't foresee problems with getting NetBSD up and running.

At the moment, I'm kind of having the opinion this could be a good
way forward, and we should probably try out our smoke/regression scripting
on their setup to see if it's workable.

And if it turns out to be, then great, and we migrate all of our jenkins
there then decommission ours (including the master. :)

What do people reckon?

Regards and best wishes,

Justin Clift



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] GlusterFS.next (a.k.a. 4.0) status/planning meeting

[Jeff Darcy]

Perhaps it's not obvious to the broader community, but a bunch of people
have put a bunch of work into various projects under the "4.0" banner.
Some of the results can be seen in the various feature pages here:

http://www.gluster.org/community/documentation/index.php/Planning40

Now that the various subproject feature pages have been updated, it's
time to get people together and decide what 4.0 is *really* going to be.
To that end, I'd like to schedule an IRC meeting for February 6 at 12:00
UTC - that's this Friday, same time as the triage/community meetings but
on Friday instead of Tuesday/Wednesday.  An initial agenda includes:

* Introduction and expectation-setting

* Project-by-project status and planning

* Discussion of future meeting formats and times

* Discussion of collaboration tools (e.g. gluster.org wiki or
  Freedcamp) going forward.

Anyone with an interest in the future of GlusterFS is welcome to attend.
This is *not* a Red Hat only effort, tied to Red Hat product needs and
schedules and strategies.  This is a chance for the community to come
together and define what the next generation of "distributed file
systems for the real world" will look like.  I hope to see everyone
there.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Possibility to move our Jenkins setup to the CentOS Jenkins infra

[Lalatendu Mohanty]

On 02/02/2015 12:25 PM, Niels de Vos wrote:

On Sat, Jan 31, 2015 at 07:43:39AM -0500, Justin Clift wrote:

Hi all,

One of the things which has been a fair drag for the developer part of
the Gluster Community, is maintaining our own Jenkins infrastructure.

When chatting with the CentOS guys (pre-FOSDEM) the other day, there is
the possibility we could use their Jenkins infrastructure instead.

They have a *bunch* of physical nodes (100+), and several other projects
are using it as well (Ceph, and others).

This would potentially make things easier for us, and they're not limited
to running things on CentOS either.  They already have FreeBSD nodes,
and don't foresee problems with getting NetBSD up and running.

At the moment, I'm kind of having the opinion this could be a good
way forward, and we should probably try out our smoke/regression scripting
on their setup to see if it's workable.

And if it turns out to be, then great, and we migrate all of our jenkins
there then decommission ours (including the master. :)

What do people reckon?

Sounds good to me in general. Is there a document that describes their
Jenkins setupi (like provisioning of the slaves)? Note that we do not
(yet) have a way of automated multi-host tests in Jenkins, but that
should be looked into one day.  Also, is it possible for Gluster
Community members to log into a Jenkins slave and troubleshoot specific
failures?

Thanks,
Niels


+1. We just need to make sure, CentOS would give required access to 
gluster community members even if they are not doing anything in CentOS 
community.


Thanks,
Lala



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] RFC: d_off encoding at client/protocol layer

[Krishnan Parthasarathi]

> > IOW, given a d_off and a common routine, pass the d_off with this (i.e
> > current xlator) to get a subvol that the d_off belongs to. This routine
> > would decode the d_off for the leaf ID as encoded in the client/protocol
> > layer, and match its subvol relative to this and send that for further
> > processing. (it may consult the graph or store the range of IDs that any
> > subvol has w.r.t client/protocol and deliver the result appropriately).
> 
> What happens to this scheme when bricks are repeatedly added/removed?

IIUC, the leaf xlator encoding proposed should be performed during graph
initialization and would remain static for the lifetime of the graph.
When bricks are added or removed, it would trigger a graph change, and
the new encoding would be computed. Further, it is guaranteed that
ongoing (readdir) FOPs would complete in the same (old) graph and therefore
the encoding should be unaffected by bricks being added/removed.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Integrating liburcu source into the glusterfs source tree

[Anand Avati]

Apologies for the top post.

Adopting RCU is a good step. Some questions and thoughts -

Does urcu work on non Linux systems, netbsd? IIRC there were Linux specific
permissions on the rcu patent? Maybe only for the kernel? Would be good to
confirm.

Glusterd is a good place for the first prototype adoption of rcu, esp
figuring out the nuances of liburcu (in my view). The perfect use case for
liburcu is still brewing in the form of epoll multithreading. That patch
creates the perfect conditions on the server side with many threads
servicing many clients bouncing the cacheline on so many shared objects and
locks - where rcu comes to the rescue. Starting with the events.c shared FD
registry, client_t registry, call-pool registry, inode table, each of these
are candidates which ask for rcu conversion. The unfortunate part is that
cacheline bouncing fixes are all or nothing. As long as there is at least
one shared lock in the hot path, the hard work gone into all the previous
shared lock fixes remain latent. However the end result is well worth all
the efforts.

Thanks

On Thu, Jan 29, 2015, 03:35 Kaushal M  wrote:

Hi all,

I had started a thread previously on the efforts we are undertaking to
improve thread synchronization in GlusterD [1]. I had mentioned that we
will be using RCU for synchronization and the userspace RCU library
(liburcu) [2] for implementation.

I am now in a almost in a position to submit changes to Gerrit for review.
But, I have an obstacle of making liburcu available on the jenkins slaves.

I have begun development using the 0.8.6 version of liburcu, which is the
latest stable release. EPEL has liburcu packages for CentOS 6 and 7, but
they are the of the older 0.7.* versions. Fedora has packages more recent
packages, but they are still older, 0.8.1. [3].

Considering the above situation with binary packages, I'm considering
adding liburcu into the GlusterFS tree as a part of /contrib. This will be
similar in vein to the argp-standalone library.

liburcu is licensed under LGPL-v2.1, so I don't think there is going to be
any problem including it. But IANAL, so I would like to know of if this
would if this is okay from a legal perspective.

I'll add the liburcu source to our tree and push the change for review. I'm
not really familiar with autotools, so I'll need some help integrating it
into our build system. I'll update the list when I have pushed the change
for review.

In the meantime, I'd like to know if anyone has any objections to this
plan. And also want to know of any alternative approaches.

 ~kaushal

[1]: http://

www.gluster.org
/

pipermail
/

gluster-devel

/2014-December/043382.html


[2]: http:// urcu.so/ 

[3]: https ://

apps.fedoraproject.org
/packages/
userspace-rcu


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http:// 
www.gluster.org 
/mailman/ listinfo
/
gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Integrating liburcu source into the glusterfs source tree

[Kaushal M]

liburcu is licensed under LGPLv2.1, and can be used by any software
compatible with LGPL. IBM, the owners of the patent, provided their
approval for this licensing [1]. We are good with regards to this.

The liburcu homepage mentions that it has been tested on Linux and FreeBSD,
but it should work on NetBSD as well. NetBSD has actively maintained
package of liburcu [3] required by KnotDNS (another project which uses
liburcu), so I'm assuming there aren't any problems there as well. We will
test our changes on these three platforms to guarantee that it indeed
works.

We've been referring to the PhD dissertation on RCU by Paul McKenney [4]
for help with implementation. Sections 5 and 6 of the dissertation discuss
RCU design patterns and examples of conversion to non-RCU code to RCU. This
has been a good reference for us so far.

~kaushal

[1]: https://github.com/urcu/userspace-rcu/blob/master/lgpl-relicensing.txt
[2]: http://urcu.so/ Under 'Architectures supported'
[3]: http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/devel/userspace-rcu/
[4]: http://www.rdrop.com/~paulmck/RCU/RCUdissertation.2004.07.14e1.pdf

On Tue, Feb 3, 2015 at 10:24 AM, Anand Avati  wrote:

> Apologies for the top post.
>
> Adopting RCU is a good step. Some questions and thoughts -
>
> Does urcu work on non Linux systems, netbsd? IIRC there were Linux
> specific permissions on the rcu patent? Maybe only for the kernel? Would be
> good to confirm.
>
> Glusterd is a good place for the first prototype adoption of rcu, esp
> figuring out the nuances of liburcu (in my view). The perfect use case for
> liburcu is still brewing in the form of epoll multithreading. That patch
> creates the perfect conditions on the server side with many threads
> servicing many clients bouncing the cacheline on so many shared objects and
> locks - where rcu comes to the rescue. Starting with the events.c shared FD
> registry, client_t registry, call-pool registry, inode table, each of these
> are candidates which ask for rcu conversion. The unfortunate part is that
> cacheline bouncing fixes are all or nothing. As long as there is at least
> one shared lock in the hot path, the hard work gone into all the previous
> shared lock fixes remain latent. However the end result is well worth all
> the efforts.
>
> Thanks
>
> On Thu, Jan 29, 2015, 03:35 Kaushal M  wrote:
>
> Hi all,
>
> I had started a thread previously on the efforts we are undertaking to
> improve thread synchronization in GlusterD [1]. I had mentioned that we
> will be using RCU for synchronization and the userspace RCU library
> (liburcu) [2] for implementation.
>
> I am now in a almost in a position to submit changes to Gerrit for review.
> But, I have an obstacle of making liburcu available on the jenkins slaves.
>
> I have begun development using the 0.8.6 version of liburcu, which is the
> latest stable release. EPEL has liburcu packages for CentOS 6 and 7, but
> they are the of the older 0.7.* versions. Fedora has packages more recent
> packages, but they are still older, 0.8.1. [3].
>
> Considering the above situation with binary packages, I'm considering
> adding liburcu into the GlusterFS tree as a part of /contrib. This will be
> similar in vein to the argp-standalone library.
>
> liburcu is licensed under LGPL-v2.1, so I don't think there is going to be
> any problem including it. But IANAL, so I would like to know of if this
> would if this is okay from a legal perspective.
>
> I'll add the liburcu source to our tree and push the change for review.
> I'm not really familiar with autotools, so I'll need some help integrating
> it into our build system. I'll update the list when I have pushed the
> change for review.
>
> In the meantime, I'd like to know if anyone has any objections to this
> plan. And also want to know of any alternative approaches.
>
>  ~kaushal
>
> [1]: http://
> 
> www.gluster.org
> 
> /
> 
> pipermail
> 
> /
> 
> gluster-devel
> 
> /2014-December/043382.html
> 
>
> [2]: http:// urcu.so/ 
>
> [3]: https ://
> 
> apps.fedoraproject.org
> /packages/
> userspace-rcu
> 
>
> ___
> Gluster-devel mailing list
> Gluster-devel@g

Re: [Gluster-devel] 3.6.2 volume heal

[Raghavendra Bhat]

On Monday 02 February 2015 09:07 PM, David F. Robinson wrote:
I upgraded one of my bricks from 3.6.1 to 3.6.2 and I can no longer do 
a 'gluster volume heal homegfs info'.  It hangs and never returns any 
information.
I was trying to ensure that gfs01a had finished healing before 
upgrading the other machines (gfs01b, gfs02a, gfs02b) in my 
configuration (see below).

'gluster volume homegfs statistics' still works fine.
Do I need to upgrade my other bricks to get the 'gluster volume heal 
homegfs info' working?  Or, should I fix this issue before upgrading 
my other machines?

Volume Name: homegfs
Type: Distributed-Replicate
Volume ID: 1e32672a-f1b7-4b58-ba94-58c085e59071
Status: Started
Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: gfsib01a.corvidtec.com:/data/brick01a/homegfs
Brick2: gfsib01b.corvidtec.com:/data/brick01b/homegfs
Brick3: gfsib01a.corvidtec.com:/data/brick02a/homegfs
Brick4: gfsib01b.corvidtec.com:/data/brick02b/homegfs
Brick5: gfsib02a.corvidtec.com:/data/brick01a/homegfs
Brick6: gfsib02b.corvidtec.com:/data/brick01b/homegfs
Brick7: gfsib02a.corvidtec.com:/data/brick02a/homegfs
Brick8: gfsib02b.corvidtec.com:/data/brick02b/homegfs
Options Reconfigured:
performance.io-thread-count: 32
performance.cache-size: 128MB
performance.write-behind-window-size: 128MB
server.allow-insecure: on
network.ping-timeout: 10
storage.owner-gid: 100
geo-replication.indexing: off
geo-replication.ignore-pid-check: on
changelog.changelog: on
changelog.fsync-interval: 3
changelog.rollover-time: 15
server.manage-gids: on


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


CCing Pranith, the maintainer of replicate. In the meantime can you 
please provide the logs from the machine where you have upgraded?


Regards,
Raghavendra Bhat
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.6.2 volume heal

[Pranith Kumar Karampuri]

On 02/03/2015 12:13 PM, Raghavendra Bhat wrote:

On Monday 02 February 2015 09:07 PM, David F. Robinson wrote:
I upgraded one of my bricks from 3.6.1 to 3.6.2 and I can no longer 
do a 'gluster volume heal homegfs info'.  It hangs and never returns 
any information.
I was trying to ensure that gfs01a had finished healing before 
upgrading the other machines (gfs01b, gfs02a, gfs02b) in my 
configuration (see below).

'gluster volume homegfs statistics' still works fine.
Do I need to upgrade my other bricks to get the 'gluster volume heal 
homegfs info' working?  Or, should I fix this issue before upgrading 
my other machines?

Volume Name: homegfs
Type: Distributed-Replicate
Volume ID: 1e32672a-f1b7-4b58-ba94-58c085e59071
Status: Started
Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: gfsib01a.corvidtec.com:/data/brick01a/homegfs
Brick2: gfsib01b.corvidtec.com:/data/brick01b/homegfs
Brick3: gfsib01a.corvidtec.com:/data/brick02a/homegfs
Brick4: gfsib01b.corvidtec.com:/data/brick02b/homegfs
Brick5: gfsib02a.corvidtec.com:/data/brick01a/homegfs
Brick6: gfsib02b.corvidtec.com:/data/brick01b/homegfs
Brick7: gfsib02a.corvidtec.com:/data/brick02a/homegfs
Brick8: gfsib02b.corvidtec.com:/data/brick02b/homegfs
Options Reconfigured:
performance.io-thread-count: 32
performance.cache-size: 128MB
performance.write-behind-window-size: 128MB
server.allow-insecure: on
network.ping-timeout: 10
storage.owner-gid: 100
geo-replication.indexing: off
geo-replication.ignore-pid-check: on
changelog.changelog: on
changelog.fsync-interval: 3
changelog.rollover-time: 15
server.manage-gids: on


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


CCing Pranith, the maintainer of replicate. In the meantime can you 
please provide the logs from the machine where you have upgraded?
Anuradha already followed up with David. Seems like he got out of this 
situation by upgrading the other node and removing some files which went 
into split-brain.
Next steps we are following up with David will be to check if the heal 
info command is run from 3.6.1 nodes or 3.6.2 nodes.


Pranith


Regards,
Raghavendra Bhat


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel