Re: [Gluster-devel] Leader Election Xlator Design Document

[Avra Sengupta]

With the exception of Rajesh Joseph, we have not received any review 
comments on the design. However inclined I am to believe that, we have 
such an extremely solid design that no one has any issues or concerns or 
queries regarding it, am afraid I have to assume otherwise.


So please review the design once, in terms of how the component you work 
on or own would be affected by it. How your component would interact 
with it or would want to use it, or wouldn't have anything to do with 
it, is something we need to hear about, especially at this juncture, or 
else your feedback at a later point in time, no matter how valuable will 
end up being too little too late.


Regards,
Avra

On 02/01/2017 04:52 PM, Avra Sengupta wrote:

Hi,

Leadership election is an integral part of server side replication, 
targeted for Gluster 4.0. The following is the design document for 
Leadership Election Xlator (LEX). It is a modular translator designed 
to work both with and independently of JBR. I would like to request 
you to kindly go through the same, and provide your feedback.


https://docs.google.com/document/d/1m7pLHKnzqUjcb3RQo8wxaRzENyxq1h1r385jnwUGc2A/edit?usp=sharing 



Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Leader Election Xlator Design Document

[Avra Sengupta]

Hi,

Leadership election is an integral part of server side replication, 
targeted for Gluster 4.0. The following is the design document for 
Leadership Election Xlator (LEX). It is a modular translator designed to 
work both with and independently of JBR. I would like to request you to 
kindly go through the same, and provide your feedback.


https://docs.google.com/document/d/1m7pLHKnzqUjcb3RQo8wxaRzENyxq1h1r385jnwUGc2A/edit?usp=sharing

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Invitation: Re: Question on merging zfs snapshot supp... @ Tue Dec 20, 2016 2:30pm - 3:30pm (IST) (sri...@marirs.net.in)

[Avra Sengupta]

Works fine for us Sriram. Friday 2-3 pm.

Regards,
Avra

On 01/12/2017 11:54 AM, sri...@marirs.net.in wrote:

Hi Avra,

Sorry for the late reply, could we have the meeting tomorrow? 2-3 pm?

Sriram


On Wed, Jan 11, 2017, at 11:58 AM, Avra Sengupta wrote:

Hi,

We can have a discussion tomorrow i.e 12th January from 3pm to 4 pm. 
Does that time work for you?


Meeting Link : https://bluejeans.com/u/asengupt/ 
<https://www.google.com/url?q=https%3A%2F%2Fbluejeans.com%2Fu%2Fasengupt%2F=D=2=AFQjCNHgp0xCwA9DqgdbAc9s2OxthUEHRA>


Regards,
Avra

On 01/10/2017 09:35 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hello Rajesh, Avra,

Could we have a discussion on the below? This week sometime?

Sriram


On Mon, Jan 2, 2017, at 04:56 PM, Rajesh Joseph wrote:

Sure, will setup it from next week onward.
-Rajesh

On Mon, Jan 2, 2017 at 4:38 PM, <sri...@marirs.net.in 
<mailto:sri...@marirs.net.in>> wrote:



Hi Rajesh,

Right now bi-weekly should be ok, with progress we could
decide. I'll continue to rework the initial patch set and post
it for review. We'll take it from there, is that ok with you?

Sriram


On Mon, Jan 2, 2017, at 03:32 PM, Rajesh Joseph wrote:



On Mon, Jan 2, 2017 at 3:19 PM, <sri...@marirs.net.in
<mailto:sri...@marirs.net.in>> wrote:


Hi Avra,

Is the below request ok with you?

Sriram


On Wed, Dec 21, 2016, at 10:00 AM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote:

Hi Avra/Rajesh,

In continuation to the discussion we'd yesterday, I'd be
working on the change we'd initiated sometime back for
pluggable FS specific snapshot implementation. We'd be
moving our gluster deployements to "master" (stable) once
this feature goes in. Since, glusterd2.0 release is
scheduled release next year, I'd be happy if some of the
work done here is re-usable to glusterd2.0 as well.

Let me know, if this is ok. Like Rajesh mentioned in the
call, could we've a weekly meeting for the same feature?



Hi Sriram,
I was on vacation so could not reply to your mail.
I am OK with having a regular sync-up on this issue. Let's
take this to conclusion.
Do we need a weekly meeting or bi-weekly meeting is fine?
Best Regards,
Rajesh





Sriram

On Mon, Dec 19, 2016, at 03:55 PM, aseng...@redhat.com
<mailto:aseng...@redhat.com> wrote:



more details »

<https://www.google.com/calendar/event?action=VIEW=YWJoMmtwNHAzc3EyNDgybTRjb2llaW1jNm8gc3JpcmFtQG1hcmlycy5uZXQuaW4=MTkjYXNlbmd1cHRAcmVkaGF0LmNvbTYyNWZlYjFmYzg2NWRkNGI2YzAyY2FlYmVkMTIwM2VlZmMxZTY0Mzg=Asia/Calcutta=en>


  Re: [Gluster-devel] Question on merging zfs
  snapshot support into the mainline glusterfs

Hi Sriram,

Could you please join the hangout, so that we can
discuss snapshot plugabbility. Thanks

Meeting Link: https://bluejeans.com/u/asengupt/

<https://www.google.com/url?q=https%3A%2F%2Fbluejeans.com%2Fu%2Fasengupt%2F=D=2=AFQjCNHgp0xCwA9DqgdbAc9s2OxthUEHRA>


Regards, Avra

On 12/19/2016 01:38 PM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote: > Hi Avra, > >
Could you help on the below request? May I abandon the
previous submitted patches, and could we consider the
latest one? > > Sriram > > > On Thu, Dec 15, 2016, at
12:57 PM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote: >> Hi Avra, >> >>
Thanks for the reply, >> >> But the problem I see here
is the previous patch set sent would'nt compile
individually. So, I merged the changes into a single
patch , which i'd posted today. Is it ok to drop all the
previous posted patches and consider from the new one?
Please suggest. >> >> Sriram >> >> >> On Thu, Dec 15,
2016, at 12:45 PM, Avra Sengupta wrote: >>> Hi Sriram,
>>> >>> I have already provided comments on the new
patch. It seems this new patch while addressing merge
cloflicts, has undone some previous patches. I suggest
you send this patch on top of the previous
patchset(http://review.gluster.org/#/c/15554/1

<https://www.google.com/url?q=http%3A%2F%2Freview.gluster.org%2F%23%2Fc%2F15554%2F1=D=2=AFQjCNE4gL3TlKKImxMOU_yOKoCFnP27BA>)
instead of creating a new one. This will allow you to
view the diff between the new version and the previous
version, and will give u an idea if the diff is
something that you added in the patch or got added as
part of merge conflict. >>> >>> Regards, >>> Avra >>>

Re: [Gluster-devel] Invitation: Re: Question on merging zfs snapshot supp... @ Tue Dec 20, 2016 2:30pm - 3:30pm (IST) (sri...@marirs.net.in)

[Avra Sengupta]

Hi,

We can have a discussion tomorrow i.e 12th January from 3pm to 4 pm. 
Does that time work for you?


Meeting Link : https://bluejeans.com/u/asengupt/ 
<https://www.google.com/url?q=https%3A%2F%2Fbluejeans.com%2Fu%2Fasengupt%2F=D=2=AFQjCNHgp0xCwA9DqgdbAc9s2OxthUEHRA>


Regards,
Avra

On 01/10/2017 09:35 PM, sri...@marirs.net.in wrote:

Hello Rajesh, Avra,

Could we have a discussion on the below? This week sometime?

Sriram


On Mon, Jan 2, 2017, at 04:56 PM, Rajesh Joseph wrote:

Sure, will setup it from next week onward.
-Rajesh

On Mon, Jan 2, 2017 at 4:38 PM, <sri...@marirs.net.in 
<mailto:sri...@marirs.net.in>> wrote:



Hi Rajesh,

Right now bi-weekly should be ok, with progress we could decide.
I'll continue to rework the initial patch set and post it for
review. We'll take it from there, is that ok with you?

Sriram


On Mon, Jan 2, 2017, at 03:32 PM, Rajesh Joseph wrote:



On Mon, Jan 2, 2017 at 3:19 PM, <sri...@marirs.net.in
<mailto:sri...@marirs.net.in>> wrote:


Hi Avra,

Is the below request ok with you?

Sriram


On Wed, Dec 21, 2016, at 10:00 AM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote:

Hi Avra/Rajesh,

In continuation to the discussion we'd yesterday, I'd be
working on the change we'd initiated sometime back for
pluggable FS specific snapshot implementation. We'd be
moving our gluster deployements to "master" (stable) once
this feature goes in. Since, glusterd2.0 release is
scheduled release next year, I'd be happy if some of the
work done here is re-usable to glusterd2.0 as well.

Let me know, if this is ok. Like Rajesh mentioned in the
call, could we've a weekly meeting for the same feature?



Hi Sriram,
I was on vacation so could not reply to your mail.
I am OK with having a regular sync-up on this issue. Let's take
this to conclusion.
Do we need a weekly meeting or bi-weekly meeting is fine?
Best Regards,
Rajesh





Sriram

On Mon, Dec 19, 2016, at 03:55 PM, aseng...@redhat.com
<mailto:aseng...@redhat.com> wrote:



more details »

<https://www.google.com/calendar/event?action=VIEW=YWJoMmtwNHAzc3EyNDgybTRjb2llaW1jNm8gc3JpcmFtQG1hcmlycy5uZXQuaW4=MTkjYXNlbmd1cHRAcmVkaGF0LmNvbTYyNWZlYjFmYzg2NWRkNGI2YzAyY2FlYmVkMTIwM2VlZmMxZTY0Mzg=Asia/Calcutta=en>


  Re: [Gluster-devel] Question on merging zfs snapshot
  support into the mainline glusterfs

Hi Sriram,

Could you please join the hangout, so that we can discuss
snapshot plugabbility. Thanks

Meeting Link: https://bluejeans.com/u/asengupt/

<https://www.google.com/url?q=https%3A%2F%2Fbluejeans.com%2Fu%2Fasengupt%2F=D=2=AFQjCNHgp0xCwA9DqgdbAc9s2OxthUEHRA>


Regards, Avra

On 12/19/2016 01:38 PM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote: > Hi Avra, > > Could
you help on the below request? May I abandon the previous
submitted patches, and could we consider the latest one? >
> Sriram > > > On Thu, Dec 15, 2016, at 12:57 PM,
sri...@marirs.net.in <mailto:sri...@marirs.net.in> wrote:
>> Hi Avra, >> >> Thanks for the reply, >> >> But the
problem I see here is the previous patch set sent would'nt
compile individually. So, I merged the changes into a
single patch , which i'd posted today. Is it ok to drop
all the previous posted patches and consider from the new
one? Please suggest. >> >> Sriram >> >> >> On Thu, Dec 15,
2016, at 12:45 PM, Avra Sengupta wrote: >>> Hi Sriram, >>>
>>> I have already provided comments on the new patch. It
seems this new patch while addressing merge cloflicts, has
undone some previous patches. I suggest you send this
patch on top of the previous
patchset(http://review.gluster.org/#/c/15554/1

<https://www.google.com/url?q=http%3A%2F%2Freview.gluster.org%2F%23%2Fc%2F15554%2F1=D=2=AFQjCNE4gL3TlKKImxMOU_yOKoCFnP27BA>)
instead of creating a new one. This will allow you to view
the diff between the new version and the previous version,
and will give u an idea if the diff is something that you
added in the patch or got added as part of merge conflict.
>>> >>> Regards, >>> Avra >>> >>> On 12/15/2016 12:09 PM,
sri...@marirs.net.in <mailto:sri...@marirs.net.in> wrote:
>>>> Hi Avra, >>>> >>>> I've update the patch according to
the comments below. And created a single patch which does

[Gluster-devel] FDL and Reconciliation Overview and Walkthrough

[Avra Sengupta]

Hi,

Jeff gave us an overview and walkthrough of FDL and Reconciliation. The 
presentation can be viewed in the link below:


https://bluejeans.com/s/B8_U3/

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Avra Sengupta]

Hi Sriram,

I have already provided comments on the new patch. It seems this new 
patch while addressing merge cloflicts, has undone some previous 
patches. I suggest you send this patch on top of the previous 
patchset(http://review.gluster.org/#/c/15554/1) instead of creating a 
new one. This will allow you to view the diff between the new version 
and the previous version, and will give u an idea if the diff is 
something that you added in the patch or got added as part of merge 
conflict.


Regards,
Avra

On 12/15/2016 12:09 PM, sri...@marirs.net.in wrote:

Hi Avra,

I've update the patch according to the comments below. And created a 
single patch which does the initial modularization. Fixed the 
tab->space issue as well. I've raised a new review request for the 
same bug ID here:

http://review.gluster.org/#/c/16138/

Added, Rajesh and You as the reviewers, let me know if I need to do 
anything else.


Could you have a look and let me know?

(Sorry for the delay in creating this)

Sriram

On Thu, Oct 13, 2016, at 12:15 PM, Avra Sengupta wrote:

Hi Sriram,

The point I was trying to make is, that we want that each patch 
should compile by itself, and pass regression. So for that to happen, 
we need to consolidate these patches(the first three) into one patch, 
and have the necessary make file changes into that patch too.


http://review.gluster.org/#/c/15554/
http://review.gluster.org/#/c/1/
http://review.gluster.org/#/c/15556/

That will give us one single patch, that contains the changes of 
having the current code moved into separate files, and it should get 
compiled on it's own, and should pass regression. Also, we use 
spaces, and not tabs in the code. So we will need to get those 
changed too. Thanks.


Regards,
Avra

On 10/12/2016 10:46 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Could you let me know on the below request?

Sriram


On Tue, Oct 4, 2016, at 11:16 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

I checked the comment, the series of patches, (There are nine 
patches) for which I've posted for a review below. They've all the 
necessary makefiles to compile.


Would you want me to consolidate all'em and post them as a single 
patch? (I thought that would be a little confusing, since it'd 
changes with different intentions).


Sriram


On Mon, Oct 3, 2016, at 03:54 PM, Avra Sengupta wrote:

Hi Sriram,

I posted a comment into the first patch. It doesn't compile by 
itself. We need to update the respective makefiles to be able to 
compile it. Then we can introduce the tabular structure in the 
same patch to have the framework set for the zfs snapshots. Thanks.


Regards,
Avra

On 09/30/2016 10:24 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Could you have a look into the below request?

Sriram


On Fri, Sep 23, 2016, at 04:10 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Have submitted the patches for Modularizing snapshot,

https://bugzilla.redhat.com/show_bug.cgi?id=1377437

This is the patch set:

http://review.gluster.org/15554 This patch follows the 
discussion from the gluster-devel mail chain of, ...
http://review.gluster.org/1 Referring to bugID:1377437, 
Modularizing snapshot for plugin based modules.
http://review.gluster.org/15556 - This is third patch in the 
series for the bug=1377437
http://review.gluster.org/15557 [BugId:1377437][Patch4]: 
Refering to the bug ID,
http://review.gluster.org/15558 [BugId:1377437][Patch5]: 
Refering to the bug ID,
http://review.gluster.org/15559 [BugId:1377437][Patch6]: 
Refering to the bug ID,
http://review.gluster.org/15560 [BugId:1377437][Patch7]: 
Refering to the bug ID. * This patch has some minor ...
http://review.gluster.org/15561 [BugId:1377437][Patch8]: 
Refering to the bug ID, this commit has minor fixes ...
http://review.gluster.org/15562 [BugId:1377437][Patch9]: 
Refering to the bug ID, - Minor header file ...


Primarily, focused on moving lvm based implementation into 
plugins. Have spread the commits across nine patches, some of 
them are minors, except a couple of ones which does the real 
work. Others are minors. Followed this method since, it would be 
easy for a review (accept/reject). Let me know if there is 
something off the methods followed with gluster devel. Thanks


Sriram

On Mon, Sep 19, 2016, at 10:58 PM, Avra Sengupta wrote:

Hi Sriram,

I have created a bug for this 
(https://bugzilla.redhat.com/show_bug.cgi?id=1377437). The plan 
is that for the first patch as mentioned below, let's not 
meddle with the zfs code at all. What we are looking at is 
segregating the lvm based code as is today, from the management 
infrastructure (which is addressed in your patch), and creating 
a table based pluggable infra(refer to gd_svc_cli_actors[] in 
xlators/mgmt/glusterd/src/glusterd-handler.c and other similar 
tables in gluster code base to get a understanding of what 

[Gluster-devel] Release 3.10 feature proposal : Introduce force option in snapshot restore

[Avra Sengupta]

Hi,

Essentially a snapshot of a volume, which was taken when the volume had 
nfs.ganesha option enabled, has a export.conf file for it. This file 
will be restored to the Ganesha Export Directory, which has been moved 
to the shared storage. Currently we fail the snapshot restore in a 
scenario, where the snapshot has the said conf file, but the shared 
storage is not available. This behaviour is expected. However there is 
no option for the user to proceed with the snapshot at this point in time.


We will introduce a force option for snapshot restore, which will enable 
the user to restore this particular snapshot in the above explained 
scenario, thereby abandoning the export.conf. The reason for introducing 
the force option is to make the user explicitly ask for the saved 
export.conf to be abandoned in such a scenario.


Example of proposed option in restore cli:
gluster snapshot restore [force]


Here is the BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1403188 



Design doc: http://review.gluster.org/#/c/16097/ 



Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Avra Sengupta]

Hi Sriram,

The point I was trying to make is, that we want that each patch should 
compile by itself, and pass regression. So for that to happen, we need 
to consolidate these patches(the first three) into one patch, and have 
the necessary make file changes into that patch too.


http://review.gluster.org/#/c/15554/
http://review.gluster.org/#/c/1/
http://review.gluster.org/#/c/15556/

That will give us one single patch, that contains the changes of having 
the current code moved into separate files, and it should get compiled 
on it's own, and should pass regression. Also, we use spaces, and not 
tabs in the code. So we will need to get those changed too. Thanks.


Regards,
Avra

On 10/12/2016 10:46 PM, sri...@marirs.net.in wrote:

Hi Avra,

Could you let me know on the below request?

Sriram


On Tue, Oct 4, 2016, at 11:16 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

I checked the comment, the series of patches, (There are nine 
patches) for which I've posted for a review below. They've all the 
necessary makefiles to compile.


Would you want me to consolidate all'em and post them as a single 
patch? (I thought that would be a little confusing, since it'd 
changes with different intentions).


Sriram


On Mon, Oct 3, 2016, at 03:54 PM, Avra Sengupta wrote:

Hi Sriram,

I posted a comment into the first patch. It doesn't compile by 
itself. We need to update the respective makefiles to be able to 
compile it. Then we can introduce the tabular structure in the same 
patch to have the framework set for the zfs snapshots. Thanks.


Regards,
Avra

On 09/30/2016 10:24 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Could you have a look into the below request?

Sriram


On Fri, Sep 23, 2016, at 04:10 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Have submitted the patches for Modularizing snapshot,

https://bugzilla.redhat.com/show_bug.cgi?id=1377437

This is the patch set:

http://review.gluster.org/15554 This patch follows the discussion 
from the gluster-devel mail chain of, ...
http://review.gluster.org/1 Referring to bugID:1377437, 
Modularizing snapshot for plugin based modules.
http://review.gluster.org/15556 - This is third patch in the 
series for the bug=1377437
http://review.gluster.org/15557 [BugId:1377437][Patch4]: Refering 
to the bug ID,
http://review.gluster.org/15558 [BugId:1377437][Patch5]: Refering 
to the bug ID,
http://review.gluster.org/15559 [BugId:1377437][Patch6]: Refering 
to the bug ID,
http://review.gluster.org/15560 [BugId:1377437][Patch7]: Refering 
to the bug ID. * This patch has some minor ...
http://review.gluster.org/15561 [BugId:1377437][Patch8]: Refering 
to the bug ID, this commit has minor fixes ...
http://review.gluster.org/15562 [BugId:1377437][Patch9]: Refering 
to the bug ID, - Minor header file ...


Primarily, focused on moving lvm based implementation into 
plugins. Have spread the commits across nine patches, some of them 
are minors, except a couple of ones which does the real work. 
Others are minors. Followed this method since, it would be easy 
for a review (accept/reject). Let me know if there is something 
off the methods followed with gluster devel. Thanks


Sriram

On Mon, Sep 19, 2016, at 10:58 PM, Avra Sengupta wrote:

Hi Sriram,

I have created a bug for this 
(https://bugzilla.redhat.com/show_bug.cgi?id=1377437). The plan 
is that for the first patch as mentioned below, let's not meddle 
with the zfs code at all. What we are looking at is segregating 
the lvm based code as is today, from the management 
infrastructure (which is addressed in your patch), and creating a 
table based pluggable infra(refer to gd_svc_cli_actors[] in 
xlators/mgmt/glusterd/src/glusterd-handler.c and other similar 
tables in gluster code base to get a understanding of what I am 
conveying), which can be used to call this code and still achieve 
the same results as we do today.


Once this code is merged, we can use the same infra to start 
pushing in the zfs code (rest of your current patch). Please let 
me know if you have further queries regarding this. Thanks.


Regards,
Avra

On 09/19/2016 07:52 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Do you have a bug id for this changes? Or may I raise a new one?

Sriram


On Fri, Sep 16, 2016, at 11:37 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Thanks Avra,

I'll send this patch to gluster master in a while.

Sriram


On Wed, Sep 14, 2016, at 03:08 PM, Avra Sengupta wrote:

Hi Sriram,

Sorry for the delay in response. I started going through the 
commits in the github repo. I finished going through the first 
commit, where you create a plugin structure and move code. 
Following is the commit link:


https://github.com/sriramster/glusterfs/commit/7bf157525539541ebf0aa36a380bbedb2cae5440

FIrst of all, the overall approach of using plugins, and 
maintaining plu

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Avra Sengupta]

Hi Sriram,

I posted a comment into the first patch. It doesn't compile by itself. 
We need to update the respective makefiles to be able to compile it. 
Then we can introduce the tabular structure in the same patch to have 
the framework set for the zfs snapshots. Thanks.


Regards,
Avra

On 09/30/2016 10:24 AM, sri...@marirs.net.in wrote:

Hi Avra,

Could you have a look into the below request?

Sriram


On Fri, Sep 23, 2016, at 04:10 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Have submitted the patches for Modularizing snapshot,

https://bugzilla.redhat.com/show_bug.cgi?id=1377437

This is the patch set:

http://review.gluster.org/15554 This patch follows the discussion 
from the gluster-devel mail chain of, ...
http://review.gluster.org/1 Referring to bugID:1377437, 
Modularizing snapshot for plugin based modules.
http://review.gluster.org/15556 - This is third patch in the series 
for the bug=1377437
http://review.gluster.org/15557 [BugId:1377437][Patch4]: Refering to 
the bug ID,
http://review.gluster.org/15558 [BugId:1377437][Patch5]: Refering to 
the bug ID,
http://review.gluster.org/15559 [BugId:1377437][Patch6]: Refering to 
the bug ID,
http://review.gluster.org/15560 [BugId:1377437][Patch7]: Refering to 
the bug ID. * This patch has some minor ...
http://review.gluster.org/15561 [BugId:1377437][Patch8]: Refering to 
the bug ID, this commit has minor fixes ...
http://review.gluster.org/15562 [BugId:1377437][Patch9]: Refering to 
the bug ID, - Minor header file ...


Primarily, focused on moving lvm based implementation into plugins. 
Have spread the commits across nine patches, some of them are minors, 
except a couple of ones which does the real work. Others are minors. 
Followed this method since, it would be easy for a review 
(accept/reject). Let me know if there is something off the methods 
followed with gluster devel. Thanks


Sriram

On Mon, Sep 19, 2016, at 10:58 PM, Avra Sengupta wrote:

Hi Sriram,

I have created a bug for this 
(https://bugzilla.redhat.com/show_bug.cgi?id=1377437). The plan is 
that for the first patch as mentioned below, let's not meddle with 
the zfs code at all. What we are looking at is segregating the lvm 
based code as is today, from the management infrastructure (which is 
addressed in your patch), and creating a table based pluggable 
infra(refer to gd_svc_cli_actors[] in 
xlators/mgmt/glusterd/src/glusterd-handler.c and other similar 
tables in gluster code base to get a understanding of what I am 
conveying), which can be used to call this code and still achieve 
the same results as we do today.


Once this code is merged, we can use the same infra to start pushing 
in the zfs code (rest of your current patch). Please let me know if 
you have further queries regarding this. Thanks.


Regards,
Avra

On 09/19/2016 07:52 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Do you have a bug id for this changes? Or may I raise a new one?

Sriram


On Fri, Sep 16, 2016, at 11:37 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Thanks Avra,

I'll send this patch to gluster master in a while.

Sriram


On Wed, Sep 14, 2016, at 03:08 PM, Avra Sengupta wrote:

Hi Sriram,

Sorry for the delay in response. I started going through the 
commits in the github repo. I finished going through the first 
commit, where you create a plugin structure and move code. 
Following is the commit link:


https://github.com/sriramster/glusterfs/commit/7bf157525539541ebf0aa36a380bbedb2cae5440

FIrst of all, the overall approach of using plugins, and 
maintaining plugins that is used in the patch is in sync with 
what we had discussed. There are some gaps though, like in the 
zfs functions the snap brick is mounted without updating labels, 
and in restore you perform a zfs rollback, which significantly 
changes the behavior between how a lvm based snapshot and a zfs 
based snapshot.


But before we get into these details, I would request you to 
kindly send this particular patch to the gluster master branch, 
as that is how we formally review patches, and I would say this 
particular patch in itself is ready for a formal review. Once we 
straighten out the quirks in this patch, we can significantly 
start moving the other dependent patches to master and reviewing 
them. Thanks.


Regards,
Avra

P.S : Adding gluster-devel

On 09/13/2016 01:14 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

You'd time to look into the below request?

Sriram


On Thu, Sep 8, 2016, at 01:20 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Thank you. Please, let me know your feedback. It would be 
helpful on continuing from then.


Sriram


On Thu, Sep 8, 2016, at 01:18 PM, Avra Sengupta wrote:

Hi Sriram,

Rajesh is on a vacation, and will be available towards the end 
of next week. He will be sharing his feedback once he is back. 
Meanwhile I will have a 

Re: [Gluster-devel] Multiplexing - good news, bad news, and a plea for help

[Avra Sengupta]

On 09/19/2016 11:34 PM, Jeff Darcy wrote:

I would like to collaborate in investigating the memory-management, and
also bringing multiplexing to snapshots. For starters, will be going
through your patch(1400+ lines of change, that's one big ass patch :p)

That's nothing.  I've seen 7000-line patches go in, without even any
evidence of testing.  ;)

But seriously, 1400 lines is a lot to digest.  Especially when many of
those lines are scattered in dark corners of the glusterfsd code.  If
you or anyone else would like, I could schedule a walk-through to go
over some of the less obvious pieces.


I think that would be great, and would certainly get things moving on a 
faster pace.

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Multiplexing - good news, bad news, and a plea for help

[Avra Sengupta]

On 09/19/2016 06:56 PM, Jeff Darcy wrote:

I have brick multiplexing[1] functional to the point that it passes all basic 
AFR, EC, and quota tests.  There are still some issues with tiering, and I 
wouldn't consider snapshots functional at all, but it seemed like a good point 
to see how well it works.  I ran some *very simple* tests with 20 volumes, each 
2x distribute on top of 2x replicate.

First, the good news: it worked!  Getting 80 bricks to come up in the same 
process, and then run I/O correctly across all of those, is pretty cool.  Also, 
memory consumption is *way* down.  RSS size went from 1.1GB before (total 
across 80 processes) to about 400MB (one process) with multiplexing.  Each 
process seems to consume approximately 8MB globally plus 5MB per brick, so 
(8+5)*80 = 1040 vs. 8+(5*80) = 408.  Just considering the amount of memory, 
this means we could support about three times as many bricks as before.  When 
memory *contention* is considered, the difference is likely to be even greater.

Bad news: some of our code doesn't scale very well in terms of CPU use.  To 
test performance I ran a test which would create 20,000 files across all 20 
volumes, then write and delete them, all using 100 client threads.  This is 
similar to what smallfile does, but deliberately constructed to use a minimum 
of disk space - at any given, only one file per thread (maximum) actually has 
4KB worth of data in it.  This allows me to run it against SSDs or even 
ramdisks even with high brick counts, to factor out slow disks in a study of 
CPU/memory issues.  Here are some results and observations.

* On my first run, the multiplexed version of the test took 77% longer to run 
than the non-multiplexed version (5:42 vs. 3:13).  And that was after I'd done 
some hacking to use 16 epoll threads.  There's something a bit broken about 
trying to set that option normally, so that the value you set doesn't actually 
make it to the place that tries to spawn the threads.  Bumping this up further 
to 32 threads didn't seem to help.

* A little profiling showed me that we're spending almost all of our time in 
pthread_spin_lock.  I disabled the code to use spinlocks instead of regular 
mutexes, which immediately improved performance and also reduced CPU time by 
almost 50%.

* The next round of profiling showed that a lot of the locking is in mem-pool 
code, and a lot of that in turn is from dictionary code.  Changing the dict 
code to use malloc/free instead of mem_get/mem_put gave another noticeable 
boost.

At this point run time was down to 4:50, which is 20% better than where I 
started but still far short of non-multiplexed performance.  I can drive that 
down still further by converting more things to use malloc/free.  There seems 
to be a significant opportunity here to improve performance - even without 
multiplexing - by taking a more careful look at our memory-management 
strategies:

* Tune the mem-pool implementation to scale better with hundreds of threads.

* Use mem-pools more selectively, or even abandon them altogether.

* Try a different memory allocator such as jemalloc.

I'd certainly appreciate some help/collaboration in studying these options 
further.  It's a great opportunity to make a large impact on overall 
performance without a lot of code or specialized knowledge.  Even so, however, 
I don't think memory management is our only internal scalability problem.  
There must be something else limiting parallelism, and quite severely at that.  
My first guess is io-threads, so I'll be looking into that first, but if 
anybody else has any ideas please let me know.  There's no *good* reason why 
running many bricks in one process should be slower than running them in 
separate processes.  If it remains slower, then the limit on the number of 
bricks and volumes we can support will remain unreasonably low.  Also, the 
problems I'm seeing here probably don't *only* affect multiplexing.  Excessive 
memory/CPU use and poor parallelism are issues that we kind of need to address 
anyway, so if anybody has any ideas please let me know.
I would like to collaborate in investigating the memory-management, and 
also bringing multiplexing to snapshots. For starters, will be going 
through your patch(1400+ lines of change, that's one big ass patch :p) 
and seeing how it aligns as far as snapshots go. Thanks.


Regards,
Avra




[1] http://review.gluster.org/#/c/14763/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Avra Sengupta]

Hi Sriram,

I have created a bug for this 
(https://bugzilla.redhat.com/show_bug.cgi?id=1377437). The plan is that 
for the first patch as mentioned below, let's not meddle with the zfs 
code at all. What we are looking at is segregating the lvm based code as 
is today, from the management infrastructure (which is addressed in your 
patch), and creating a table based pluggable infra(refer to 
gd_svc_cli_actors[] in xlators/mgmt/glusterd/src/glusterd-handler.c and 
other similar tables in gluster code base to get a understanding of what 
I am conveying), which can be used to call this code and still achieve 
the same results as we do today.


Once this code is merged, we can use the same infra to start pushing in 
the zfs code (rest of your current patch). Please let me know if you 
have further queries regarding this. Thanks.


Regards,
Avra

On 09/19/2016 07:52 PM, sri...@marirs.net.in wrote:

Hi Avra,

Do you have a bug id for this changes? Or may I raise a new one?

Sriram


On Fri, Sep 16, 2016, at 11:37 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Thanks Avra,

I'll send this patch to gluster master in a while.

Sriram


On Wed, Sep 14, 2016, at 03:08 PM, Avra Sengupta wrote:

Hi Sriram,

Sorry for the delay in response. I started going through the commits 
in the github repo. I finished going through the first commit, where 
you create a plugin structure and move code. Following is the commit 
link:


https://github.com/sriramster/glusterfs/commit/7bf157525539541ebf0aa36a380bbedb2cae5440

FIrst of all, the overall approach of using plugins, and maintaining 
plugins that is used in the patch is in sync with what we had 
discussed. There are some gaps though, like in the zfs functions the 
snap brick is mounted without updating labels, and in restore you 
perform a zfs rollback, which significantly changes the behavior 
between how a lvm based snapshot and a zfs based snapshot.


But before we get into these details, I would request you to kindly 
send this particular patch to the gluster master branch, as that is 
how we formally review patches, and I would say this particular 
patch in itself is ready for a formal review. Once we straighten out 
the quirks in this patch, we can significantly start moving the 
other dependent patches to master and reviewing them. Thanks.


Regards,
Avra

P.S : Adding gluster-devel

On 09/13/2016 01:14 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

You'd time to look into the below request?

Sriram


On Thu, Sep 8, 2016, at 01:20 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Thank you. Please, let me know your feedback. It would be helpful 
on continuing from then.


Sriram


On Thu, Sep 8, 2016, at 01:18 PM, Avra Sengupta wrote:

Hi Sriram,

Rajesh is on a vacation, and will be available towards the end of 
next week. He will be sharing his feedback once he is back. 
Meanwhile I will have a look at the patch and share my feedback 
with you. But it will take me some time to go through it. Thanks.


Regards,
Avra

On 09/08/2016 01:09 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hello Rajesh,

Sorry to bother. Could you have a look at the below request?

Sriram


On Tue, Sep 6, 2016, at 11:27 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hello Rajesh,

Sorry for the delayed mail, was on leave. Could you let me know 
the feedback?


Sriram


On Fri, Sep 2, 2016, at 10:08 AM, Rajesh Joseph wrote:

+ Avra
Hi Srirram,

Sorry, I was on leave therefore could not reply.
Added Avra who is also working on the snapshot component for 
review.

Will take a look at your changes today.
Thanks & Regards,
Rajesh


On Thu, Sep 1, 2016 at 1:22 PM, <sri...@marirs.net.in 
<mailto:sri...@marirs.net.in>> wrote:



Hello Rajesh,

Could you've a look at the below request?

Sriram

On Tue, Aug 30, 2016, at 01:03 PM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote:

Hi Rajesh,

Continuing from the discussion we've had below and
suggestions made by you, had created a plugin like
structure (A generic plugin model) and added snapshot to
be the first plugin implementation. Could you've a look
if the approach is fine? I've not raised a official
review request yet. Could you give an initial review of
the model?

https://github.com/sriramster/glusterfs/tree/sriram_dev

Things done,

- Created a new folder for glusterd plugins and added
snapshot as a plugin. Like this,

$ROOT/xlators/mgmt/glusterd/plugins +
|
+ __ snapshot/src

Moved LVM related snapshot implementation to
xlators/mgmt/glusterd/plugins/snapshot/src/lvm-snapshot.c

- Mostly isolated, glusterd code from snapshot
implementation by using logging, error codes and messages
from glusterd and libglusterfs.
- This way, i though we could get complete isolation of

[Gluster-devel] make install again compiling source

[Avra Sengupta]

Hi,

I ran "make -j" on the latest master, followed by make install. The make 
install, by itself is doing a fresh compile every time (and totally 
ignoring the make i did before it). Is there any recent change, which 
would cause this. Thanks.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Running glusterd with valgrind

[Avra Sengupta]

Hi Samikshan,

I found the crash on RHEL 6.5, and it's constantly reproducible on my 
setup.


Regards,
Avra

On 09/15/2016 06:21 PM, Samikshan Bairagya wrote:



On 09/15/2016 03:48 PM, Avra Sengupta wrote:

Hi,

I was trying to run valgrind with glusterd using the following command:

valgrind --leak-check=full --log-file=/tmp/glusterd.log glusterd

This command used to work before, rather seamlessly but now glusterd
crashes with the following bt:

/usr/local/lib/libglusterfs.so.0(_gf_msg_backtrace_nomem+0x9c)[0x4c3772c] 


/usr/local/lib/libglusterfs.so.0(gf_print_trace+0x314)[0x4c426f4]
/lib64/libc.so.6[0x3344a329a0]
/lib64/libc.so.6(gsignal+0x35)[0x3344a32925]
/lib64/libc.so.6(abort+0x175)[0x3344a34105]
/lib64/libc.so.6[0x3344a2ba4e]
/lib64/libc.so.6(__assert_perror_fail+0x0)[0x3344a2bb10]
/usr/lib64/liburcu-bp.so.1(rcu_bp_register+0x288)[0xf7974a8]
/usr/lib64/liburcu-bp.so.1(rcu_read_lock_bp+0x4e)[0xf79751e]
/usr/local/lib/glusterfs/3.10dev/xlator/mgmt/glusterd.so(+0x10e07e)[0xf52e07e] 



/usr/local/lib/glusterfs/3.10dev/xlator/mgmt/glusterd.so(+0x619cb)[0xf4819cb] 



/usr/local/lib/glusterfs/3.10dev/xlator/mgmt/glusterd.so(+0x62394)[0xf482394] 



/usr/local/lib/libglusterfs.so.0(synctask_wrap+0x10)[0x4c70a50]
/lib64/libc.so.6[0x3344a43bf0]

Is this a known issue? Is there another way to run gluster with 
valgrind.




Hey Avra,

Don't think its a known issue. I tried reproducing this on RHEL7 and 
Fedora24 machines with gluster built against current updated master; 
but I didn't hit this crash. Which distro are you getting this issue on?


Regards,
Samikshan



Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Running glusterd with valgrind

[Avra Sengupta]

Hi,

I was trying to run valgrind with glusterd using the following command:

valgrind --leak-check=full --log-file=/tmp/glusterd.log glusterd

This command used to work before, rather seamlessly but now glusterd 
crashes with the following bt:


/usr/local/lib/libglusterfs.so.0(_gf_msg_backtrace_nomem+0x9c)[0x4c3772c]
/usr/local/lib/libglusterfs.so.0(gf_print_trace+0x314)[0x4c426f4]
/lib64/libc.so.6[0x3344a329a0]
/lib64/libc.so.6(gsignal+0x35)[0x3344a32925]
/lib64/libc.so.6(abort+0x175)[0x3344a34105]
/lib64/libc.so.6[0x3344a2ba4e]
/lib64/libc.so.6(__assert_perror_fail+0x0)[0x3344a2bb10]
/usr/lib64/liburcu-bp.so.1(rcu_bp_register+0x288)[0xf7974a8]
/usr/lib64/liburcu-bp.so.1(rcu_read_lock_bp+0x4e)[0xf79751e]
/usr/local/lib/glusterfs/3.10dev/xlator/mgmt/glusterd.so(+0x10e07e)[0xf52e07e]
/usr/local/lib/glusterfs/3.10dev/xlator/mgmt/glusterd.so(+0x619cb)[0xf4819cb]
/usr/local/lib/glusterfs/3.10dev/xlator/mgmt/glusterd.so(+0x62394)[0xf482394]
/usr/local/lib/libglusterfs.so.0(synctask_wrap+0x10)[0x4c70a50]
/lib64/libc.so.6[0x3344a43bf0]

Is this a known issue? Is there another way to run gluster with valgrind.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Avra Sengupta]

Hi Sriram,

Sorry for the delay in response. I started going through the commits in 
the github repo. I finished going through the first commit, where you 
create a plugin structure and move code. Following is the commit link:


https://github.com/sriramster/glusterfs/commit/7bf157525539541ebf0aa36a380bbedb2cae5440

FIrst of all, the overall approach of using plugins, and maintaining 
plugins that is used in the patch is in sync with what we had discussed. 
There are some gaps though, like in the zfs functions the snap brick is 
mounted without updating labels, and in restore you perform a zfs 
rollback, which significantly changes the behavior between how a lvm 
based snapshot and a zfs based snapshot.


But before we get into these details, I would request you to kindly send 
this particular patch to the gluster master branch, as that is how we 
formally review patches, and I would say this particular patch in itself 
is ready for a formal review. Once we straighten out the quirks in this 
patch, we can significantly start moving the other dependent patches to 
master and reviewing them. Thanks.


Regards,
Avra

P.S : Adding gluster-devel

On 09/13/2016 01:14 AM, sri...@marirs.net.in wrote:

Hi Avra,

You'd time to look into the below request?

Sriram


On Thu, Sep 8, 2016, at 01:20 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hi Avra,

Thank you. Please, let me know your feedback. It would be helpful on 
continuing from then.


Sriram


On Thu, Sep 8, 2016, at 01:18 PM, Avra Sengupta wrote:

Hi Sriram,

Rajesh is on a vacation, and will be available towards the end of 
next week. He will be sharing his feedback once he is back. 
Meanwhile I will have a look at the patch and share my feedback with 
you. But it will take me some time to go through it. Thanks.


Regards,
Avra

On 09/08/2016 01:09 PM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hello Rajesh,

Sorry to bother. Could you have a look at the below request?

Sriram


On Tue, Sep 6, 2016, at 11:27 AM, sri...@marirs.net.in 
<mailto:sri...@marirs.net.in> wrote:

Hello Rajesh,

Sorry for the delayed mail, was on leave. Could you let me know 
the feedback?


Sriram


On Fri, Sep 2, 2016, at 10:08 AM, Rajesh Joseph wrote:

+ Avra
Hi Srirram,

Sorry, I was on leave therefore could not reply.
Added Avra who is also working on the snapshot component for review.
Will take a look at your changes today.
Thanks & Regards,
Rajesh


On Thu, Sep 1, 2016 at 1:22 PM, <sri...@marirs.net.in 
<mailto:sri...@marirs.net.in>> wrote:



Hello Rajesh,

Could you've a look at the below request?

Sriram

On Tue, Aug 30, 2016, at 01:03 PM, sri...@marirs.net.in
<mailto:sri...@marirs.net.in> wrote:

Hi Rajesh,

Continuing from the discussion we've had below and
suggestions made by you, had created a plugin like structure
(A generic plugin model) and added snapshot to be the first
plugin implementation. Could you've a look if the approach
is fine? I've not raised a official review request yet.
Could you give an initial review of the model?

https://github.com/sriramster/glusterfs/tree/sriram_dev
<https://github.com/sriramster/glusterfs/tree/sriram_dev>

Things done,

- Created a new folder for glusterd plugins and added
snapshot as a plugin. Like this,

$ROOT/xlators/mgmt/glusterd/plugins +
|
+ __ snapshot/src

Moved LVM related snapshot implementation to
xlators/mgmt/glusterd/plugins/snapshot/src/lvm-snapshot.c

- Mostly isolated, glusterd code from snapshot
implementation by using logging, error codes and messages
from glusterd and libglusterfs.
- This way, i though we could get complete isolation of
snapshot plugin implementation which avoids most of compiler
and linking dependency issues.
- Created a library of the above like libgsnapshot.so and
linking it with glusterd.so to get this working.
- The complete isolation also makes us to avoid reverse
dependency like some api's inside plugin/snapshot being
dependent on glusterd.so

TODO's :

- Need to create glusterd_snapshot_ops structure which would
be used to register snapshot related API's with glusterd.so.
- Add command line snapshot plugin option, so that it picks
up on compilation.
- If any missed implementation for plugin.
- Cleanup and get a review ready branch.

Let me know if this looks ok? Or need to any more into the
list.

Sriram

On Fri, Jul 22, 2016, at 02:43 PM, Rajesh Joseph wrote:



On Thu, Jul 21, 2016 at 3:07 AM, Vijay Bellur
<vbel...@redhat.com <mailto:vbel...@redhat.com>> wrote:

On 07/19/2016 11:01 AM, Atin Mukherjee wrote:



On Tue, Jul 19, 2016 at 7:29 PM, Rajesh Joseph
<rjos...@redhat.com <mailto:rjos...@redhat.com>
<mailto:rjos...@redhat.com
<mailto:rjo

Re: [Gluster-devel] Checklist for snapshot component for upstream release

[Avra Sengupta]

Hi Pranith,

The following set of automated and manual tests need to pass before 
doing a release for snapshot component:
1. The entire snapshot regression suite present in the source 
repository, which as of now consist of:


   a. ./basic/volume-snapshot.t
   b. ./basic/volume-snapshot-clone.t
   c. ./basic/volume-snapshot-xml.t
   d. All tests present in ./bugs/snapshot

2. Manual test of using snapshot scheduler.
3. Till the eventing test framework is integrated with the regression 
suite, manual test of all 28 snapshot events.


Regards,
Avra

On 09/03/2016 12:26 AM, Pranith Kumar Karampuri wrote:

hi,
Did you get a chance to decide on the tests that need to be 
done before doing a release for snapshot component? Could you let me 
know who will be providing with the list?


I can update it at 
https://public.pad.fsfe.org/p/gluster-component-release-checklist 



--
Aravinda & Pranith


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] CFP Gluster Developer Summit

[Avra Sengupta]

Hi,

I would like to propose the following talks:

1. Ttitle: Server side replication
Theme: Gluster.Next
Abstract: Achieving availability and correctness without 
compromising performance, is always a challenge in designing new storage 
replication technologies. In this talk I would talk about the following:


 * Challenges faced by client side replication
 * How we can address these challenges with server side replication
 * Use cases for both server side and client side replication

2. Title: Scalability Issues and probable solutions
Theme: Stability and performance
 Abstract: Scalability in terms of number of nodes, and number of 
volumes in a cluster as of today is something I plan to explore in this 
talk. I would like to discuss the numbers we reach today, and probable 
solutions to get to better numbers. I would talk about the following:


 * Bottlenecks in scalability due to Glusterd and probable solutions
 * Scalability issues due to snapshot bricks' resource consumption and
   probable solutions


Regards,
Avra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] mount_dir value seems clobbered in all /var/lib/glusterd/vols//bricks/: files

[Avra Sengupta]

In that case it is expected behaviour. Thanks.

Regards,
Avra

On 08/05/2016 11:36 AM, Milind Changire wrote:

The bricks are NOT lvm mounted.
The bricks are just directories on the root file-system.

Milind

On 08/05/2016 11:25 AM, Avra Sengupta wrote:

Hi Milind,

Are the bricks lvm mounterd bricks. This field is populated for lvm
mounted bricks, and used by them. For regular bricks, which don't have a
mount point this valus is ignored.

Regards,
Avra

On 08/04/2016 07:44 PM, Atin Mukherjee wrote:

glusterd_get_brick_mount_dir () does a brick_dir++  which seems to
cause this problem and removing this line fixes the problem. Commit
f846e54b introduced it.

Ccing Avra/Rajesh

mount_dir is used by snapshot, however I am just wondering how are we
surviving this case.

~Atin

On Thu, Aug 4, 2016 at 5:39 PM, Milind Changire <mchan...@redhat.com
<mailto:mchan...@redhat.com>> wrote:

here's one of the brick definition files for a volume named 
"twoXtwo"


[root@f24node0 bricks]# cat f24node1\:-glustervols-twoXtwo-dir
hostname=f24node1
path=/glustervols/twoXtwo/dir
real_path=/glustervols/twoXtwo/dir
listen-port=0
rdma.listen-port=0
decommissioned=0
brick-id=twoXtwo-client-1
mount_dir=/lustervols/twoXtwo/dir  <-- shouldn't the 
value be

/glustervols/...
   there's a missing 
'g'

   after the first '/'
snap-status=0


This *should* happen for all volumes and for all such brick 
definition

files or whatever they are called.
BTW, I'm working with the upstream mainline sources, if that helps.

I'm running a 2x2 distribute-replicate volume.
4 nodes with 1 brick per node.
1 brick for the hot tier for tiering.

As far as I can tell, I haven't done anything fancy with the setup.
And I have confirmed that there is no directory named '/lustervols'
on any of my cluster nodes.

--
Milind
___
Gluster-devel mailing list
Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-devel




--

--Atin




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] mount_dir value seems clobbered in all /var/lib/glusterd/vols//bricks/: files

[Avra Sengupta]

Hi Milind,

Are the bricks lvm mounterd bricks. This field is populated for lvm 
mounted bricks, and used by them. For regular bricks, which don't have a 
mount point this valus is ignored.


Regards,
Avra

On 08/04/2016 07:44 PM, Atin Mukherjee wrote:
glusterd_get_brick_mount_dir () does a brick_dir++ which seems to 
cause this problem and removing this line fixes the problem. Commit 
f846e54b introduced it.


Ccing Avra/Rajesh

mount_dir is used by snapshot, however I am just wondering how are we 
surviving this case.


~Atin

On Thu, Aug 4, 2016 at 5:39 PM, Milind Changire > wrote:


here's one of the brick definition files for a volume named "twoXtwo"

[root@f24node0 bricks]# cat f24node1\:-glustervols-twoXtwo-dir
hostname=f24node1
path=/glustervols/twoXtwo/dir
real_path=/glustervols/twoXtwo/dir
listen-port=0
rdma.listen-port=0
decommissioned=0
brick-id=twoXtwo-client-1
mount_dir=/lustervols/twoXtwo/dir  <-- shouldn't the value be
 /glustervols/...
   there's a missing 'g'
   after the first '/'
snap-status=0


This *should* happen for all volumes and for all such brick definition
files or whatever they are called.
BTW, I'm working with the upstream mainline sources, if that helps.

I'm running a 2x2 distribute-replicate volume.
4 nodes with 1 brick per node.
1 brick for the hot tier for tiering.

As far as I can tell, I haven't done anything fancy with the setup.
And I have confirmed that there is no directory named '/lustervols'
on any of my cluster nodes.

-- 
Milind

___
Gluster-devel mailing list
Gluster-devel@gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-devel




--

--Atin


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] regression failed : snapshot/bug-1316437.t

[Avra Sengupta]

Had a look at the patch. What you are trying to do is, to re-use the 
port and if not successfult, you are getting a new port. I have some 
comments in the patch, but to me this looks mostly fine.


On 07/25/2016 07:14 PM, Atin Mukherjee wrote:



On Mon, Jul 25, 2016 at 7:12 PM, Atin Mukherjee <amukh...@redhat.com 
<mailto:amukh...@redhat.com>> wrote:




On Mon, Jul 25, 2016 at 5:37 PM, Atin Mukherjee
<amukh...@redhat.com <mailto:amukh...@redhat.com>> wrote:



On Mon, Jul 25, 2016 at 4:34 PM, Avra Sengupta
<aseng...@redhat.com <mailto:aseng...@redhat.com>> wrote:

The crux of the problem is that as of today, brick
processes on restart try to reuse the old port they were
using (assuming that no other process will be using it,
and not consulting pmap_registry_alloc() before using it).
With a recent change, pmap_registry_alloc (), reassigns
older ports that were used, but are now free. Hence snapd
now gets a port that was previously used by a brick and
tries to bind to it, whereas the older brick process
without consulting pmap table blindly tries to connect to
it, and hence we see this problem.

Now coming to the fix, I feel brick process should not try
to get the older port and should just take a new port
every time it comes up. We will not run out of ports with
this change coz, now pmap allocates old ports again, and
the previous port being used by the brick process will
eventually be reused. If anyone sees any concern with this
approach, please feel free to raise so now.


Looks to be OK, but I'll think through it and get back to you
by a day or two if I have any objections.


If we are conservative about bricks not binding to a different
port on a restart, I've an alternative approach here [1] . Neither
it has a full fledged commit message nor a BZ. I've just put this
up for your input?


Read it as "binding" instead "not binding"


[1] http://review.gluster.org/15005



While awaiting feedback from you guys, I have sent this
patch (http://review.gluster.org/15001), which moves the
said test case to bad tests for now, and after we
collectively reach to a conclusion on the fix, we will
remove this from bad test.

Regards,
    Avra


On 07/25/2016 02:33 PM, Avra Sengupta wrote:

The failure suggests that the port snapd is trying to
bind to is already in use. But snapd has been modified to
use a new port everytime. I am looking into this.

On 07/25/2016 02:23 PM, Nithya Balachandran wrote:

More failures:

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22452/console

I see these messages in the snapd.log:

[2016-07-22 05:31:52.482282] I
[rpcsvc.c:2199:rpcsvc_set_outstanding_rpc_limit]
0-rpc-service: Configured rpc.outstanding-rpc-limit with
value 64
[2016-07-22 05:31:52.482352] W [MSGID: 101002]
[options.c:954:xl_opt_validate] 0-patchy-server: option
'listen-port' is deprecated, preferred is
'transport.socket.listen-port', continuing with correction
[2016-07-22 05:31:52.482436] E
[socket.c:771:__socket_server_bind] 0-tcp.patchy-server:
binding to  failed: Address already in use
[2016-07-22 05:31:52.482447] E
[socket.c:774:__socket_server_bind] 0-tcp.patchy-server:
Port is already in use
[2016-07-22 05:31:52.482459] W
[rpcsvc.c:1630:rpcsvc_create_listener] 0-rpc-service:
listening on transport failed
[2016-07-22 05:31:52.482469] W [MSGID: 115045]
[server.c:1061:init] 0-patchy-server: creation of
listener failed
[2016-07-22 05:31:52.482481] E [MSGID: 101019]
[xlator.c:433:xlator_init] 0-patchy-server:
Initialization of volume 'patchy-server' failed, review
your volfile again
[2016-07-22 05:31:52.482491] E [MSGID: 101066]
[graph.c:324:glusterfs_graph_init] 0-patchy-server:
initializing translator failed
[2016-07-22 05:31:52.482499] E [MSGID: 101176]
[graph.c:670:glusterfs_graph_activate] 0-graph: init failed

On Mon, Jul 25, 2016 at 12:00 PM, Ashish Pandey
<aspan...@redhat.com <mailto:aspan...@redhat.com>> wrote:

Hi,

Following test has failed 3 times in last two days -

./tests/bugs/snapshot/bug-1316437.t

https://build.gluster.org/job/rackspa

Re: [Gluster-devel] regression failed : snapshot/bug-1316437.t

[Avra Sengupta]

The crux of the problem is that as of today, brick processes on restart 
try to reuse the old port they were using (assuming that no other 
process will be using it, and not consulting pmap_registry_alloc() 
before using it). With a recent change, pmap_registry_alloc (), 
reassigns older ports that were used, but are now free. Hence snapd now 
gets a port that was previously used by a brick and tries to bind to it, 
whereas the older brick process without consulting pmap table blindly 
tries to connect to it, and hence we see this problem.


Now coming to the fix, I feel brick process should not try to get the 
older port and should just take a new port every time it comes up. We 
will not run out of ports with this change coz, now pmap allocates old 
ports again, and the previous port being used by the brick process will 
eventually be reused. If anyone sees any concern with this approach, 
please feel free to raise so now.


While awaiting feedback from you guys, I have sent this patch 
(http://review.gluster.org/15001), which moves the said test case to bad 
tests for now, and after we collectively reach to a conclusion on the 
fix, we will remove this from bad test.


Regards,
Avra

On 07/25/2016 02:33 PM, Avra Sengupta wrote:
The failure suggests that the port snapd is trying to bind to is 
already in use. But snapd has been modified to use a new port 
everytime. I am looking into this.


On 07/25/2016 02:23 PM, Nithya Balachandran wrote:

More failures:
https://build.gluster.org/job/rackspace-regression-2GB-triggered/22452/console

I see these messages in the snapd.log:

[2016-07-22 05:31:52.482282] I 
[rpcsvc.c:2199:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: 
Configured rpc.outstanding-rpc-limit with value 64
[2016-07-22 05:31:52.482352] W [MSGID: 101002] 
[options.c:954:xl_opt_validate] 0-patchy-server: option 'listen-port' 
is deprecated, preferred is 'transport.socket.listen-port', 
continuing with correction
[2016-07-22 05:31:52.482436] E [socket.c:771:__socket_server_bind] 
0-tcp.patchy-server: binding to  failed: Address already in use
[2016-07-22 05:31:52.482447] E [socket.c:774:__socket_server_bind] 
0-tcp.patchy-server: Port is already in use
[2016-07-22 05:31:52.482459] W [rpcsvc.c:1630:rpcsvc_create_listener] 
0-rpc-service: listening on transport failed
[2016-07-22 05:31:52.482469] W [MSGID: 115045] [server.c:1061:init] 
0-patchy-server: creation of listener failed
[2016-07-22 05:31:52.482481] E [MSGID: 101019] 
[xlator.c:433:xlator_init] 0-patchy-server: Initialization of volume 
'patchy-server' failed, review your volfile again
[2016-07-22 05:31:52.482491] E [MSGID: 101066] 
[graph.c:324:glusterfs_graph_init] 0-patchy-server: initializing 
translator failed
[2016-07-22 05:31:52.482499] E [MSGID: 101176] 
[graph.c:670:glusterfs_graph_activate] 0-graph: init failed


On Mon, Jul 25, 2016 at 12:00 PM, Ashish Pandey <aspan...@redhat.com 
<mailto:aspan...@redhat.com>> wrote:


Hi,

Following test has failed 3 times in last two days -

./tests/bugs/snapshot/bug-1316437.t

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22470/consoleFull

Please take a look at it and check if it spurious failure or not.

Ashish

___
Gluster-devel mailing list
Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-devel






___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] regression failed : snapshot/bug-1316437.t

[Avra Sengupta]

The failure suggests that the port snapd is trying to bind to is already 
in use. But snapd has been modified to use a new port everytime. I am 
looking into this.


On 07/25/2016 02:23 PM, Nithya Balachandran wrote:

More failures:
https://build.gluster.org/job/rackspace-regression-2GB-triggered/22452/console

I see these messages in the snapd.log:

[2016-07-22 05:31:52.482282] I 
[rpcsvc.c:2199:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: 
Configured rpc.outstanding-rpc-limit with value 64
[2016-07-22 05:31:52.482352] W [MSGID: 101002] 
[options.c:954:xl_opt_validate] 0-patchy-server: option 'listen-port' 
is deprecated, preferred is 'transport.socket.listen-port', continuing 
with correction
[2016-07-22 05:31:52.482436] E [socket.c:771:__socket_server_bind] 
0-tcp.patchy-server: binding to  failed: Address already in use
[2016-07-22 05:31:52.482447] E [socket.c:774:__socket_server_bind] 
0-tcp.patchy-server: Port is already in use
[2016-07-22 05:31:52.482459] W [rpcsvc.c:1630:rpcsvc_create_listener] 
0-rpc-service: listening on transport failed
[2016-07-22 05:31:52.482469] W [MSGID: 115045] [server.c:1061:init] 
0-patchy-server: creation of listener failed
[2016-07-22 05:31:52.482481] E [MSGID: 101019] 
[xlator.c:433:xlator_init] 0-patchy-server: Initialization of volume 
'patchy-server' failed, review your volfile again
[2016-07-22 05:31:52.482491] E [MSGID: 101066] 
[graph.c:324:glusterfs_graph_init] 0-patchy-server: initializing 
translator failed
[2016-07-22 05:31:52.482499] E [MSGID: 101176] 
[graph.c:670:glusterfs_graph_activate] 0-graph: init failed


On Mon, Jul 25, 2016 at 12:00 PM, Ashish Pandey > wrote:


Hi,

Following test has failed 3 times in last two days -

./tests/bugs/snapshot/bug-1316437.t

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22445/consoleFull

https://build.gluster.org/job/rackspace-regression-2GB-triggered/22470/consoleFull

Please take a look at it and check if it spurious failure or not.

Ashish

___
Gluster-devel mailing list
Gluster-devel@gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Snapshot Scheduler

[Avra Sengupta]

On 07/13/2016 02:37 AM, Niels de Vos wrote:

On Wed, Jul 13, 2016 at 12:37:17AM +0530, Avra Sengupta wrote:

Thanks Joe for the feedback. We are aware of the following issue, and we
will try and address this by going for a more generic approach, which will
not have platform dependencies.

I'm mostly in favour of using the standard functionalities that other
components already provide. Use systemd-timers when available, and cron
as fallback would have my preference. Not sure how much my opinion
counts, but I hope you'll take it into consideration. Writing a bug-free
scheduler from scratch is difficult :-)

Niels

Neils,

Thanks for the suggestion. I have been an advocate of having one single 
scheduler for Gluster from the beginning, such as it is not strictly 
tied to snapshots, but can be used by other components. Such a scheduler 
would be policy based and modular enough in both functionality and 
implementation to support any component's requirement, preferably in a 
plug and play fashion without much hassle to be needed from the other 
components end.


We were suggested to use cron back then, with the same argument as to 
not re-invent the wheel. While it makes perfect sense not to re-do 
what's already done, I would this time around try to aim at achieving 
the above mentioned goal first, irrespective of the underlying infra. 
That being said, we have not ruled out the use of either systemd-timers 
or cron, and we are currently estimating the scope of the feature in 
respect of the time in hand, and the resources at disposal, and hence 
asking for feedback. :)


Regards,
Avra





On 07/12/2016 11:59 PM, Joe Julian wrote:

cron isn't installed by default on Arch rather scheduling is done by
systemd timers. We might want to consider using systemd.timer for
systemd distros and crontab for legacy distros.


On 07/08/2016 03:01 AM, Avra Sengupta wrote:

Hi,

Snaphsots in gluster have a scheduler, which relies heavily on
crontab, and the shared storage. I would like people using this
scheduler, or for people to use this scheduler, and provide us
feedback on it's experience. We are looking for feedback on ease of
use, complexity of features, additional feature support etc.

It will help us in deciding if we need to revamp the existing
scheduler, or maybe rethink relying on crontab and re-writing our
own, thus providing us more flexibility. Thanks.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Snapshot Scheduler

[Avra Sengupta]

Thanks Joe for the feedback. We are aware of the following issue, and we 
will try and address this by going for a more generic approach, which 
will not have platform dependencies.


On 07/12/2016 11:59 PM, Joe Julian wrote:
cron isn't installed by default on Arch rather scheduling is done by 
systemd timers. We might want to consider using systemd.timer for 
systemd distros and crontab for legacy distros.



On 07/08/2016 03:01 AM, Avra Sengupta wrote:

Hi,

Snaphsots in gluster have a scheduler, which relies heavily on 
crontab, and the shared storage. I would like people using this 
scheduler, or for people to use this scheduler, and provide us 
feedback on it's experience. We are looking for feedback on ease of 
use, complexity of features, additional feature support etc.


It will help us in deciding if we need to revamp the existing 
scheduler, or maybe rethink relying on crontab and re-writing our 
own, thus providing us more flexibility. Thanks.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Snapshot Scheduler

[Avra Sengupta]

Thanks Alastair for the feedback. As of today we have auto-delete which 
when enabled deletes the oldest snapshot on exceeding the 
snap-max-soft-limit. Is this what you were trying to achieve, or were 
you thinking of more of a policy based approach, where like creation, 
deletion policies can be set, to target specific snapshots?



On 07/12/2016 11:45 PM, Alastair Neil wrote:
I don't know if I did something wrong, but I found the location that 
the scheduler wanted the shared storage was problematic as I recall it 
was under /run/gluster/snaps.  On CentOS 7 this failed to mount on 
boot.  I hacked the scheduler to use a location under /var/lib.


I also think there needs to be a way to schedule the removal of snapshots.

-Alastair


On 8 July 2016 at 06:01, Avra Sengupta <aseng...@redhat.com 
<mailto:aseng...@redhat.com>> wrote:


Hi,

Snaphsots in gluster have a scheduler, which relies heavily on
crontab, and the shared storage. I would like people using this
scheduler, or for people to use this scheduler, and provide us
feedback on it's experience. We are looking for feedback on ease
of use, complexity of features, additional feature support etc.

It will help us in deciding if we need to revamp the existing
scheduler, or maybe rethink relying on crontab and re-writing our
own, thus providing us more flexibility. Thanks.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] tests/basic/op_errnos.t failure

[Avra Sengupta]

Atin,

I am not sure about the docker containers, but both the failures you 
mentioned are in slave29, which as Talur explained is missing the 
appropriate backend filesystem. Owing to this, op-errno.t is just the 
tip of the iceberg, and every other test that uses lvm will fail in this 
particular slave will fail too.


Talur,
Thanks for looking into it. It is indeed strange this. I checked the 
dmesg and the /var/log/messages in this slave and I couldn't find any 
relevant log.


On 07/12/2016 05:29 PM, Raghavendra Talur wrote:

I checked the machine.

Here is the df -hT output
[jenkins@slave29 ~]$ cat /etc/fstab
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more 
info

#
/dev/xvda1  /   ext3 
 defaults,noatime,barrier=0 1 1

tmpfs   /dev/shmtmpfs defaults0 0
devpts  /dev/ptsdevpts  gid=5,mode=620 
 0 0

sysfs   /syssysfs defaults0 0
proc/proc   proc  defaults0 0
#/dev/xvdc1 noneswap  sw  0 0


We don't see a xfs device mounted at /d and / is of type ext3 which 
does not support fallocate. The uptime of the machine is 73 days 
though. I don't know how the /d xfs partition vanished.


On Tue, Jul 12, 2016 at 4:54 PM, Atin Mukherjee <amukh...@redhat.com 
<mailto:amukh...@redhat.com>> wrote:



https://build.gluster.org/job/rackspace-regression-2GB-triggered/22156/consoleFull
- another failure

On Tue, Jul 12, 2016 at 4:42 PM, Atin Mukherjee
<amukh...@redhat.com <mailto:amukh...@redhat.com>> wrote:



On Tue, Jul 12, 2016 at 4:36 PM, Avra Sengupta
<aseng...@redhat.com <mailto:aseng...@redhat.com>> wrote:

Hi Atin,

Please check the testcase result in the console. It
clearly states the reason of the failure. A quick search
of 30815, as shown in the testcase shows that the error
that is generated is a thinp issue, and we can see
fallocate failing and lvm not properly being setup in the
environment.


While this is valid for my docker containers, I am just
wondering why did this happen in jenkins slave?


Regards,
Avra

P.S Here are the logs from the console stating so.

*02:50:34* [09:50:34] Running tests in file 
./tests/basic/op_errnos.t
*02:50:41* fallocate: /d/backends/patchy_snap_vhd: fallocate 
failed: Operation not supported
*02:50:41* losetup: /d/backends/patchy_snap_vhd: warning: file 
smaller than 512 bytes, the loop device maybe be useless or invisible for 
system tools.
*02:50:41*Device /d/backends/patchy_snap_loop not found (or 
ignored by filtering).
*02:50:41*Device /d/backends/patchy_snap_loop not found (or 
ignored by filtering).
*02:50:41*Unable to add physical volume 
'/d/backends/patchy_snap_loop' to volume group 'patchy_snap_vg_1'.
*02:50:41*Volume group "patchy_snap_vg_1" not found
*02:50:41*Cannot process volume group patchy_snap_vg_1
*02:50:42*Volume group "patchy_snap_vg_1" not found
*02:50:42*Cannot process volume group patchy_snap_vg_1
*02:50:42* /dev/patchy_snap_vg_1/brick_lvm: No such file or 
directory
*02:50:42* Usage: mkfs.xfs
*02:50:42* /* blocksize */  [-b log=n|size=num]
*02:50:42* /* data subvol */[-d 
agcount=n,agsize=n,file,name=xxx,size=num,
*02:50:42*  
(sunit=value,swidth=value|su=num,sw=num),
*02:50:42*  sectlog=n|sectsize=num
*02:50:42* /* inode size */ [-i 
log=n|perblock=n|size=num,maxpct=n,attr=0|1|2,
*02:50:42*  projid32bit=0|1]
*02:50:42* /* log subvol */ [-l 
agnum=n,internal,size=num,logdev=xxx,version=n
*02:50:42*  
sunit=value|su=num,sectlog=n|sectsize=num,
*02:50:42*  lazy-count=0|1]
*02:50:42* /* label */  [-L label (maximum 12 
characters)]
*02:50:42* /* naming */ [-n log=n|size=num,version=2|ci]
*02:50:42* /* prototype file */ [-p fname]
*02:50:42* /* quiet */  [-q]
*02:50:42* /* realtime subvol */[-r 
extsize=num,size=num,rtdev=xxx]
*02:50:42* /* sectorsize */ [-s log=n|size=num]
*02:50:42* /* version */[-V]
*02:50:42*  devicename
*02:50:42*  is required unless -d name=xxx is given.
*02:50:42*  is xxx (bytes), xxxs (sectors), 

Re: [Gluster-devel] tests/basic/op_errnos.t failure

[Avra Sengupta]

Hi Atin,

Please check the testcase result in the console. It clearly states the 
reason of the failure. A quick search of 30815, as shown in the testcase 
shows that the error that is generated is a thinp issue, and we can see 
fallocate failing and lvm not properly being setup in the environment.


Regards,
Avra

P.S Here are the logs from the console stating so.

*02:50:34* [09:50:34] Running tests in file ./tests/basic/op_errnos.t
*02:50:41* fallocate: /d/backends/patchy_snap_vhd: fallocate failed: Operation 
not supported
*02:50:41* losetup: /d/backends/patchy_snap_vhd: warning: file smaller than 512 
bytes, the loop device maybe be useless or invisible for system tools.
*02:50:41*Device /d/backends/patchy_snap_loop not found (or ignored by 
filtering).
*02:50:41*Device /d/backends/patchy_snap_loop not found (or ignored by 
filtering).
*02:50:41*Unable to add physical volume '/d/backends/patchy_snap_loop' to 
volume group 'patchy_snap_vg_1'.
*02:50:41*Volume group "patchy_snap_vg_1" not found
*02:50:41*Cannot process volume group patchy_snap_vg_1
*02:50:42*Volume group "patchy_snap_vg_1" not found
*02:50:42*Cannot process volume group patchy_snap_vg_1
*02:50:42* /dev/patchy_snap_vg_1/brick_lvm: No such file or directory
*02:50:42* Usage: mkfs.xfs
*02:50:42* /* blocksize */  [-b log=n|size=num]
*02:50:42* /* data subvol */[-d agcount=n,agsize=n,file,name=xxx,size=num,
*02:50:42*  (sunit=value,swidth=value|su=num,sw=num),
*02:50:42*  sectlog=n|sectsize=num
*02:50:42* /* inode size */ [-i 
log=n|perblock=n|size=num,maxpct=n,attr=0|1|2,
*02:50:42*  projid32bit=0|1]
*02:50:42* /* log subvol */ [-l 
agnum=n,internal,size=num,logdev=xxx,version=n
*02:50:42*  sunit=value|su=num,sectlog=n|sectsize=num,
*02:50:42*  lazy-count=0|1]
*02:50:42* /* label */  [-L label (maximum 12 characters)]
*02:50:42* /* naming */ [-n log=n|size=num,version=2|ci]
*02:50:42* /* prototype file */ [-p fname]
*02:50:42* /* quiet */  [-q]
*02:50:42* /* realtime subvol */[-r extsize=num,size=num,rtdev=xxx]
*02:50:42* /* sectorsize */ [-s log=n|size=num]
*02:50:42* /* version */[-V]
*02:50:42*  devicename
*02:50:42*  is required unless -d name=xxx is given.
*02:50:42*  is xxx (bytes), xxxs (sectors), xxxb (fs blocks), xxxk (xxx 
KiB),
*02:50:42*xxxm (xxx MiB), xxxg (xxx GiB), xxxt (xxx TiB) or xxxp (xxx 
PiB).
*02:50:42*  is xxx (512 byte blocks).
*02:50:42* mount: special device /dev/patchy_snap_vg_1/brick_lvm does not exist
*02:50:53* ./tests/basic/op_errnos.t ..
*02:50:53* 1..21
*02:50:53* ok 1, LINENUM:12
*02:50:53* ok 2, LINENUM:13
*02:50:53* ok 3, LINENUM:14
*02:50:53* ok 4, LINENUM:16
*02:50:53* ok 5, LINENUM:18
*02:50:53* ok 6, LINENUM:19
*02:50:53* ok 7, LINENUM:20



On 07/12/2016 03:47 PM, Atin Mukherjee wrote:

Hi Avra,

The above fails locally as well along with few regression failures I 
observed and one of them are at [1]


not ok 12 Got "  30807" instead of "30809", LINENUM:26
FAILED COMMAND: 30809 get-op_errno-xml snapshot restore snap1

not ok 17 Got "  30815" instead of "30812", LINENUM:31
FAILED COMMAND: 30812 get-op_errno-xml snapshot create snap1 patchy 
no-timestamp


[1] 
https://build.gluster.org/job/rackspace-regression-2GB-triggered/22154/console


--Atin


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Snapshot Scheduler

[Avra Sengupta]

Hi,

Snaphsots in gluster have a scheduler, which relies heavily on crontab, 
and the shared storage. I would like people using this scheduler, or for 
people to use this scheduler, and provide us feedback on it's 
experience. We are looking for feedback on ease of use, complexity of 
features, additional feature support etc.


It will help us in deciding if we need to revamp the existing scheduler, 
or maybe rethink relying on crontab and re-writing our own, thus 
providing us more flexibility. Thanks.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Handling locks in NSR

[Avra Sengupta]

Hi,

I have sent a patch(http://review.gluster.org/#/c/14226/1) in accordance 
to lock/unlock fops in jbr-server and the discussion we had below. 
Please feel free to review the same. Thanks.


Regards,
Avra

On 03/03/2016 12:21 PM, Avra Sengupta wrote:

On 03/03/2016 02:29 AM, Shyam wrote:

On 03/02/2016 03:10 AM, Avra Sengupta wrote:

Hi,

All fops in NSR, follow a specific workflow as described in this
UML(https://docs.google.com/presentation/d/1lxwox72n6ovfOwzmdlNCZBJ5vQcCaONvZva0aLWKUqk/edit?usp=sharing). 


However all locking fops will follow a slightly different workflow as
described below. This is a first proposed draft for handling locks, and
we would like to hear your concerns and queries regarding the same.


This change, to handle locking FOPs differently, is due to what 
limitation/problem? (apologies if I missed an earlier thread on the 
same)


My understanding is that this is due to the fact that the actual FOP 
could fail/block (non-blocking/blocking) as there is an existing lock 
held, and hence just adding a journal entry and meeting quorum, is 
not sufficient for the success of the FOP (it is necessary though to 
handle lock preservation in the event of leadership change), rather 
acquiring the lock is. Is this understanding right?
Yes it is right, the change in approach for handling locks is to avoid 
getting into a deadlock amogst the followers.


Based on the above understanding of mine, and the discussion below, 
the intention seems to be to place the locking xlator below the 
journal. What if we place this xlator above the journal, but add 
requirements that FOPs handled by this xlator needs to reach the 
journal?


Assuming we adopt this strategy (i.e the locks xlator is above the 
journal xlator), a successful lock acquisition by the locks xlator is 
not enough to guarantee that the lock is preserved across the replica 
group, hence it has to reach the journal and as a result pass through 
other replica members journal and locks xlators as well.


If we do the above, what are the advantages and repercussions of the 
same?
Why would we want to put the locking xlator above the journal. Is 
there a use case for that?
Firstly, we would have to modify the locking xlator to make it pass 
through.
We would also introduce a small window where we perform the lock 
successfully, but have a failure on the journal. We would then have to 
release the lock because we failed to journal it. In the previous 
approach, if we fail to journal it, we wouldn't even go to the locking 
xlator. Logically it makes the locking xlator dependent on the 
journal's output, whereas ideally the journal should be dependent on 
the locking xlator's output.


Some of the points noted here (like conflicting non-blocking locks 
when the previous lock is not yet released) could be handled. Also in 
your scheme, what happens to blocking lock requests, the FOP will 
block, there is no async return to handle the success/failure of the 
same.
Yes the FOP will block on blocking lock requests. I assume that's the 
behaviour today. Please correct me if I am wrong.


The downside is that on reconciliation we need to, potentially, undo 
some of the locks that are held by the locks xlator (in the new 
leader), which is outside the scope of the journal xlator.
Yes we need to do lock cleanup on reconciliation, which is anyways 
outside the scope of the journal xlator. The reconciliation daemon 
will compare the terms on each replica node, and either acquire or 
release locks accordingly.



I also assume we need to do the same for the leases xlator as well, 
right?
Yes, as long as we handle locking properly leases xlators shouldn't be 
a problem.





1. On receiving the lock, the leader will Journal the lock himself, and
then try to actually acquire the lock. At this point in time, if it
fails to acquire the lock, then it will invalidate the journal entry,
and return a -ve ack back to the client. However, if it is 
successful in

acquiring the lock, it will mark the journal entry as complete, and
forward the fop to the followers.

2. The followers on receiving the fop, will journal it, and then try to
actually acquire the lock. If it fails to acquire the lock, then it 
will

invalidate the journal entry, and return a -ve ack back to the leader.
If it is successful in acquiring the lock, it will mark the journal
entry as complete,and send a +ve ack to the leader.

3. The leader on receiving all acks, will perform a quorum check. If
quorum meets, it will send a +ve ack to the client. If the quorum 
fails,

it will send a rollback to the followers.

4. The followers on receiving the rollback, will journal it first, and
then release the acquired lock. It will update the rollback entry in 
the

journal as complete and send an ack to the leader.

5. The leader on receiving the rollback acks, will journal it's own
rollback, and then release the acquired lock. It will update the
rollback entry in the journal, and send a -ve ack

[Gluster-devel] Testcase broken due to posix iatt commit

[Avra Sengupta]

Hi Raghavendra,

As part of the patch (http://review.gluster.org/#/c/13730/16), the 
inode_ctx is not created in posix_acl_ctx_get(). Because of this the 
testcase in http://review.gluster.org/#/c/13623/ breaks. It fails with 
the following logs:


[2016-03-28 13:43:39.216168] D [MSGID: 0] 
[io-threads.c:346:iot_schedule] 0-patchy-io-threads: CREATE scheduled as 
normal fop
[2016-03-28 13:43:39.216495] E [posix-acl.c:199:acl_permits] (--> 
/usr/local/lib/libglusterfs.so.0(_gf_log_callingfn+0x1ba)[0x7f6fea72780a] (--> 
/usr/local/lib/glusterfs/3.8dev/xlator/features/access-control.so(+0x49c4)[0x7f6fde5499c4] 
(--> 
/usr/local/lib/glusterfs/3.8dev/xlator/features/access-control.so(+0xa855)[0x7f6fde54f855] 
(--> 
/usr/local/lib/glusterfs/3.8dev/xlator/features/locks.so(+0xd37e)[0x7f6fde33837e] 
(--> 
/usr/local/lib/glusterfs/3.8dev/xlator/features/upcall.so(+0x640f)[0x7f6fde12040f] 
) 0-patchy-access-control: inode ctx is NULL for 
----0001
[2016-03-28 13:43:39.216544] I [MSGID: 115071] 
[server-rpc-fops.c:1612:server_create_cbk] 0-patchy-server: 8: CREATE 
/file1 (----0001/file1) ==> (Permission 
denied) [Permission denied]


Is it because we missed the inode_ctx creation that was being done by 
posix_acl_ctx_get() previously? Can you please shed some light on this


Regards,
Avra


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Handling locks in NSR

[Avra Sengupta]

On 03/02/2016 02:02 PM, Venky Shankar wrote:

On Wed, Mar 02, 2016 at 01:40:08PM +0530, Avra Sengupta wrote:

Hi,

All fops in NSR, follow a specific workflow as described in this 
UML(https://docs.google.com/presentation/d/1lxwox72n6ovfOwzmdlNCZBJ5vQcCaONvZva0aLWKUqk/edit?usp=sharing).
However all locking fops will follow a slightly different workflow as
described below. This is a first proposed draft for handling locks, and we
would like to hear your concerns and queries regarding the same.

1. On receiving the lock, the leader will Journal the lock himself, and then
try to actually acquire the lock. At this point in time, if it fails to
acquire the lock, then it will invalidate the journal entry, and return a
-ve ack back to the client. However, if it is successful in acquiring the
lock, it will mark the journal entry as complete, and forward the fop to the
followers.

So, does a contending non-blocking lock operation check only on the leader
since the followers might have not yet ack'd an earlier lock operation?
A non-blocking lock follows the same work flow, and thereby checks on 
the leader first. In this case, it would be blocked on the leader, till 
the leader releases the lock. Then it will follow the same workflow.



2. The followers on receiving the fop, will journal it, and then try to
actually acquire the lock. If it fails to acquire the lock, then it will
invalidate the journal entry, and return a -ve ack back to the leader. If it
is successful in acquiring the lock, it will mark the journal entry as
complete,and send a +ve ack to the leader.

3. The leader on receiving all acks, will perform a quorum check. If quorum
meets, it will send a +ve ack to the client. If the quorum fails, it will
send a rollback to the followers.

4. The followers on receiving the rollback, will journal it first, and then
release the acquired lock. It will update the rollback entry in the journal
as complete and send an ack to the leader.

What happens if the rollback fails for whatever reason?
The leader receives a -ve rollback ack, but there's little it can do 
about it. Depending on the failure, it will be resolved during 
reconciliation



5. The leader on receiving the rollback acks, will journal it's own
rollback, and then release the acquired lock. It will update the rollback
entry in the journal, and send a -ve ack to the client.

Few things to be noted in the above workflow are:
1. It will be a synchronous operation, across the replica volume.
2. Reconciliation will take care of nodes who have missed out the locks.
3. On a client disconnect, there will be a lock-timeout on whose expiration
all locks held by that particular client will be released.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Gap in protocol client-server handshake

[Avra Sengupta]

Hi,

Currently on a successful connection between protocol server and client, 
the protocol client initiates a CHILD_UP event in the client stack. At 
this point in time, only the connection between server and client is 
established, and there is no guarantee that the server side stack is 
ready to serve requests. It works fine now, as most server side 
translators are not dependent on any other factors, before being able to 
serve requests today and hence they are up by the time the client stack 
translators receive the CHILD_UP (initiated by client handshake).


The gap here is exposed when certain server side translators like 
NSR-Server for example, have a couple of protocol clients as their 
child(connecting them to other bricks), and they can't really serve 
requests till a quorum of their children are up. Hence these translators 
*should* defer sending CHILD_UP till they have enough children up, and 
the same CHILD_UP event needs to be propagated to the client stack 
translators.


I have sent a patch(http://review.gluster.org/#/c/13549/) addressing 
this, where we maintain a child_up variable in both the protocol client 
and protocol server translators. The protocol server updates this value 
based on the CHILD_UP and CHILD_DOWN events it receives from the 
translators below it. On receiving such an event it forwards that event 
to the client. The protocol client on receiving such an event forwards 
it up the client stack, thereby letting the client translators correctly 
know that the server is up and ready to serve.


The clients connecting later(long after a server has initialized and 
processed it's CHILD_UP events), receive a child_up status as part of 
the handshake, and based on the status of the server's child_up, either 
propagate a CHILD_UP event or defer it.


Please have a look at the patch, and kindly state if it you have any 
concerns or you foresee any scenarios of interest which we might have 
missed.


Regards,
Avra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NSR: Suggestions for a new name

[Avra Sengupta]

Well, we got quite a few suggestions. So I went ahead and created a 
doodle poll. Please find the link below for the poll, and vote for the 
name you think will be the best.


http://doodle.com/poll/h7gfdhswrbsxxiaa

Regards,
Avra

On 01/21/2016 12:21 PM, Avra Sengupta wrote:

On 01/21/2016 12:20 PM, Atin Mukherjee wrote:

Etherpad link please?
Oops My Bad. Here it is 
https://public.pad.fsfe.org/p/NSR_name_suggestions


On 01/21/2016 12:19 PM, Avra Sengupta wrote:

Thanks for the suggestion Pranith. To make things interesting, we have
created an etherpad where people can put their suggestions. Somewhere
around mid of feb, we will look at all the suggestions we have got, 
have

a community vote and zero in on one. The suggester of the winning name
gets a goody.

Feel free to add more than one entry.

Regards,
Avra

On 01/21/2016 10:08 AM, Pranith Kumar Karampuri wrote:


On 01/19/2016 08:00 PM, Avra Sengupta wrote:

Hi,

The leader election based replication has been called NSR or "New
Style Replication" for a while now. We would like to have a new name
for the same that's less generic. It can be something like "Leader
Driven Replication" or something more specific that would make sense
a few years down the line too.

We would love to hear more suggestions from the community. Thanks

If I had a chance to name AFR (Automatic File Replication) I would
have named it Automatic Data replication. Feel free to use it if you
like it.

Pranith

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Reviewers needed for NSR client and server patches

[Avra Sengupta]

Hi,

We have two patches(mentioned below) for NSR client and NSR server 
available. These patches provide the basic client and server 
functionality as described in the design 
(https://docs.google.com/document/d/1bbxwjUmKNhA08wTmqJGkVd_KNCyaAMhpzx4dswokyyA/edit?usp=sharing). 
It would be great if people interested, could have a look at the patches 
and review them.


NSR Client patch : http://review.gluster.org/#/c/12388/
NSR Server patch : http://review.gluster.org/#/c/12705/

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NSR: Suggestions for a new name

[Avra Sengupta]

Thanks for the suggestion Pranith. To make things interesting, we have 
created an etherpad where people can put their suggestions. Somewhere 
around mid of feb, we will look at all the suggestions we have got, have 
a community vote and zero in on one. The suggester of the winning name 
gets a goody.


Feel free to add more than one entry.

Regards,
Avra

On 01/21/2016 10:08 AM, Pranith Kumar Karampuri wrote:



On 01/19/2016 08:00 PM, Avra Sengupta wrote:

Hi,

The leader election based replication has been called NSR or "New 
Style Replication" for a while now. We would like to have a new name 
for the same that's less generic. It can be something like "Leader 
Driven Replication" or something more specific that would make sense 
a few years down the line too.


We would love to hear more suggestions from the community. Thanks


If I had a chance to name AFR (Automatic File Replication) I would 
have named it Automatic Data replication. Feel free to use it if you 
like it.


Pranith


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NSR: Suggestions for a new name

[Avra Sengupta]

On 01/21/2016 12:20 PM, Atin Mukherjee wrote:

Etherpad link please?

Oops My Bad. Here it is https://public.pad.fsfe.org/p/NSR_name_suggestions


On 01/21/2016 12:19 PM, Avra Sengupta wrote:

Thanks for the suggestion Pranith. To make things interesting, we have
created an etherpad where people can put their suggestions. Somewhere
around mid of feb, we will look at all the suggestions we have got, have
a community vote and zero in on one. The suggester of the winning name
gets a goody.

Feel free to add more than one entry.

Regards,
Avra

On 01/21/2016 10:08 AM, Pranith Kumar Karampuri wrote:


On 01/19/2016 08:00 PM, Avra Sengupta wrote:

Hi,

The leader election based replication has been called NSR or "New
Style Replication" for a while now. We would like to have a new name
for the same that's less generic. It can be something like "Leader
Driven Replication" or something more specific that would make sense
a few years down the line too.

We would love to hear more suggestions from the community. Thanks

If I had a chance to name AFR (Automatic File Replication) I would
have named it Automatic Data replication. Feel free to use it if you
like it.

Pranith

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7.7 update

[Avra Sengupta]

Adding http://review.gluster.org/#/c/13119/ to the list. Hopefully it 
will go in today.


On 01/20/2016 01:31 PM, Venky Shankar wrote:



Pranith Kumar Karampuri wrote:

https://public.pad.fsfe.org/p/glusterfs-3.7.7 is the final list of
patches I am waiting for before making 3.7.7 release.

Please let me know if I need to wait for any other patches. It would be
great if we make the tag tomorrow.


Backport of http://review.gluster.org/#/c/13120/

But that needs http://review.gluster.org/#/c/13041/ to be merged.



Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



Venky
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] NSR: Suggestions for a new name

[Avra Sengupta]

Hi,

The leader election based replication has been called NSR or "New Style 
Replication" for a while now. We would like to have a new name for the 
same that's less generic. It can be something like "Leader Driven 
Replication" or something more specific that would make sense a few 
years down the line too.


We would love to hear more suggestions from the community. Thanks

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] NetBSD tests not running to completion.

[Avra Sengupta]

On 01/07/2016 02:39 PM, Emmanuel Dreyfus wrote:

On Wed, Jan 06, 2016 at 05:49:04PM +0530, Ravishankar N wrote:

I re triggered NetBSD regressions for http://review.gluster.org/#/c/13041/3
but they are being run in silent mode and are not completing. Can some one
from the infra-team take a look? The last 22 tests in
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/ have
failed. Highly unlikely that something is wrong with all those patches.

I note your latest test compelted with an error in mount-nfs-auth.t:
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/13260/consoleFull

Would you have the jenkins build that did not complete s that I can have a
look at it?

Generally speaking, I have to pôint that NetBSD regression does show light
on generic bugs, we had a recent exemple with quota-nfs.t. For now there
are not other well supported platforms, but if you want glusterfs to
be really portable, removing mandatory NetBSD regression is not a good idea:
portability bugs will crop.

Even a daily or weekly regression run seems a bad idea to me. If you do not
prevent integration of patches that break NetBSD regression, that will get
in, and tests will break one by one over time. I have a first hand
experience of this situation, when I was actually trying to catch on with
NetBSD regression. Many time I reached something reliable enough to become
mandatory, and got broken by a new patch before it became actualy mandatory.
Why is this a bad idea? This approach seems to be the middle ground 
between, not accepting any patches because of netbsd regressions 
failing, and totally removing the mandate of having NetBSD regressions 
passing. It isn't going to be any different than it is today, just that 
a weekly regression will help concentrate our efforts(in debugging the 
issues reposrted by NetBSD regression) and allow us to be more 
productive. It's a simple matter of assigning ownership of triaging the 
regressions to people who own the particuar testcases that fail the 
regression.


All the time that is spent on monitoring patches, and re-trigerring 
regressions can be spent elsewhere. Also as a project we need to decide, 
if relaxing a few requirements can reduce the turn around time between a 
patch being posted upstream, and the patch being merged, without 
actually affecting portability, then what reason do we have for not 
pursuing such an approach.
  


IMO, relaxing NetBSD regression requirement means the project drops the goal
of being portable.



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD tests not running to completion.

[Avra Sengupta]

On 01/07/2016 07:24 PM, Jeff Darcy wrote:

I'd prefer a "defined level of effort" approach which *might* reduce the
benefit we derive from NetBSD testing but *definitely* keeps the cost
under control.

Did we identify the worst offenders within the spurious failing tests?
We could ignore their output on NetBSD (this is how I started)

There do seem to be patterns - ironically, NFS-related tests seem to show up a 
lot - but I haven't studied this enough to give a detailed answer.  More to the 
point, is there really much difference between running tests all the time and 
ignoring certain ones, vs. running them nightly/weekly and triaging the results 
manually?  Besides resource consumption, I mean.  If we find something in a 
nightly/weekly test that closer inspection leads us to believe is a generic and 
serious problem, we should be able to create a Linux reproducer or even block 
merges by fiat.  Then the only difference is whether we default to allowing 
merges to occur despite NetBSD failures or default to blocking them.  Either 
way we can make exceptions.
Agree with your point. If we are ready to make exceptions, then we might 
as well not block all the patches. As Jeff suggested, triaging the 
nightly/weekly results manually and making any serious issues a blocker 
should suffice. We have been following the current model for so long, 
and have been constantly facing issues because of it. I feel there is no 
harm in trying the nightly/weekly approach for once (even temporarily if 
we have to), and see how things work out.

___
Gluster-infra mailing list
gluster-in...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-infra


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Logging framework in Gluter 4.0

[Avra Sengupta]

Hi,

As almost all the components targeted for Gluster 4.0 have moved from 
design phase to implementation phase on some level or another, I feel 
it's time to get some consensus on the logging framework we are going to 
use. Are we going to stick with the message ID formatted logging 
framework in use today, or shall we move on to a better solution.


Some good to have features I would like to have in the logging framework 
are:
1. Specific distinction between logs of various transactions (or 
operations), that would make it a lot easier to make sense than looking 
at some serialized log dump
2. We often encounter issues, and then find ourselves wishing the 
process was running in debug mode so that we could have gotten some 
debug logs. If there is any solution that enables me tracability of the 
process's flow, without compromising the space constraints of the logs 
that would be great to have.


This is kind of an open floor question I am putting across to the 
community, with no specific solution in mind at this point in time, but 
a concern that we should rather decide on something sooner in the 
development cycle than later.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

[Avra Sengupta]

My 2 cents on this:

The decision we take on this should have certain rationale, and I see 
two key things affecting that decision.
1. How much of the code we are planning to write now, is going to make 
it to the final product. If we are sure that a sizeable amount of code 
we are writing now, is gonna change over the course of time, then it 
makes sense to have that kinda development in experimental branch.
2. Is the code we are planning to deliver meddle with existing 
infrastructure. If so, then it should definitely go into experimental


Now analyzing NSR based on the above two constraints :
1. We definitely plan to use more than 80% of the code we write now, and 
plan to go about it in an incremental module by module kinda way.
2. We hardly have any overlap with existing infrastructure, and we can 
easily integrate with the current glusterd now, and move on to Glusterd 
2, as and when it is ready for consumption.


Based on the above analysis, I would say NSR can go right into master, 
and we can easily make sure that it's not built as part of any release. 
Now what NSR would follow, isn;t necessary for other translators to 
follow. For eg. I would think Glusterd2 would have to be in experimental 
coz it might meddle with current glusterd (but Atin would know better). 
The point being, the decision we take need not be a collective decision 
for all components, that would be enforced as a process, but rather 
should be a decision taken by individual components based on merit.



On 10/28/2015 08:32 PM, Shyam wrote:

Sending this along again.

- Are we decided on *experimental*?
- If so, what else needs attention in the patch below?
- (Re)views please... (views as in "What are your views on this?", not 
"Have you viewed this?" ;) )


Shyam

On 10/12/2015 02:18 PM, Shyam wrote:

In an effort to push this along, update the change with suggested edits
and comments till now, request a review and further comments so that we
make this official at some (sooner) point in time.

http://review.gluster.org/#/c/12321/2
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] gluster readdocs unaccessible

[Avra Sengupta]

Hi,

I am unable to access gluster.readdocs.org . Is anyone else facing the 
same issue.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] gluster readdocs unaccessible

[Avra Sengupta]

On 10/14/2015 02:05 PM, Sankarshan Mukhopadhyay wrote:

On Wed, Oct 14, 2015 at 2:03 PM, Avra Sengupta <aseng...@redhat.com> wrote:

I am unable to access gluster.readdocs.org . Is anyone else facing the same
issue.

<https://gluster.readthedocs.org/en/latest/> ?



Thanks. Looks like I messed up the url :p
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GlusterD 2.0 status updates

[Avra Sengupta]

On 09/07/2015 05:50 PM, Kaushal M wrote:

Hi Richard,
Thanks a lot for you feedback. I've done my replies inline.

On Sat, Sep 5, 2015 at 5:46 AM, Richard Wareing  wrote:

Hey Atin (and the wider community),

This looks interesting, though I have a couple questions:

1. Language choice - Why the divergence from Python (which I'm no fan of) which 
is already heavily used in GlusterFS?  It seems a bit strange to me to 
introduce yet another language into the GlusterFS code base.  Doing this will 
make things less cohesive, harder to test, make it more difficult for 
contributors to understand the code base and improve their coding skills to be 
effective contributors.  I'm a bit concerned we are setting a precedent that 
development will switch to the new flavor of the day.  If a decision has been 
made to shift away from Python for the portions of GlusterFS where performance 
isn't a concern, will the portions currently written in Python be re-written as 
well?  I also question the wisdom of a language will a shallow developer pool, 
and a less development process (somewhat of an ironic choice IMHO).


One of our aims for GlusterD-2.0 was to switch to a higher level
language. While C is good for solving lower level performance critical
problems, it isn't best suited for the kind of management tasks we
want GlusterD to focus on. The choice of Go over Python as the higher
level language, was mainly driven by the following
- Go is easier to a hang of for a developer coming from a C
background. IMO for a developer new to both Go and Python, it's easier
to start producing working code in Go. The structure and syntax of the
language and the related tools make it easier.
- Go has built into the language support (think goroutines, channels)
for easily implementing the concurrency patterns that we have in the
current GlusterD codebase. This makes it easier for us think about
newer designs based on our understanding of the existing
implementation.
- We have concerns about the concurrency and threading capabilities of
Python. We have faced a lot of problems with doing concurrency and
threading in GlusterD (though this is mostly down to bad design).
Python has known issues with threading, which doesn't give us
confidence as python novices.
- Go has a pretty good standard library (possibly the best standard
library), which provides us with almost everything required. This
reduces the number of dependencies that we pull in.

That's not to say Go doesn't have it's drawbacks. The major drawback
that I see currently with Go is that it doesn't have a way to do
plugins (dynamically load and execute binaries). But there have been
recent developments in this area. Go 1.5 landed support for building
Go packages as dynamic libraries. There is design ready to support
runtime loadability (plugins) ready, which is targeted for inclusion
in Go 1.6. As Go follows a 6 month release cycle, 1.6 is scheduled for
a Feb 2016 release, we need not wait too long for this to land. In the
meantime, we plan to build up the rest of the infrastructure required.
[AVRA]: There are other components like snapshot, geo-rep management 
which are closely tied with the glusterd code base today. In order to 
scale glusterd we have to change the current design and most of these 
components might have to re-write some parts of it, to play well with 
the new glusterd. This is completely acceptable, as it brings along 
solution to a lot of pain points these components have been suffering from.


But we also have to look at the developer base currently contributing to 
these components too. Most of the snapshot and geo-rep code base (going 
with these examples as I myself have contributed code in them), is 
written in C and python, and the developers contributing to the same are 
well versed in those languages. For them to be expected to pick up a new 
language, and re-writing the components in that language, might be a bit 
far fetched. I would assume, glusterd would be providing an interface, 
for other components to interact with it, without enforcing the choice 
of language. And if so, would such an interface have any impact on the 
design or performance of these components.



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Snapshot test Spurious Failure

[Avra Sengupta]

It's not a spurious failure. In the patch 
http://review.gluster.org/#/c/12031/3, you are using list_for_each_entry 
in clear_bricklist(), and deleting an item from the list. That is not a 
right practice. Instead you should use  list_for_each_entry_safe.


Regards,
Avra

On 09/04/2015 11:50 AM, Avra Sengupta wrote:

Hi,

I am having a look at the core. Will update shortly.

Regards,
Avra

On 09/04/2015 11:46 AM, Joseph Fernandes wrote:


./tests/bugs/snapshot/bug-1227646.t
https://build.gluster.org/job/rackspace-regression-2GB-triggered/14021/consoleFull 







___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Snapshot test Spurious Failure

[Avra Sengupta]

Hi,

I am having a look at the core. Will update shortly.

Regards,
Avra

On 09/04/2015 11:46 AM, Joseph Fernandes wrote:


./tests/bugs/snapshot/bug-1227646.t
https://build.gluster.org/job/rackspace-regression-2GB-triggered/14021/consoleFull



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] NetBSD builds failing

[Avra Sengupta]

Hi,

NetBSD regression runs are failing coz of a build error.

https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9867/consoleFull
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9865/consoleFull
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9864/consoleFull
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9863/consoleFull
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9862/consoleFull

Is anyone aware of any patch being merged, which might be causing this. 
All builds are failing at the following:


Making install in glupy
 /home/jenkins/root/workspace/rackspace-netbsd7-regression-triggered/install-sh 
-c -d '/usr/pkg/lib/python2.7/site-packages/gluster/glupy'
 /usr/bin/install -c -m 644 
/home/jenkins/root/workspace/rackspace-netbsd7-regression-triggered/xlators/features/glupy/src/glupy/__init__.py
 '/usr/pkg/lib/python2.7/site-packages/gluster/glupy'
Byte-compiling python modules...
__init__.pyTraceback (most recent call last):
  File "", line 18, in 
  File "/usr/pkg/lib/python2.7/py_compile.py", line 123, in compile
with open(cfile, 'wb') as fc:
IOError: [Errno 1] Operation not permitted: 
'/usr/pkg/lib/python2.7/site-packages/gluster/glupy/__init__.pyc'
*** Error code 1

Stop.
make[6]: stopped in /build/scratch/xlators/features/glupy/src/glupy
*** Error code 1

Stop.
make[5]: stopped in /build/scratch/xlators/features/glupy/src/glupy
*** Error code 1

Stop.
make[4]: stopped in /build/scratch/xlators/features/glupy/src
*** Error code 1

Stop.
make[3]: stopped in /build/scratch/xlators/features/glupy
*** Error code 1

Stop.
make[2]: stopped in /build/scratch/xlators/features
*** Error code 1

Stop.
make[1]: stopped in /build/scratch/xlators
*** Error code 1

Stop.
make: stopped in /build/scratch
+ RET=1
+ '[' 1 '!=' 0 ']'
+ exit 1
Build step 'Ex?cuter un script shell' marked build as failure
Finished: FAILURE

Regards,
Avra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD builds failing

[Avra Sengupta]

On 08/31/2015 12:10 PM, Emmanuel Dreyfus wrote:

Avra Sengupta <aseng...@redhat.com> wrote:


IOError: [Errno 1] Operation not permitted:

'/usr/pkg/lib/python2.7/site-packages/gluster/glupy/__init__.pyc'

My fault: this is the immutable flag that prevents installation. I
removed it from /usr/pkg/lib/python2.7/site-packages/gluster/ on all
slaves. Retrigger the build and it will pass.



They are now passing. Thanks Emmanuel.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Spurious failure of ./tests/basic/tier/tier_lookup_heal.t

[Avra Sengupta]

Hi,

NetBSD regressions seem to be failing regularly with 
./tests/basic/tier/tier_lookup_heal.t Following are a could of times it 
has been encountered. Can you please look into this, as it is blocking 
NetBSD regression runs.


https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9576/console
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9575/console
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/9570/console

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Netbsd build failure

[Avra Sengupta]

On 08/22/2015 06:56 AM, Emmanuel Dreyfus wrote:

Emmanuel Dreyfus m...@netbsd.org wrote:


Yes, this is again a test corrupting random system files.
I started rebuild of nbslave7[149] from image...

Done.

Thanks a lot Emmanuel. The tests seem to be starting fine now.




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Fresh NetBSD regression failures

[Avra Sengupta]

Hi,

All NetBSD regression failures are again failing (more like refusing to 
build), with the following error.


[2015-08-21 10:53:51.N]:++ G_LOG:./tests/basic/meta.t: TEST: 18 
Started volinfo_field patchy Status ++

Is someone aware of this issue. Right now no NetBSD regressions are 
running coz of this.


Regards,
Avra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Netbsd build failure

[Avra Sengupta]

+ Adding Vijaikumar

On 08/20/2015 04:19 PM, Niels de Vos wrote:

On Thu, Aug 20, 2015 at 03:05:56AM -0400, Susant Palai wrote:

Hi,
   I tried running netbsd regression twice on a patch. And twice it failed at 
the same point. Here is the error:

snip
Build GlusterFS
***

+ '/opt/qa/build.sh'
   File /usr/pkg/lib/python2.7/site.py, line 601
 [2015-08-19 05:45:06.N]:++ 
G_LOG:./tests/basic/quota-anon-fd-nfs.t: TEST: 85 ! fd_write 3 content 
++
This particular test is currently in bad test and I believe Vijaikumar 
is looking into it. Could you please make sure if there is any other 
failure(apart from this), which is failing the regression runs.

^
SyntaxError: invalid token
+ RET=1
+ '[' 1 '!=' 0 ']'
+ exit 1
Build step 'Ex?cuter un script shell' marked build as failure
Finished: FAILURE
/snip

  Requesting you to take look into it.

Which Jenkins slave was this? Got a link to the job that failed?

This looks again like a NetBSD slave where logs from regression tests
are overwriting random files. The /usr/pkg/lib/python2.7/site.py file
should be valid Python, and not contain these logs...

Anyone has any ideas why this happens?

Thanks,
Niels


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression failures

[Avra Sengupta]

Lot of test runs are failing with the following:

+ '/opt/qa/build.sh'
  File /usr/pkg/lib/python2.7/site.py, line 601
[2015-08-19 05:45:06.N]:++ G_LOG:./tests/basic/quota-anon-fd-nfs.t: 
TEST: 85 ! fd_write 3 content ++
   ^
SyntaxError: invalid token
+ RET=1

Does it make sense to anybody.

Regards,
Avra

On 08/19/2015 12:28 PM, Emmanuel Dreyfus wrote:

Kotresh Hiremath Ravishankar khire...@redhat.com wrote:


Since the geo-rep regression tests are failing only in NetBSD, Is there
a way we can mask it's run only in NetBSD and let it run in linux?
I am working on geo-rep issues with NetBSD. Once these are fixed we can
enable on NetBSD as well.

Yes, I can wipe them from regression.sh before running the tests, like
we do for tests/bugs (never ported), tests/basic/tier/tier.t  and
tests/basic/ec (the two later used to pass but started exhibiting too
much spurious failures).



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [release-3.6] compile error: 'GF_REPLACE_OP_START' undeclared

[Avra Sengupta]

Still hitting this on freebsd and netbsd smoke runs on release 3.6 
branch. Are we merging patches on release 3.6 branch for now even with 
these failures. I have two such patches that need to be merged.


Regards,
Avra

On 07/06/2015 02:32 PM, Niels de Vos wrote:

On Mon, Jul 06, 2015 at 02:19:07PM +0530, Raghavendra Bhat wrote:

On 07/06/2015 01:39 PM, Niels de Vos wrote:

On Mon, Jul 06, 2015 at 12:09:28PM +0530, Raghavendra Bhat wrote:

On 07/06/2015 09:52 AM, Kaushal M wrote:

I checked on NetBSD-7.0_BETA and FreeBSD-10.1. I couldn't reproduce
this. I'll try on NetBSD-6 next.

~kaushal

I think it has to be included before 3.6.4 is made G.A. I can wait till the
fix for this issue is merged before making 3.6.4. Does it sound ok? Or
should I go ahead with 3.6.4 and make a quick 3.6.5 with this fix?

I only care about getting http://review.gluster.org/11335 merged :-)

This is a patch I promised to take into release-3.5. It would be nicer
to have this change included in the release-3.6 branch before I merge
the 3.5 backport. At the moment, 3.5.5 is waiting on this patch. But I
do not think you really need to delay 3.6.4 off for that one. It should
be fine if it lands in 3.6.5. (The compile error looks more like a 3.6.4
blocker.)

Niels

Niels,

The patch you mentioned has received the acks and also has passed the linux
regression tests. But it seem to have failed netbsd regression tests.

Yes, at least the smoke tests on NetBSD and FreeBSD fail with the
compile error mentioned in the subject of this email :)

Thanks,
Niels



Regards,
Raghavendra Bhat


Regards,
Raghavendra Bhat


On Mon, Jul 6, 2015 at 8:38 AM, Kaushal M kshlms...@gmail.com wrote:

Krutika hit this last week, and let us (GlusterD maintiners) know of
it. I volunteered to look into this, but couldn't find time. I'll do
it now.

~kaushal

On Sun, Jul 5, 2015 at 10:43 PM, Atin Mukherjee
atin.mukherje...@gmail.com wrote:

I remember Krutika reporting it few days back. So it seems like its not
fixed yet. If there is no taker I will send a patch tomorrow.

-Atin
Sent from one plus one

On Jul 5, 2015 9:58 PM, Niels de Vos nde...@redhat.com wrote:

Hi,

it seems that the current release-3.6 branch does not compile on
FreedBSD and NetBSD (not sure why it compiles on CentOS-6). These errors
are thrown:

   --- glusterd_la-glusterd-op-sm.lo ---
 CC   glusterd_la-glusterd-op-sm.lo

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:
In function 'glusterd_op_start_rb_timer':

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:3685:19:
error: 'GF_REPLACE_OP_START' undeclared (first use in this function)

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:3685:19:
note: each undeclared identifier is reported only once for each function it
appears in

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:
In function 'glusterd_bricks_select_status_volume':

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:5800:34:
warning: unused variable 'snapd'
   *** [glusterd_la-glusterd-op-sm.lo] Error code 1


Could someone send a (pointer to the) backport that addresses this?

Thanks,
Niels


On Sun, Jul 05, 2015 at 08:59:32AM -0700, Gluster Build System (Code
Review) wrote:

Gluster Build System has posted comments on this change.

Change subject: nfs: make it possible to disable nfs.mount-rmtab
..


Patch Set 1: -Verified

Build Failed

http://build.gluster.org/job/compare-bug-version-and-git-branch/9953/ :
SUCCESS

http://build.gluster.org/job/freebsd-smoke/8551/ : FAILURE

http://build.gluster.org/job/smoke/19820/ : SUCCESS

http://build.gluster.org/job/netbsd6-smoke/7808/ : FAILURE

--
To view, visit http://review.gluster.org/11335
To unsubscribe, visit http://review.gluster.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d
Gerrit-PatchSet: 1
Gerrit-Project: glusterfs
Gerrit-Branch: release-3.6
Gerrit-Owner: Niels de Vos nde...@redhat.com
Gerrit-Reviewer: Gluster Build System jenk...@build.gluster.com
Gerrit-Reviewer: Kaleb KEITHLEY kkeit...@redhat.com
Gerrit-Reviewer: NetBSD Build System jenk...@build.gluster.org
Gerrit-Reviewer: Niels de Vos nde...@redhat.com
Gerrit-Reviewer: Raghavendra Bhat rab...@redhat.com
Gerrit-Reviewer: jiffin tony Thottan jthot...@redhat.com
Gerrit-HasComments: No

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [release-3.6] compile error: 'GF_REPLACE_OP_START' undeclared

[Avra Sengupta]

+ Adding Raghavendra Bhat.

When is the next GA planned on this branch? And can we take patches in 
this branch while this is being investigated.


Regards,
Avra

On 08/18/2015 12:07 PM, Avra Sengupta wrote:
Still hitting this on freebsd and netbsd smoke runs on release 3.6 
branch. Are we merging patches on release 3.6 branch for now even with 
these failures. I have two such patches that need to be merged.


Regards,
Avra

On 07/06/2015 02:32 PM, Niels de Vos wrote:

On Mon, Jul 06, 2015 at 02:19:07PM +0530, Raghavendra Bhat wrote:

On 07/06/2015 01:39 PM, Niels de Vos wrote:

On Mon, Jul 06, 2015 at 12:09:28PM +0530, Raghavendra Bhat wrote:

On 07/06/2015 09:52 AM, Kaushal M wrote:

I checked on NetBSD-7.0_BETA and FreeBSD-10.1. I couldn't reproduce
this. I'll try on NetBSD-6 next.

~kaushal
I think it has to be included before 3.6.4 is made G.A. I can wait 
till the
fix for this issue is merged before making 3.6.4. Does it sound 
ok? Or

should I go ahead with 3.6.4 and make a quick 3.6.5 with this fix?

I only care about getting http://review.gluster.org/11335 merged :-)

This is a patch I promised to take into release-3.5. It would be nicer
to have this change included in the release-3.6 branch before I merge
the 3.5 backport. At the moment, 3.5.5 is waiting on this patch. But I
do not think you really need to delay 3.6.4 off for that one. It 
should
be fine if it lands in 3.6.5. (The compile error looks more like a 
3.6.4

blocker.)

Niels

Niels,

The patch you mentioned has received the acks and also has passed 
the linux

regression tests. But it seem to have failed netbsd regression tests.

Yes, at least the smoke tests on NetBSD and FreeBSD fail with the
compile error mentioned in the subject of this email :)

Thanks,
Niels



Regards,
Raghavendra Bhat


Regards,
Raghavendra Bhat

On Mon, Jul 6, 2015 at 8:38 AM, Kaushal M kshlms...@gmail.com 
wrote:
Krutika hit this last week, and let us (GlusterD maintiners) 
know of
it. I volunteered to look into this, but couldn't find time. 
I'll do

it now.

~kaushal

On Sun, Jul 5, 2015 at 10:43 PM, Atin Mukherjee
atin.mukherje...@gmail.com wrote:
I remember Krutika reporting it few days back. So it seems like 
its not

fixed yet. If there is no taker I will send a patch tomorrow.

-Atin
Sent from one plus one

On Jul 5, 2015 9:58 PM, Niels de Vos nde...@redhat.com wrote:

Hi,

it seems that the current release-3.6 branch does not compile on
FreedBSD and NetBSD (not sure why it compiles on CentOS-6). 
These errors

are thrown:

   --- glusterd_la-glusterd-op-sm.lo ---
 CC   glusterd_la-glusterd-op-sm.lo

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c: 


In function 'glusterd_op_start_rb_timer':

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:3685:19: 

error: 'GF_REPLACE_OP_START' undeclared (first use in this 
function)


/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:3685:19: 

note: each undeclared identifier is reported only once for 
each function it

appears in

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c: 


In function 'glusterd_bricks_select_status_volume':

/home/jenkins/root/workspace/netbsd6-smoke/xlators/mgmt/glusterd/src/glusterd-op-sm.c:5800:34: 


warning: unused variable 'snapd'
   *** [glusterd_la-glusterd-op-sm.lo] Error code 1


Could someone send a (pointer to the) backport that addresses 
this?


Thanks,
Niels


On Sun, Jul 05, 2015 at 08:59:32AM -0700, Gluster Build System 
(Code

Review) wrote:

Gluster Build System has posted comments on this change.

Change subject: nfs: make it possible to disable nfs.mount-rmtab
.. 




Patch Set 1: -Verified

Build Failed

http://build.gluster.org/job/compare-bug-version-and-git-branch/9953/ 
:

SUCCESS

http://build.gluster.org/job/freebsd-smoke/8551/ : FAILURE

http://build.gluster.org/job/smoke/19820/ : SUCCESS

http://build.gluster.org/job/netbsd6-smoke/7808/ : FAILURE

--
To view, visit http://review.gluster.org/11335
To unsubscribe, visit http://review.gluster.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I40c4d8d754932f86fb2b1b2588843390464c773d
Gerrit-PatchSet: 1
Gerrit-Project: glusterfs
Gerrit-Branch: release-3.6
Gerrit-Owner: Niels de Vos nde...@redhat.com
Gerrit-Reviewer: Gluster Build System 
jenk...@build.gluster.com

Gerrit-Reviewer: Kaleb KEITHLEY kkeit...@redhat.com
Gerrit-Reviewer: NetBSD Build System jenk...@build.gluster.org
Gerrit-Reviewer: Niels de Vos nde...@redhat.com
Gerrit-Reviewer: Raghavendra Bhat rab...@redhat.com
Gerrit-Reviewer: jiffin tony Thottan jthot...@redhat.com
Gerrit-HasComments: No

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing

Re: [Gluster-devel] [Gluster-users] Backup bricks?

[Avra Sengupta]

Yes for long term backups LVM snapshots might not be the solution. There 
is no side effect in backing up the bricks. The data would indeed be 
readable. And if you back up /var/lib/glusterd/vols/volname on each 
volume as well, you can effectively recreate the volume from the bricks 
at a later stage.


Regards,
Avra

On 08/17/2015 04:11 PM, Thibault Godouet wrote:


Thanks Avra.

I am aware of the Gluster snapshots, but didn't think about using them 
on the offsite replica.  That could indeed cover the short term 
backups, and be used to do longer term backups from.


What I perhaps wasn't clear about is that we'll need longer term 
backups to tape (e.g. to keep multiple years).  I don't think keeping 
LVM snapshots for that long would really work.
So basically my initial question was on whether backing up the brick 
instead of the volume, which would be significantly faster, would be a 
good idea: would the data be readable ok? Any known side effect that 
could cause issues?


On 17 Aug 2015 10:12 am, Avra Sengupta aseng...@redhat.com 
mailto:aseng...@redhat.com wrote:


Hi Thibault,

Instead of backing up, individual bricks or the entire thin
logical volume, you can take a gluster volume snapshot, and you
will have a point in time backup of the volume.

gluster snapshots internally use thin lv snapshots, so you can't
move the backup out of the system. Also having the backup on the
same filesystem as the data doesn't protect you from device
failure scenarios. However in events of any other data loss or
corruption, you can restore the volume from the snapshot, mount
the read-only snapshot and copy the necessary files.

In order to take backup at a remore site, using geo-rep is
recommended.

Regards,
Avra

On 08/17/2015 02:27 PM, Thibault Godouet wrote:


I have a 1 x 2 = 2 volume geo-replicated to a single-brick volume
in another physical site, where I would like to set up a backup.

I could setup a backup on a mount of the volume, but a quick test
shows it is slow in this setup (presumably because there are
loads of small files on there).

Instead I thought I could maybe backup the filesystem where the
brick is (or rather a snapshot of the thin logical volume).  My
understanding is that all the files will be in there, and
readable, so it seems to me it would be fine to back things up
from there.

Is that right, or am I missing something here?

Note the .glusterfs directory would also be backed up too,
although I'm not sure whether that would be of any use in a backup.

More generally is there a recommended way to setup backups?

Thanks,
Thibault.



___
Gluster-users mailing list
gluster-us...@gluster.org mailto:gluster-us...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression failures

[Avra Sengupta]

On 08/18/2015 09:25 AM, Atin Mukherjee wrote:


On 08/17/2015 02:20 PM, Avra Sengupta wrote:

That patch itself might not pass all regressions as it might fail at the
geo-rep test. I have sent a patch (http://review.gluster.org/#/c/11934/)
with both the tests being moved to bad test. Talur could you please
abandon 11933.

It seems like we need to move tests/geo-rep/georep-basic-dr-tarssh.t as
well to the bad test?

Yes looks like it. I will resend the patch with this change.

Regards,
Avra

On 08/17/2015 02:12 PM, Atin Mukherjee wrote:

tests/basic/mount-nfs-auth.t has been already been added to bad test by
http://review.gluster.org/11933

~Atin

On 08/17/2015 02:09 PM, Avra Sengupta wrote:

Will send a patch moving ./tests/basic/mount-nfs-auth.t and
./tests/geo-rep/georep-basic-dr-rsync.t to bad test.

Regards,
Avra

On 08/17/2015 12:45 PM, Avra Sengupta wrote:

On 08/17/2015 12:29 PM, Vijaikumar M wrote:

On Monday 17 August 2015 12:22 PM, Avra Sengupta wrote:

Hi,

The NetBSD regression tests are continuously failing with errors in
the following tests:

./tests/basic/mount-nfs-auth.t
./tests/basic/quota-anon-fd-nfs.t

quota-anon-fd-nfs.t is known issues with NFS client caching so it is
marked as bad test, final test will be marked as success even if this
test fails.

Yes it seems ./tests/geo-rep/georep-basic-dr-rsync.t also fails in
the runs where quota-anon-fd-nfs.t fails, and that marks the final
tests as failure.




Is there any recent change that is trigerring this behaviour. Also
currently one machine is running NetBSD tests. Can someone with
access to Jenkins, bring up a few more slaves to run NetBSD
regressions in parallel.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] NetBSD regression failures

[Avra Sengupta]

Hi,

The NetBSD regression tests are continuously failing with errors in the 
following tests:


./tests/basic/mount-nfs-auth.t
./tests/basic/quota-anon-fd-nfs.t

Is there any recent change that is trigerring this behaviour. Also 
currently one machine is running NetBSD tests. Can someone with access 
to Jenkins, bring up a few more slaves to run NetBSD regressions in 
parallel.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression failures

[Avra Sengupta]

On 08/17/2015 12:29 PM, Vijaikumar M wrote:



On Monday 17 August 2015 12:22 PM, Avra Sengupta wrote:

Hi,

The NetBSD regression tests are continuously failing with errors in 
the following tests:


./tests/basic/mount-nfs-auth.t
./tests/basic/quota-anon-fd-nfs.t
quota-anon-fd-nfs.t is known issues with NFS client caching so it is 
marked as bad test, final test will be marked as success even if this 
test fails.
Yes it seems ./tests/geo-rep/georep-basic-dr-rsync.t also fails in the 
runs where quota-anon-fd-nfs.t fails, and that marks the final tests as 
failure.








Is there any recent change that is trigerring this behaviour. Also 
currently one machine is running NetBSD tests. Can someone with 
access to Jenkins, bring up a few more slaves to run NetBSD 
regressions in parallel.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression failures

[Avra Sengupta]

Will send a patch moving ./tests/basic/mount-nfs-auth.t and 
./tests/geo-rep/georep-basic-dr-rsync.t to bad test.


Regards,
Avra

On 08/17/2015 12:45 PM, Avra Sengupta wrote:

On 08/17/2015 12:29 PM, Vijaikumar M wrote:



On Monday 17 August 2015 12:22 PM, Avra Sengupta wrote:

Hi,

The NetBSD regression tests are continuously failing with errors in 
the following tests:


./tests/basic/mount-nfs-auth.t
./tests/basic/quota-anon-fd-nfs.t
quota-anon-fd-nfs.t is known issues with NFS client caching so it is 
marked as bad test, final test will be marked as success even if this 
test fails.
Yes it seems ./tests/geo-rep/georep-basic-dr-rsync.t also fails in 
the runs where quota-anon-fd-nfs.t fails, and that marks the final 
tests as failure.








Is there any recent change that is trigerring this behaviour. Also 
currently one machine is running NetBSD tests. Can someone with 
access to Jenkins, bring up a few more slaves to run NetBSD 
regressions in parallel.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression failures

[Avra Sengupta]

That patch itself might not pass all regressions as it might fail at the 
geo-rep test. I have sent a patch (http://review.gluster.org/#/c/11934/) 
with both the tests being moved to bad test. Talur could you please 
abandon 11933.


Regards,
Avra

On 08/17/2015 02:12 PM, Atin Mukherjee wrote:

tests/basic/mount-nfs-auth.t has been already been added to bad test by
http://review.gluster.org/11933

~Atin

On 08/17/2015 02:09 PM, Avra Sengupta wrote:

Will send a patch moving ./tests/basic/mount-nfs-auth.t and
./tests/geo-rep/georep-basic-dr-rsync.t to bad test.

Regards,
Avra

On 08/17/2015 12:45 PM, Avra Sengupta wrote:

On 08/17/2015 12:29 PM, Vijaikumar M wrote:


On Monday 17 August 2015 12:22 PM, Avra Sengupta wrote:

Hi,

The NetBSD regression tests are continuously failing with errors in
the following tests:

./tests/basic/mount-nfs-auth.t
./tests/basic/quota-anon-fd-nfs.t

quota-anon-fd-nfs.t is known issues with NFS client caching so it is
marked as bad test, final test will be marked as success even if this
test fails.

Yes it seems ./tests/geo-rep/georep-basic-dr-rsync.t also fails in
the runs where quota-anon-fd-nfs.t fails, and that marks the final
tests as failure.





Is there any recent change that is trigerring this behaviour. Also
currently one machine is running NetBSD tests. Can someone with
access to Jenkins, bring up a few more slaves to run NetBSD
regressions in parallel.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] NSR in Gluster.Next

[Avra Sengupta]

Hi Jeff,

I am looking into the NSR feature for Gluster.Next. Currently I have 
started going through the feature page, and the design 
(http://review.gluster.org/#/c/8915/), and the current NSR 
codebase(patches pointed to in the feature page). Could you point me to 
any other documentation I might have missed, or a primer of sorts I 
should be looking into.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Non-Uniform opErrstr output in all gluster --xml commands

[Avra Sengupta]

On 08/05/2015 03:06 PM, Atin Mukherjee wrote:


On 08/05/2015 02:58 PM, Avra Sengupta wrote:

Hi,

As reported in https://bugzilla.redhat.com/show_bug.cgi?id=1218732, in
the event where there is no opErrstr, some gluster commands'(like
snapshot status, volume status etc.) xml output shows
opErrstr(null)/opErrstr, while other commands show just
opErrstr/. This non-uniform output is troublesome for people who
parse this.

IMHO showing opErrstr(null)/opErrstr is much more descriptive. I
would like to propose making sure all commands follow this uniform
approach, instead of displaying just opErrstr/.

Makes sense to me as long as its uniform. However the existing ones
which follow opErrstr/ need change in parsing logic :
To attain uniformity, one or the other parsing logic will break. we 
might as well get it right for once.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Non-Uniform opErrstr output in all gluster --xml commands

[Avra Sengupta]

In that case will stick with opErrstr/ for all the null elements.

On 08/05/2015 04:10 PM, Prashanth Pai wrote:

Having (null) is not common in xml convention. Usually, it's either

opErrstr/

or

opErrstr/opErrstr

Regards,
  -Prashanth Pai

- Original Message -

From: Avra Sengupta aseng...@redhat.com
To: Atin Mukherjee amukh...@redhat.com, Gluster Devel 
gluster-devel@gluster.org, gluster-us...@gluster.org,
kauff...@cs.uchicago.edu, Dusmant Kumar Pati dp...@redhat.com, Shubhendu 
Tripathi shtri...@redhat.com,
msvb...@redhat.com
Sent: Wednesday, August 5, 2015 3:16:10 PM
Subject: Re: [Gluster-devel] Non-Uniform opErrstr output in all gluster --xml 
commands

On 08/05/2015 03:06 PM, Atin Mukherjee wrote:

On 08/05/2015 02:58 PM, Avra Sengupta wrote:

Hi,

As reported in https://bugzilla.redhat.com/show_bug.cgi?id=1218732, in
the event where there is no opErrstr, some gluster commands'(like
snapshot status, volume status etc.) xml output shows
opErrstr(null)/opErrstr, while other commands show just
opErrstr/. This non-uniform output is troublesome for people who
parse this.

IMHO showing opErrstr(null)/opErrstr is much more descriptive. I
would like to propose making sure all commands follow this uniform
approach, instead of displaying just opErrstr/.

Makes sense to me as long as its uniform. However the existing ones
which follow opErrstr/ need change in parsing logic :

To attain uniformity, one or the other parsing logic will break. we
might as well get it right for once.

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GlusterD 2.0 2-3 months plan

[Avra Sengupta]

Hi,

I have a few queries. Some of them might be pre-mature given that this 
is just the initial mail scoping the plan, but nevertheless please find 
them inline


Regards,
Avra

On 08/04/2015 02:57 PM, Krishnan Parthasarathi wrote:

# GlusterD 2.0 plan (Aug-Oct '15)

[This text in this email structured using markdown format]

This document outlines efforts planned towards [thousand-node-glusterd][1] in
the coming 2-3 months. The following email thread on gluster-devel provides
context on previous discussions around glusterd-2.0 -
http://www.gluster.org/pipermail/gluster-users/2014-September/018639.html

## High-level tasks

1. Define a repeatable environment using Vagrant+ansible.

 - Encourages early adoption.

1. Introduction of etcd/consul to store configuration information.

 - Define deployment steps where applicable, targeting existing
   gluster-users.

 - Add admin guide doc.
Is this configuration store going to be closely tied with the below 
mentioned transaction framework? As in are commands which are not ported 
to the transaction framework, still gonna be able to leverage this?


1. Build a transaction framework to support implementation of gluster commands.

 - Define interfaces for commands to integrate with the transaction
   framework.

 - Add developer doc for porting commands into this framework - `Porting 
Guide`.
We currently have 3 transaction frameworks in the present glusterd, and 
each one of them have their own set of issues (repeated storage on 
individual nodes, handshake, peer management, rollover of failed 
operation, high availability, non-modular code fragments, and lack of 
better logging being the most excruciating pain points). This is a great 
oppurtunity for us to address all these issues, while we are designing a 
new transaction framework, and even though implementing everything at 
one shot might not be likely we should opt for a design which would 
cater to all these needs. Awaiting your design publication :)


1. Implement basic volume management and peer commands.

 - e.g, volume-create, volume-start, volume-add-brick, volume-remove-brick,
   volume-stop, volume-delete, peer-probe, peer-detach and peer-status.

 - Add some of these to the `Porting Guide`.

## Progress on current activities

1. Setting up codebase
 - Has consul as the default configuration store

 - Uses [libkv][2] library to interface our store. Makes it possible to 
move to
   other options if required.


 - Begun implementing REST endpoints defined by [heketi][3].

 - Begun implementing volume file generation in Go; This support existing
   volume configurations only. Other volume file generation proposals are
   out of the current scope.
Can you please comment on the decision for choosing go as the language. 
glusterd being the management interface interacts with almost all of 
them, in a very intrusive manner. How adaptable will Go be, in a heavy C 
laden gluster community, where every component today has atleast more 
than 90% of the code writen in C.


2. Cross language RPC framework
 - Exploring the following [options][4]

 - gRPC - This is really interesting and provides a lot of features.
   But this is currently in alpha and also requires the use of
   protobuf3, which itself is in alpha.

 - Apache Thrift - The C implementation requires the use of Glib,
   which we are not comfortable with and feel is a little too much for
   what we need.

 - JSON-RPC - This works wonderfully well with Go. But requires a lot
   of manual boiler-plate code writing for C to do the serialization and
   deserialization.

 - Properties that we are looking for in frameworks

 - Code generation for serialization

 - Optional stub code generation for the RPC services themselves

 - Ease of programming in the framework

## Activities lined up

1.  Setting up development environment

 - We choose to use Vagrant+Ansible.

2.  Settle down on a cross language RPC framework

 - This also involves getting RPC handlers in glusterfs codebase.

3.  Design/Prototype a transaction framework

 - Aim to make integrating new features easier than in current glusterd.

 - Should make porting of existing commands simple.

[1]: 
http://www.gluster.org/community/documentation/index.php/Features/thousand-node-glusterd
[2]: https://github.com/docker/libkv
[3]: https://github.com/heketi/heketi/wiki/API
[4]: http://www.gluster.org/pipermail/gluster-devel/2015-August/046307.html

~GlusterD team
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [FAILED] regression tests: tests/bugs/distribute/bug-1066798.t, tests/basic/volume-snapshot.t

[Avra Sengupta]

The particular slave (slave21) containing the cores is down. I however 
have access to slave0, so trying to recreate it on that slave and will 
analyze the core when I get it.


Regards,
Avra

On 07/20/2015 03:19 PM, Ravishankar N wrote:
One more core for volume-snapshot.t: 
https://build.gluster.org/job/rackspace-regression-2GB-triggered/12605/consoleFull


On 07/20/2015 03:00 PM, Raghavendra Talur wrote:

Adding Susant and Avra for dht and snapshot test cases respectively.


On Mon, Jul 20, 2015 at 11:45 AM, Milind Changire 
milindchang...@gmail.com wrote:



http://build.gluster.org/job/rackspace-regression-2GB-triggered/12541/consoleFull


http://build.gluster.org/job/rackspace-regression-2GB-triggered/12499/consoleFull


Please advise.

--
Milind


___
Gluster-devel mailing list
Gluster-devel@gluster.org mailto:Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




--
*Raghavendra Talur *



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [FAILED] regression tests: tests/bugs/distribute/bug-1066798.t, tests/basic/volume-snapshot.t

[Avra Sengupta]

/rpc-lib/src/rpcsvc.c:699
#15 0x7f8f3d6173e0 in rpcsvc_notify (trans=0x7f8f20008e20, 
mydata=0x24b8430, event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f8f18000e20)
at 
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpcsvc.c:793
#16 0x7f8f3d61caeb in rpc_transport_notify (this=0x7f8f20008e20, 
event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f8f18000e20)
at 
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-transport.c:538
#17 0x7f8f3134187b in socket_event_poll_in (this=0x7f8f20008e20) at 
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2285
#18 0x7f8f31341dd1 in socket_event_handler (fd=20, idx=10, 
data=0x7f8f20008e20, poll_in=1, poll_out=0, poll_err=0)
at 
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2398
#19 0x7f8f3d8d09e4 in event_dispatch_epoll_handler 
(event_pool=0x249ec90, event=0x7f8f2dfb5e70) at 
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:570
#20 0x7f8f3d8d0dd2 in event_dispatch_epoll_worker (data=0x24a97f0) 
at 
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:673

#21 0x7f8f3cb379d1 in start_thread () from ./lib64/libpthread.so.0
#22 0x7f8f3c4a18fd in clone () from ./lib64/libc.so.6

Regards,
Avra


On 07/20/2015 04:38 PM, Avra Sengupta wrote:
The particular slave (slave21) containing the cores is down. I however 
have access to slave0, so trying to recreate it on that slave and will 
analyze the core when I get it.


Regards,
Avra

On 07/20/2015 03:19 PM, Ravishankar N wrote:
One more core for volume-snapshot.t: 
https://build.gluster.org/job/rackspace-regression-2GB-triggered/12605/consoleFull


On 07/20/2015 03:00 PM, Raghavendra Talur wrote:

Adding Susant and Avra for dht and snapshot test cases respectively.


On Mon, Jul 20, 2015 at 11:45 AM, Milind Changire 
milindchang...@gmail.com wrote:



http://build.gluster.org/job/rackspace-regression-2GB-triggered/12541/consoleFull


http://build.gluster.org/job/rackspace-regression-2GB-triggered/12499/consoleFull


Please advise.

--
Milind


___
Gluster-devel mailing list
Gluster-devel@gluster.org mailto:Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




--
*Raghavendra Talur *



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Testcase '/tests/bugs/snapshot/bug-1109889.t' failing

[Avra Sengupta]

Using slave0.cloud.gluster.org to debug the issue.

Regards,
Avra

On 07/13/2015 12:38 PM, Atin Mukherjee wrote:

Failed for me as well @
http://build.gluster.org/job/rackspace-regression-2GB-triggered/12314/consoleFull

On 07/11/2015 09:59 AM, Kaushal M wrote:

This test still keeps failing [1] spuriously even after the fix got
merged. Could you look into this?

[1]: 
http://build.gluster.org/job/rackspace-regression-2GB-triggered/12242/consoleFull

On Thu, Jul 9, 2015 at 10:15 AM, Avra Sengupta aseng...@redhat.com wrote:

Sent a patch for this yesterday. http://review.gluster.org/#/c/11579/. Could
someone please take this in

Regards,
Avra


On 07/08/2015 08:31 PM, Joseph Fernandes wrote:

Hit again ! :(


http://build.gluster.org/job/rackspace-regression-2GB-triggered/12098/consoleFull
for my patch http://review.gluster.org/#/c/11577/


- Original Message -
From: Raghavendra Gowdappa rgowd...@redhat.com
To: Nithya Balachandran nbala...@redhat.com
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Wednesday, July 8, 2015 3:13:06 PM
Subject: Re: [Gluster-devel]Testcase
'/tests/bugs/snapshot/bug-1109889.t'failing



- Original Message -

From: Nithya Balachandran nbala...@redhat.com
To: Vijaikumar M vmall...@redhat.com
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Wednesday, July 8, 2015 12:17:29 PM
Subject: Re: [Gluster-devel] Testcase
'/tests/bugs/snapshot/bug-1109889.t'failing

Also failing on:

http://build.gluster.org/job/rackspace-regression-2GB-triggered/12069/consoleFull

Regards,
Nithya

- Original Message -

From: Vijaikumar M vmall...@redhat.com
To: Gluster Devel gluster-devel@gluster.org, Avra Sengupta
aseng...@redhat.com, Rajesh Joseph
rjos...@redhat.com
Sent: Wednesday, 8 July, 2015 11:09:14 AM
Subject: [Gluster-devel] Testcase '/tests/bugs/snapshot/bug-1109889.t'
 failing

Hi,

Testcase '/tests/bugs/snapshot/bug-1109889.t' is failing consistently

Earlier this test was resulting a brick crash because of brick accepting
fops even before its xlator graph is initialised. A recent fix makes server
to reject any client connections till xlator graph is initialised. I think
stat is failing with ENOTCONN because of that. Now the question is why/how
client is trying to connect before server is initialized. Hope that helps.


http://build.gluster.org/job/rackspace-regression-2GB-triggered/12048/consoleFull

I think below test at line# 72 need to be changed:

TEST stat $M0/.snaps;

To

EXPECT_WITHIN $PROCESS_UP_TIMEOUT 0 STAT $M0/.snaps


Thanks,
Vijay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Testcase '/tests/bugs/snapshot/bug-1109889.t' failing

[Avra Sengupta]

Sent a patch for this yesterday. http://review.gluster.org/#/c/11579/. 
Could someone please take this in


Regards,
Avra

On 07/08/2015 08:31 PM, Joseph Fernandes wrote:

Hit again ! :(

http://build.gluster.org/job/rackspace-regression-2GB-triggered/12098/consoleFull
for my patch http://review.gluster.org/#/c/11577/


- Original Message -
From: Raghavendra Gowdappa rgowd...@redhat.com
To: Nithya Balachandran nbala...@redhat.com
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Wednesday, July 8, 2015 3:13:06 PM
Subject: Re: [Gluster-devel]Testcase
'/tests/bugs/snapshot/bug-1109889.t'failing



- Original Message -

From: Nithya Balachandran nbala...@redhat.com
To: Vijaikumar M vmall...@redhat.com
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Wednesday, July 8, 2015 12:17:29 PM
Subject: Re: [Gluster-devel] Testcase   '/tests/bugs/snapshot/bug-1109889.t'
failing

Also failing on:
http://build.gluster.org/job/rackspace-regression-2GB-triggered/12069/consoleFull

Regards,
Nithya

- Original Message -

From: Vijaikumar M vmall...@redhat.com
To: Gluster Devel gluster-devel@gluster.org, Avra Sengupta
aseng...@redhat.com, Rajesh Joseph
rjos...@redhat.com
Sent: Wednesday, 8 July, 2015 11:09:14 AM
Subject: [Gluster-devel] Testcase '/tests/bugs/snapshot/bug-1109889.t'
failing

Hi,

Testcase '/tests/bugs/snapshot/bug-1109889.t' is failing consistently

Earlier this test was resulting a brick crash because of brick accepting fops 
even before its xlator graph is initialised. A recent fix makes server to 
reject any client connections till xlator graph is initialised. I think stat is 
failing with ENOTCONN because of that. Now the question is why/how client is 
trying to connect before server is initialized. Hope that helps.


http://build.gluster.org/job/rackspace-regression-2GB-triggered/12048/consoleFull

I think below test at line# 72 need to be changed:

TEST stat $M0/.snaps;

To

EXPECT_WITHIN $PROCESS_UP_TIMEOUT 0 STAT $M0/.snaps


Thanks,
Vijay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Brick path used for gluster shared storage volume

[Avra Sengupta]

Hi,

Today with with enabling volume set option 
cluster.enable-shared-storage, we create a shared storage volume called 
gluster_shared_storage for the user, and mount it on all the nodes in 
the cluster. Currently this volume is used for features like 
nfs-ganesha, snapshot scheduler and geo-replication to save some 
internal data required by these features. The brick path we use to 
create this shared storage is /var/run/gluster/ss_brick.


The problem with using this brick path is /var/run/gluster is a tmpfs 
and all the brick/shared storage data will be wiped off when the node 
restarts. Hence I propose using /var/lib/glusterd/ss_brick as the brick 
path for shared storage volume as this brick and the shared storage 
volume is internally created by us (albeit on user's request), and 
contains only internal state data and no user data.


We are also aware that admins sometime take backup of /var/lib/glusterd 
to save the current state of gluster. Again this shouldn't be an issue 
as the data contained in these bricks is only internal state data and is 
very minimal.


Please let me know if there are any issues or concerns with using 
/var/lib/glusterd/ss_brick as the brick path for the shared storage, and 
also suggest an alternate brick path.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] NetBSD regressions not being triggered for patches

[Avra Sengupta]

On 06/17/2015 12:12 PM, Rajesh Joseph wrote:


- Original Message -

From: Kaushal M kshlms...@gmail.com
To: Emmanuel Dreyfus m...@netbsd.org
Cc: Gluster Devel gluster-devel@gluster.org, gluster-infra 
gluster-in...@gluster.org
Sent: Wednesday, 17 June, 2015 11:59:22 AM
Subject: Re: [Gluster-devel] [Gluster-infra] NetBSD regressions not being 
triggered for patches

cloud.gluster.org is served by Rackspace Cloud DNS. AFAICT, there is
no readily available option to do zone transfers from it. We might
have to contact the Rackspace support to find out if they can do it as
a special request.


If this is going to take time then I prefer not to block patches for NetBSD. We 
can address
any NetBSD regression caused by patches as a separate bug. Otherwise our 
regression queue will
continue to grow.
+1 for this. We shouldn't be blocking patches for NetBSD regression till 
the infra scales enough to handle the kind of load we are throwing at 
it. Once the regression framework is scalable enough, we can fix any 
regressions (if any) introduced. This will bring down the turnaround 
time, for the patch acceptance.



On Wed, Jun 17, 2015 at 11:50 AM, Emmanuel Dreyfus m...@netbsd.org wrote:

Venky Shankar yknev.shan...@gmail.com wrote:


If that's the case, then I'll vote for this even if it takes some time
to get things in workable state.

See my other mail about this: you enter a new slave VM in the DNS and it
does not resolve, or somethimes you get 20s delays. I am convinced this
is the reason why Jenkins bugs.

--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org
___
Gluster-infra mailing list
gluster-in...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-infra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Patch Needs Merging

[Avra Sengupta]

Hi,

Could you please merge the following release 3.7 patches

http://review.gluster.org/#/c/11151/
http://review.gluster.org/#/c/11159/

Regards,
Avra

On 06/11/2015 01:24 AM, Avra Sengupta wrote:

Hi,

Could you please merge the following patch:

http://review.gluster.org/#/c/11139/

Regards,
Avra


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] devrpm and smoke test fails to start

[Avra Sengupta]

Hi,

I am getting devrpm failure and smoke failures for the patch 
http://review.gluster.org/#/c/11139/. The failures are of the nature 
that the test itself doesn't initiate. For example: 
http://build.gluster.org/job/glusterfs-devrpms-el7/2527/console


I tried to re-trigger just these tests, without re-submitting the patch 
coz I don't want to lose the +1s already given by rackspace regression 
and netbsd regression, but am unable to re-trigger it. Can someone 
please do the same for me.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Patch needs merging

[Avra Sengupta]

Hi,

Can you please merge the following patches:

http://review.gluster.org/#/c/11087/

Regards,
Avra

On 06/09/2015 08:06 PM, Avra Sengupta wrote:

Thanks KP :)

On 06/09/2015 07:51 PM, Krishnan Parthasarathi wrote:

http://review.gluster.org/#/c/11042/
http://review.gluster.org/#/c/11100/

Merged.




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Patch Needs Merging

[Avra Sengupta]

Hi,

Could you please merge the following patch:

http://review.gluster.org/#/c/11139/

Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Unable to send patches to gerrit

[Avra Sengupta]

+Adding gluster-infra

On 06/11/2015 10:39 AM, Pranith Kumar Karampuri wrote:
Last time when this happened Kaushal/vijay fixed it if I remember 
correctly.

+kaushal +Vijay

Pranith
On 06/11/2015 10:38 AM, Anoop C S wrote:


On 06/11/2015 10:33 AM, Ravishankar N wrote:

I'm unable to push a patch on release-3.6, getting different
errors every time:


This happens for master too. I continuously get the following error:

error: unpack failed: error No space left on device


[ravi@tuxpad glusterfs]$ ./rfc.sh [detached HEAD a59646a] afr:
honour selfheal enable/disable volume set options Date: Sat May 30
10:23:33 2015 +0530 3 files changed, 108 insertions(+), 4
deletions(-) create mode 100644 tests/basic/afr/client-side-heal.t
Successfully rebased and updated
refs/heads/3.6_honour_heal_options. Counting objects: 11, done.
Delta compression using up to 4 threads. Compressing objects: 100%
(11/11), done. Writing objects: 100% (11/11), 1.77 KiB | 0 bytes/s,
done. Total 11 (delta 9), reused 0 (delta 0) *error: unpack failed:
error No space left on device** **fatal: Unpack error, check server
log* To ssh://itisr...@git.gluster.org/glusterfs.git ! [remote
rejected] HEAD - refs/for/release-3.6/bug-1230259 (n/a (unpacker
error)) error: failed to push some refs to
'ssh://itisr...@git.gluster.org/glusterfs.git' [ravi@tuxpad
glusterfs]$


[ravi@tuxpad glusterfs]$ ./rfc.sh [detached HEAD 8b28efd] afr:
honour selfheal enable/disable volume set options Date: Sat May 30
10:23:33 2015 +0530 3 files changed, 108 insertions(+), 4
deletions(-) create mode 100644 tests/basic/afr/client-side-heal.t
Successfully rebased and updated
refs/heads/3.6_honour_heal_options. *fatal: internal server
error** **fatal: Could not read from remote repository.** **
**Please make sure you have the correct access rights** **and the
repository exists.*


Anybody else facing problems? -Ravi



___ Gluster-devel
mailing list Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] NetBSD regressions not being triggered for patches

[Avra Sengupta]

Hi,

New patches being submitted are not getting NetBSD regressions run on 
them. Even manually they are not getting triggered. Is anyone aware of this?


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Patch needs merging

[Avra Sengupta]

Thanks KP :)

On 06/09/2015 07:51 PM, Krishnan Parthasarathi wrote:

http://review.gluster.org/#/c/11042/
http://review.gluster.org/#/c/11100/

Merged.


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Patch needs merging

[Avra Sengupta]

Hi,

Could you please merge the following patches in release 3.7 branch. It's 
got code review +1s and all regressions have passed.


http://review.gluster.org/#/c/11042/
http://review.gluster.org/#/c/11100/

Regards,
Avra


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Failure in volume-snapshot.t

[Avra Sengupta]

Requesting again. Can I get access to one of the slave machines so that 
I can investigate the failure in volume-snapshot.t


Regards,
Avra

On 06/03/2015 12:30 PM, Avra Sengupta wrote:

+ Adding gluster-infra

On 06/03/2015 12:16 PM, Avra Sengupta wrote:

Hi,

Can I get access to one of the slave machines so that I can 
investigate the failure in volume-snapshot.t


Regards,
Avra

On 06/02/2015 07:51 PM, Atin Mukherjee wrote:


Sent from Samsung Galaxy S4
On 2 Jun 2015 18:50, Krutika Dhananjay kdhan...@redhat.com 
mailto:kdhan...@redhat.com wrote:


 volume-snapshot.t failed this time on my patch @ 
http://build.gluster.org/job/rackspace-regression-2GB-triggered/9958/consoleFull , 
at test 39 again.

 A spurious failure perhaps?
Though its spurious but it does raise an alarm. Avra, mind to chip in?

 -Krutika
 

 From: Atin Mukherjee amukh...@redhat.com 
mailto:amukh...@redhat.com
 To: Anand Nekkunti anekk...@redhat.com 
mailto:anekk...@redhat.com, asen  Avra Sengupta 
aseng...@redhat.com mailto:aseng...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org 
mailto:gluster-devel@gluster.org

 Sent: Monday, June 1, 2015 2:45:06 PM
 Subject: Re: [Gluster-devel] Failure in volume-snapshot.t




 On 06/01/2015 11:41 AM, Anand Nekkunti wrote:
  HI Atin
it seems like some spurious fail (failed one time). My patch 
nothing

  to do with  snapshot. I will send my patch with rebase .
 Then, I would request Avra to take a look at it.
  Regards
  Anand.N
 
  On 05/29/2015 07:17 PM, Atin Mukherjee wrote:
  Anand,
 
  Could you check if your patch [1] fails this regression every 
time?

  Otherwise I would request Avra to take a look at [2].
 
  Snapshot delete failed with following error:
 
  snapshot delete: failed: Snapshot 
patchy2_snap1_GMT-2015.05.29-13.40.17

  might not be in an usable state.
  volume delete: patchy2: failed: Staging failed on 127.1.1.3. 
Error:

  Cannot delete Volume patchy2 ,as it has 1 snapshots. To delete the
  volume, first delete all the snapshots under it.
 
  [1] http://review.gluster.org/10586
  [2]
  
http://build.gluster.org/job/rackspace-regression-2GB-triggered/9789/consoleFull

 
 

 --
 ~Atin
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org mailto:Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel



 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org mailto:Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel








___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Will be pushing 3.7.1 tag shortly

[Avra Sengupta]

Aravinda has given it a +1

Regards,
Avra

On 06/01/2015 03:34 PM, Krishnan Parthasarathi wrote:

Could you get a +1 from Aravinda?

- Original Message -

Hi KP,

Can we have this patch in 3.7.1 before you make the release.
http://review.gluster.org/#/c/10993/

Regards,
Avra

On 06/01/2015 02:37 PM, Krishnan Parthasarathi wrote:

All,

I will be pushing GlusterFS 3.7.1 release in couple of hours. This release
is to fix BZ 1223215 - gluster volume status fails with locking failed
error message.
If there are other fixes that cause users unacceptable inconvenience,
please
get your patches merged by respective maintainers in release-3.7 branch.
Reply
to this thread if you won't be able to make it in the next couple of hours.
For those
who won't be able to even read this mail by that time, don't fret, we
expect to
release 3.7.2 in two weeks time.

~kp
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Unable to send patches to release 3.7 branch.

[Avra Sengupta]

Got it. Thanks Niels.

Regards,
Avra

On 05/29/2015 01:44 PM, Niels de Vos wrote:

On Fri, May 29, 2015 at 09:11:22AM +0530, Avra Sengupta wrote:

Hi,

Usually when a patch is backported to release 3.7 branch it contains the
following from the patch already merged in master:

 Change-Id: Ib878f39814af566b9250cf6b8ed47da0ca5b1128
 BUG: 1226120
 Signed-off-by: Avra Sengupta aseng...@redhat.com
 Reviewed-on: http://review.gluster.org/10641
 Reviewed-by: Rajesh Joseph rjos...@redhat.com
 Tested-by: NetBSD Build System
 Reviewed-by: Kaushal M kaus...@redhat.com

While trying to send this patch from release 3.7 branch I am getting the
following error from checkpatch.pl which is not letting me
send the patch, and this is new because it didn't use to happen earlier.

# ./extras/checkpatch.pl
0001-glusterd-snapshot-Return-correct-errno-in-events-of-.patch
Use of uninitialized value $gerrit_url in regexp compilation at
./extras/checkpatch.pl line 1958.
ERROR: Unrecognized url address: 'http://review.gluster.org/10313'
#16:
Reviewed-on: http://review.gluster.org/10313

ERROR: Unrecognized email address: 'NetBSD Build System'
#19:
Tested-by: NetBSD Build System

Patch not according to coding guidelines! please fix.
total: 2 errors, 0 warnings, 326 lines checked

Why is checkpatch unable to reference the gerrit url?

Maybe reformatting the commit message helps. I like to make it very
clear what was backported, and who reviewed/verified the backport.
Indenting all of that with   works for me, checkpatch.pl does not
complain about it.

For an example, see http://review.gluster.org/10882.

The backporting guidelines also describe this as well:

 
http://www.gluster.org/community/documentation/index.php/Backport_Guidelines

Cheers,
Niels


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] gluster builds are failing in rpmbuilding

[Avra Sengupta]

Resent http://review.gluster.org/11011 with the Makefile changes for 
release 3.7 branch. Unable to abandon http://review.gluster.org/11008 as 
I don't think I have permissions to do so.


Regards,
Avra

On 05/30/2015 09:20 AM, Avra Sengupta wrote:
That is because the patch that introduces glusterd-errno.h is not yet 
merged in 3.7. So glusterd-errno.h is still not present int release 
3.7. I will update the patch introducing the header file itself with 
the required change, and will abandon http://review.gluster.org/#/c/11008


Regards,
Avra

On 05/30/2015 08:29 AM, Pranith Kumar Karampuri wrote:



On 05/30/2015 08:11 AM, Pranith Kumar Karampuri wrote:



On 05/30/2015 08:10 AM, Pranith Kumar Karampuri wrote:

I see that kaleb already sent a patch for this:
http://review.gluster.org/#/c/11007 - master
http://review.gluster.org/#/c/11008 - NetBSD

I meant http://review.gluster.org/#/c/11008 for release-3.7 :-)

This fails in smoke with the following failure :-(.
make[4]: *** No rule to make target `glusterd-errno.h', needed by 
`all-am'.  Stop.

make[4]: *** Waiting for unfinished jobs

On my laptop it succeeds though :-/. Any clues?

Pranith


Pranith


I am going to abandon my patch.

Pranith

On 05/30/2015 07:54 AM, Pranith Kumar Karampuri wrote:



On 05/30/2015 07:44 AM, Pranith Kumar Karampuri wrote:



On 05/30/2015 07:33 AM, Nagaprasad Sathyanarayana wrote:
It appears to me that glusterd-errno.h was added in the patch 
http://review.gluster.org/10313, which was merged on 29th. 
Please correct me if I am wrong.
I think it is supposed to be added to Makefile as well. Let me do 
some testing.

http://review.gluster.org/11010 fixes this.

Thanks a lot Naga :-)

Pranith


Pranith


Thanks
Naga

- Original Message -
From: Nagaprasad Sathyanarayana nsath...@redhat.com
To: Pranith Kumar Karampuri pkara...@redhat.com
Cc: Gluster Devel gluster-devel@gluster.org
Sent: Saturday, May 30, 2015 7:23:21 AM
Subject: Re: [Gluster-devel] gluster builds are failing in 
rpmbuilding


Could it be due to the compilation errors?

http://build.gluster.org/job/glusterfs-devrpms-el6/9019/ :
glusterd-locks.c:24:28: error: glusterd-errno.h: No such file or 
directory

   CC glusterd_la-glusterd-mgmt-handler.lo
glusterd-locks.c: In function 'glusterd_mgmt_v3_lock':
glusterd-locks.c:557: error: 'EANOTRANS' undeclared (first use 
in this function)
glusterd-locks.c:557: error: (Each undeclared identifier is 
reported only once

glusterd-locks.c:557: error: for each function it appears in.)
make[5]: *** [glusterd_la-glusterd-locks.lo] Error 1
make[5]: *** Waiting for unfinished jobs
make[4]: *** [all-recursive] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.E46NjW (%build)
 Bad exit status from /var/tmp/rpm-tmp.E46NjW (%build)
Child return code was: 1

http://build.gluster.org/job/glusterfs-devrpms/9141/ :
glusterd-locks.c:24:28: fatal error: glusterd-errno.h: No such 
file or directory

  #include glusterd-errno.h
 ^
compilation terminated.
   CC glusterd_la-glusterd-mgmt-handler.lo
   CC glusterd_la-glusterd-mgmt.lo
make[5]: *** [glusterd_la-glusterd-locks.lo] Error 1

http://build.gluster.org/job/glusterfs-devrpms-el7/2179/ :
glusterd-locks.c:24:28: fatal error: glusterd-errno.h: No such 
file or directory

  #include glusterd-errno.h
 ^
compilation terminated.
   CC glusterd_la-glusterd-mgmt.lo
make[5]: *** [glusterd_la-glusterd-locks.lo] Error 1
make[5]: *** Waiting for unfinished jobs
glusterd-mgmt.c:26:28: fatal error: glusterd-errno.h: No such 
file or directory

  #include glusterd-errno.h
 ^
compilation terminated.


Thanks
Naga

- Original Message -
From: Pranith Kumar Karampuri pkara...@redhat.com
To: Gluster Devel gluster-devel@gluster.org
Sent: Saturday, May 30, 2015 6:57:41 AM
Subject: [Gluster-devel] gluster builds are failing in rpmbuilding

hi,
  I don't understand rpmbuild logs that well. But the 
following seems

to be the issue:
Start: build phase for 
glusterfs-3.8dev-0.314.git471b2e0.el6.src.rpm
Start: build setup for 
glusterfs-3.8dev-0.314.git471b2e0.el6.src.rpm
Finish: build setup for 
glusterfs-3.8dev-0.314.git471b2e0.el6.src.rpm

Start: rpmbuild glusterfs-3.8dev-0.314.git471b2e0.el6.src.rpm
ERROR: Exception(glusterfs-3.8dev-0.314.git471b2e0.el6.src.rpm)
Config(epel-6-x86_64) 1 minutes 5 seconds

Please feel free to take a look at the following links for 
sample runs:

http://build.gluster.org/job/glusterfs-devrpms-el6/9019/console
http://build.gluster.org/job/glusterfs-devrpms/9141/console
http://build.gluster.org/job/glusterfs-devrpms-el7/2179/console

Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Regression-failure] glusterd status

[Avra Sengupta]

Hi,

What you are refering with the glusterd being restarted might be true, 
but I don't remember if the two different peer addresses were before and 
after restarting glusterd, or after restarting glusterd and before/after 
peer status. We might need to re-do this exercise on a rackspace vm to 
get more clarity.


Regards,
Avra

On 05/26/2015 06:26 PM, Kaushal M wrote:

These are my findings regarding the issue KP mentioned with
volume-snapshot-clone.t.

 From what I gathered, the double peerinfo objects issue was observed
only via logs. Additional logging was added to
glusterd_peer_rpc_notify to log the objects address and
peerinfo-hostname . I did the same and observed the same in my logs.
But what I found was that glusterd was being restarted in between the
logs with different address. So the peerinfo objects were bound to
have different addresses.

Avra, can you confirm if the above is correct? In the case it is, then
I can help debug the issue being faced.

I've been running the test on a local vm but, I've not gotten it to
fail, even without Avra's partial fix. I spoke to Atin regarding this,
and he informed me that the failures are only hit on the rackspace
slave VMs. I'll try get access to a slave VM and check the issue out.

~kaushal

On Tue, May 26, 2015 at 3:37 PM, Krishnan Parthasarathi
kpart...@redhat.com wrote:

All,

The following are the regression test failures that are being looked
at by Atin, Avra, Kaushal and self.

1) ./tests/bugs/glusterd/bug-974007.t   to be fixed by  
http://review.gluster.org/10872

2) ./tests/basic/volume-snapshot-clone. to be fixed (partially) by  
http://review.gluster.org/10895
There is another issue ('other' part) where there are 2 peerinfo objects 
for a given peer/node. This
is being investigated by Kaushal.

Will keep this thread updated as and when more progress is made.

cheers,
KP
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Regression failure in volume-snapshot-clone.t

[Avra Sengupta]

Thanks Vijay.

Regards,
Avra

On 05/22/2015 11:46 AM, Vijay Bellur wrote:

On 05/22/2015 11:41 AM, Avra Sengupta wrote:

Hi,

Given that we are holding all patches till we get the regression tests
fixed, can't we get a couple more vms. That will help increase the pace.


Have offlined slave23.cloud.gluster.org. Please go ahead and let us 
know once you are done.


Thanks,
Vijay



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Regression failure in volume-snapshot-clone.t

[Avra Sengupta]

Hi,

Given that we are holding all patches till we get the regression tests 
fixed, can't we get a couple more vms. That will help increase the pace.


Regards,
Avra

On 05/21/2015 10:46 PM, Justin Clift wrote:

There are two extra CentOS 6 VM's online for debugging stuff with,
but they're both in use at the moment:

  * slave0.cloud.gluster.org
  * slave1.cloud.gluster.org

Sachin Pandit, Raghavendra Bhat, and Krutika Dhananjay, are using
them.  Ping them, and organise with them when you can use them.  I
was intending to turn them off today, but it sounds like they should
be left on for a while longer for people to investigate with.

Regards and best wishes,

Justin Clift

On 21 May 2015, at 14:22, Avra Sengupta aseng...@redhat.com wrote:

Hi,

Can I get access to a rackspace VM so that I can debug this particular testcase 
on it.

Regards,
Avra

 Forwarded Message 
Subject:Re: [Gluster-devel] Regression failure in 
volume-snapshot-clone.t
Date:   Thu, 21 May 2015 17:08:05 +0530
From:   Vijay Bellur vbel...@redhat.com
To: Avra Sengupta aseng...@redhat.com, gluster Devel gluster-devel@gluster.org, atin Mukherjee 
amukh...@redhat.com, Krishnan Parthasarathi kpart...@redhat.com, rjosep  Rajesh Joseph 
rjos...@redhat.com

On 05/21/2015 02:44 PM, Avra Sengupta wrote:

Hi,

I am not able to reproduce this failure in my set-up. I am aware that
Atin was able to do so successfully a few days back, and I tried
something similar with the following loop.

for i in {1..100}; do export DEBUG=1; prove -r
./tests/basic/volume-snapshot-clone.t  1; lines=`less 1 | grep All
tests successful | wc -l`; if [ $lines != 1 ];then echo TESTCASE
FAILED. BREAKING; break; fi; done

I have been running this for about an hour and half, and will continue
doing so. But till now i have not encountered a failure. Could anyone
please point out if I am missing something obvious here.



Some tests fail more frequently in the rackspace VMs where we run
regressions. Please drop a note on gluster-infra ML if you want to
offline one such VM from jenkins and run tests there.

-Vijay




___
Gluster-infra mailing list
gluster-in...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-infra

--
GlusterFS - http://www.gluster.org

An open source, distributed file system scaling to several
petabytes, and handling thousands of clients.

My personal twitter: twitter.com/realjustinclift



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Regression failure in volume-snapshot-clone.t

[Avra Sengupta]

Hi,

I am done using the machine. I have sent a patch for the issue 
(http://review.gluster.org/#/c/10895/).


Regards,
Avra

On 05/22/2015 12:05 PM, Avra Sengupta wrote:

Thanks Vijay.

Regards,
Avra

On 05/22/2015 11:46 AM, Vijay Bellur wrote:

On 05/22/2015 11:41 AM, Avra Sengupta wrote:

Hi,

Given that we are holding all patches till we get the regression tests
fixed, can't we get a couple more vms. That will help increase the 
pace.


Have offlined slave23.cloud.gluster.org. Please go ahead and let us 
know once you are done.


Thanks,
Vijay





___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Are we dropping tests/bugs/snapshot/bug-1112559.t ?

[Avra Sengupta]

The regressions have passed on http://review.gluster.org/#/c/10871/

Regards,
Avra

On 05/21/2015 12:29 PM, Atin Mukherjee wrote:


On 05/21/2015 12:27 PM, Vijay Bellur wrote:

On 05/21/2015 12:23 PM, Krishnan Parthasarathi wrote:

Smoke test has failed due to jenkins related issue. We need to
retrigger smoke
test. I am not aware of how the infrastructure works?

Anyone, any ideas?


If you are logged in to Jenkins, you should be able to find
Retrigger/Retrigger All links on the page for the job. Clicking on that
should take care of launching smoke test(s) again.

Retriggered

-Vijay


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Are we dropping tests/bugs/snapshot/bug-1112559.t ?

[Avra Sengupta]

Thanks for merging the patch. I have backported 
it(http://review.gluster.org/#/c/10871/) to release 3.7 branch as well.


Regards,
Avra

On 05/20/2015 05:52 PM, Avra Sengupta wrote:
I've sent a patch(http://review.gluster.org/#/c/10840/) to remove this 
from the test-suite. Once it get's merged I will re-open the clone of 
this bug for 3.7.0 branch, and backport the patch.


Regards,
Avra

On 05/20/2015 12:07 PM, Krishnan Parthasarathi wrote:

No concerns.

- Original Message -
Given that the fix which is tested by this patch is no longer 
present, I

think we should remove this patch from the test-suite itself. Could
anyone confirm if there are any concerns in doing so. If not I will 
send

a patch to do the same.

Regards,
Avra

On 05/08/2015 11:28 AM, Avra Sengupta wrote:

Raised a bug, and sent a patch(http://review.gluster.org/10660)
marking this test as a bad test.

Regards,
Avra

On 05/08/2015 11:24 AM, Krishnan Parthasarathi wrote:

It's a snapshot-clone feature team's decision. I have no objection if
you want it removed.

- Original Message -

On 05/08/2015 07:40 AM, Pranith Kumar Karampuri wrote:

hi,
Are we dropping tests/bugs/snapshot/bug-1112559.t? If 
yes is

the
patch for removing it already sent?

Waiting to get a confirmation from glusterd folks .

Rafi KC


Pranith






___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Regression failure in volume-snapshot-clone.t

[Avra Sengupta]

Hi,

I am not able to reproduce this failure in my set-up. I am aware that 
Atin was able to do so successfully a few days back, and I tried 
something similar with the following loop.


for i in {1..100}; do export DEBUG=1; prove -r 
./tests/basic/volume-snapshot-clone.t  1; lines=`less 1 | grep All 
tests successful | wc -l`; if [ $lines != 1 ];then echo TESTCASE 
FAILED. BREAKING; break; fi; done


I have been running this for about an hour and half, and will continue 
doing so. But till now i have not encountered a failure. Could anyone 
please point out if I am missing something obvious here.


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Fwd: Re: Regression failure in volume-snapshot-clone.t

[Avra Sengupta]

Hi,

Can I get access to a rackspace VM so that I can debug this particular 
testcase on it.


Regards,
Avra

 Forwarded Message 
Subject:Re: [Gluster-devel] Regression failure in 
volume-snapshot-clone.t
Date:   Thu, 21 May 2015 17:08:05 +0530
From:   Vijay Bellur vbel...@redhat.com
To: 	Avra Sengupta aseng...@redhat.com, gluster Devel 
gluster-devel@gluster.org, atin Mukherjee amukh...@redhat.com, 
Krishnan Parthasarathi kpart...@redhat.com, rjosep  Rajesh Joseph 
rjos...@redhat.com




On 05/21/2015 02:44 PM, Avra Sengupta wrote:

Hi,

I am not able to reproduce this failure in my set-up. I am aware that
Atin was able to do so successfully a few days back, and I tried
something similar with the following loop.

for i in {1..100}; do export DEBUG=1; prove -r
./tests/basic/volume-snapshot-clone.t  1; lines=`less 1 | grep All
tests successful | wc -l`; if [ $lines != 1 ];then echo TESTCASE
FAILED. BREAKING; break; fi; done

I have been running this for about an hour and half, and will continue
doing so. But till now i have not encountered a failure. Could anyone
please point out if I am missing something obvious here.




Some tests fail more frequently in the rackspace VMs where we run
regressions. Please drop a note on gluster-infra ML if you want to
offline one such VM from jenkins and run tests there.

-Vijay



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Are we dropping tests/bugs/snapshot/bug-1112559.t ?

[Avra Sengupta]

Given that the fix which is tested by this patch is no longer present, I 
think we should remove this patch from the test-suite itself. Could 
anyone confirm if there are any concerns in doing so. If not I will send 
a patch to do the same.


Regards,
Avra

On 05/08/2015 11:28 AM, Avra Sengupta wrote:
Raised a bug, and sent a patch(http://review.gluster.org/10660) 
marking this test as a bad test.


Regards,
Avra

On 05/08/2015 11:24 AM, Krishnan Parthasarathi wrote:
It's a snapshot-clone feature team's decision. I have no objection if 
you want it removed.


- Original Message -


On 05/08/2015 07:40 AM, Pranith Kumar Karampuri wrote:

hi,
   Are we dropping tests/bugs/snapshot/bug-1112559.t? If yes is 
the

patch for removing it already sent?

Waiting to get a confirmation from glusterd folks .

Rafi KC


Pranith






___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Are we dropping tests/bugs/snapshot/bug-1112559.t ?

[Avra Sengupta]

I've sent a patch(http://review.gluster.org/#/c/10840/) to remove this 
from the test-suite. Once it get's merged I will re-open the clone of 
this bug for 3.7.0 branch, and backport the patch.


Regards,
Avra

On 05/20/2015 12:07 PM, Krishnan Parthasarathi wrote:

No concerns.

- Original Message -

Given that the fix which is tested by this patch is no longer present, I
think we should remove this patch from the test-suite itself. Could
anyone confirm if there are any concerns in doing so. If not I will send
a patch to do the same.

Regards,
Avra

On 05/08/2015 11:28 AM, Avra Sengupta wrote:

Raised a bug, and sent a patch(http://review.gluster.org/10660)
marking this test as a bad test.

Regards,
Avra

On 05/08/2015 11:24 AM, Krishnan Parthasarathi wrote:

It's a snapshot-clone feature team's decision. I have no objection if
you want it removed.

- Original Message -

On 05/08/2015 07:40 AM, Pranith Kumar Karampuri wrote:

hi,
Are we dropping tests/bugs/snapshot/bug-1112559.t? If yes is
the
patch for removing it already sent?

Waiting to get a confirmation from glusterd folks .

Rafi KC


Pranith




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Simplify creation and set-up of meta-volume (shared storage)

[Avra Sengupta]

With this option, the volume will be created and explicitly mounted on 
all the nodes, which are currently a part of the cluster. Please note 
that new nodes added to the cluster will not have the meta volume 
mounted explicitly.


So in a case where the console tries to use the volume from a peer in 
the cluster, which was added after the option was set it will not have 
the mount available to it. Hence I feel its best that the console 
continues with the explicit mounting, and showing explicit warning 
during stop/remove-brick of the meta volume in console and allow the 
operations.


There is no other impact on the console, as far as this feature is 
concerned.


Regards,
Avra

On 05/18/2015 09:27 AM, Shubhendu Tripathi wrote:

Avra,

We are planning to provide mounting of meta volume on the nodes of the 
cluster from Console as part of volume syncing and addition of new 
nodes to the cluster. Looks like if this option is set, the explicit 
mounting of the meta volume from console is not required and it would 
be taken care by gluster.


Currently we show explicit warning during stop/remove-brick of the 
meta volume in console and allow the operations. I dont feel there 
would be any impact due to new feature.


Kindly let us know if any other impact on console or we need to take 
care of anything else as result of this feature.


Thanks and Regards,
Shubhendu

On 05/15/2015 07:30 PM, Avra Sengupta wrote:

Hi,

A shared storage meta-volume is currently being used by 
snapshot-scheduler, geo-replication, and nfs-ganesha. In order to 
simplify the creation and set-up of the same, we are introducing a 
global volume set option(cluster.enable-shared-storage).


On enabling this option, the system analyzes the number of peers in 
the cluster, which are currently connected, and chooses three such 
peers(including the node the command is issued from). From these 
peers a volume(gluster_shared_storage) is created. Depending on the 
number of peers available the volume is either a replica 3 volume(if 
there are 3 connected peers), a replica 2 volume(if there are 2 
connected peers), or a single brick volume(if there is only one node 
in the cluster). /var/run/gluster/ss_brick serves as the brick path 
on each node for the shared storage volume. We also mount the shared 
storage at /var/run/gluster/shared_storage on all the nodes in the 
cluster as part of enabling this option.


Once the volume is created, and mounted the maintainance of the 
volume like adding-bricks, removing bricks etc., is expected to be 
the onus of the user.


On disabling the option, we provide the user a warning, and on 
affirmation from the user we stop the shared storage volume, and 
unmount it from all the nodes in the cluster.


These are achieved with hook-scripts as part of the volume set 
option. If anyone is interested in having a look at the patch, it's 
available for review at http://review.gluster.org/#/c/10793/ . If 
there is any feedback and suggestion regarding the same, please feel 
free to share it.


Regards,
Avra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel




___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Simplify creation and set-up of meta-volume (shared storage)

[Avra Sengupta]

Hi,

A shared storage meta-volume is currently being used by 
snapshot-scheduler, geo-replication, and nfs-ganesha. In order to 
simplify the creation and set-up of the same, we are introducing a 
global volume set option(cluster.enable-shared-storage).


On enabling this option, the system analyzes the number of peers in the 
cluster, which are currently connected, and chooses three such 
peers(including the node the command is issued from). From these peers a 
volume(gluster_shared_storage) is created. Depending on the number of 
peers available the volume is either a replica 3 volume(if there are 3 
connected peers), a replica 2 volume(if there are 2 connected peers), or 
a single brick volume(if there is only one node in the cluster). 
/var/run/gluster/ss_brick serves as the brick path on each node for 
the shared storage volume. We also mount the shared storage at 
/var/run/gluster/shared_storage on all the nodes in the cluster as 
part of enabling this option.


Once the volume is created, and mounted the maintainance of the volume 
like adding-bricks, removing bricks etc., is expected to be the onus of 
the user.


On disabling the option, we provide the user a warning, and on 
affirmation from the user we stop the shared storage volume, and unmount 
it from all the nodes in the cluster.


These are achieved with hook-scripts as part of the volume set option. 
If anyone is interested in having a look at the patch, it's available 
for review at http://review.gluster.org/#/c/10793/ . If there is any 
feedback and suggestion regarding the same, please feel free to share it.


Regards,
Avra

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] NetBSD regression not getting migrated to the patch

[Avra Sengupta]

Hi,

I have run NetBSD regression on http://review.gluster.org/#/c/10358/ 
twice. http://build.gluster.org/job/netbsd6-smoke/6066/console and 
http://build.gluster.org/job/netbsd6-smoke/6135/console. Both the times 
the regressions have passed but the results are not getting reflected in 
the patch. This is a blocker from our end for 3.7. Can someone please 
tell me the next course of action for this?


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression not getting migrated to the patch

[Avra Sengupta]

Am facing the same issue on patches 
http://review.gluster.org/#/c/10313/  and 
http://review.gluster.org/#/c/10588/. The netbsd regression passes for 
both of them http://build.gluster.org/job/netbsd6-smoke/6065/console and 
http://build.gluster.org/job/netbsd6-smoke/6076/console, but the results 
dont reflect on the patchset.


Regards,
Avra


On 05/10/2015 10:47 AM, Avra Sengupta wrote:

Hi,

I have run NetBSD regression on http://review.gluster.org/#/c/10358/ 
twice. http://build.gluster.org/job/netbsd6-smoke/6066/console and 
http://build.gluster.org/job/netbsd6-smoke/6135/console. Both the 
times the regressions have passed but the results are not getting 
reflected in the patch. This is a blocker from our end for 3.7. Can 
someone please tell me the next course of action for this?


Regards,
Avra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Need help with snapshot regression failures

[Avra Sengupta]

Hi Pranith,

Could you please provide  a regression instance where the snapshot tests 
failed. I had a look at 
http://build.gluster.org/job/rackspace-regression-2GB-triggered/8148/consoleFull 
but, the logs for bug-1162498.t are not present for that instance. 
Similarly other instances recorded in the etherpad either doesn't have 
the regression instances, or the logs for those instances are not present.


Regards,
Avra

On 05/04/2015 11:27 AM, Pranith Kumar Karampuri wrote:

hi Rajesh/Avra,
 I do not have good understanding of snapshot, so couldn't 
investigate any of the snapshot related spurious failures present in 
https://public.pad.fsfe.org/p/gluster-spurious-failures. Could you 
guys help out?


Pranith


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel