Re: [Gluster-devel] How does Gluster works - Locating files after change sin cluster

[Raghavendra Talur]

On Wed, Sep 4, 2019 at 5:01 AM Barak Sason Rofman 
wrote:

> Hello everyone,
>
> I'm about to post several threads with question regarding how Gluster
> handles different scenarios.
> I'm looking for answers on architecture/design/"the is the idea" level,
> and not specifically implementation (however, it would be nice to know
> where the relevant code is).
>
> In this thread I want to focus on the "adding servers/bricks" scenario.
> From what I know at this point, every file that's created is given a
> 32-bit value based on it's name, and this hashing function is fixed and
> independent of any factors.
> Next, there is a function (a routing method), located on the client side,
> that *is* dependent on outside factors, such as numbers of servers (or
> bricks) in the system which determines on which server a particular file is
> located.
>
> Let's examine the following case:
> Assume (for simplicity's sake) that the hashing function assign values to
> file in 1-100 range (instead of 32-bit) and currently there are 4 servers
> in the cluster.
> In this case, files 1-25 would be located on server 1, 26-50 on server 2
> and so on.
> Now, if a 5th server is added to the cluster, then the ranges will change:
> files 1-20 will be located on server 1, 21-40 on server 2 and so on.
>
> The questions regarding this scenarios are as follows:
> 1 - Does the servers update the clients that an additional server (or
> brick) has been added to the cluster? If not, how does this happen?
>

Yes, addition of a brick happens through a gluster cli command that updates
the volume info in glusterd. Glusterd(the one which updated config and
other peers) update clients about this change.

2 - Does the server also know which files *should* be located on them? if
> so, does the servers create a link file (which specifies the "real"
> location of the file) for the files that are supposed to be moved (e.g.
> files 21-25) or actually move the data right away? Maybe this works in a
> completely different manner?
>

The addition of a brick has a step for updating the xattrs on the bricks
which marks the range for them. The creation of link files happens lazily.
Clients look up on all bricks when they don't find the file on the brick
where it is supposed to be(called hashed brick), the brick where they find
the file is called cached brick and a link file is created.

For more information on distribute mechanism refer to
https://docs.gluster.org/en/latest/Quick-Start-Guide/Architecture/#dhtdistributed-hash-table-translator
For more information on how clients get update from glusterd refer to
https://www.youtube.com/watch?v=Gq-yBYq8Gjg


> I have additional questions regarding this, but they are dependent om the
> answers to these question.
>
> Thank you all for your help.
> --
> *Barak Sason Rofman*
>
> Gluster Storage Development
>
> Red Hat Israel 
>
> 34 Jerusalem rd. Ra'anana, 43501
>
> bsaso...@redhat.com T: *+972-9-7692304*
> M: *+972-52-4326355*
> 
> ___
>
> Community Meeting Calendar:
>
> APAC Schedule -
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/836554017
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/486278655
>
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Proposal: move glusterfs development to github workflow, completely

[Raghavendra Talur]

On Fri, Aug 23, 2019, 9:12 AM Amar Tumballi  wrote:

> Hi developers,
>
> With this email, I want to understand what is the general feeling around
> this topic.
>
> We from gluster org (in github.com/gluster) have many projects which
> follow complete github workflow, where as there are few, specially the main
> one 'glusterfs', which uses 'Gerrit'.
>
> While this has worked all these years, currently, there is a huge set of
> brain-share on github workflow as many other top projects, and similar
> projects use only github as the place to develop, track and run tests etc.
> As it is possible to have all of the tools required for this project in
> github itself (code, PR, issues, CI/CD, docs), lets look at how we are
> structured today:
>
> Gerrit - glusterfs code + Review system
> Bugzilla - For bugs
> Github - For feature requests
> Trello - (not very much used) for tracking project development.
> CI/CD - CentOS-ci / Jenkins, etc but maintained from different repo.
> Docs - glusterdocs - different repo.
> Metrics - Nothing (other than github itself tracking contributors).
>
> While it may cause a minor glitch for many long time developers who are
> used to the flow, moving to github would bring all these in single place,
> makes getting new users easy, and uniform development practices for all
> gluster org repositories.
>
> As it is just the proposal, I would like to hear people's thought on this,
> and conclude on this another month, so by glusterfs-8 development time, we
> are clear about this.
>

A huge +1 to this proposal. As you said, github has wider mind share and
new developers won't have to learn tooling to contribute to gluster.

Thanks
Raghavendra Talur

>
> Can we decide on this before September 30th? Please voice your concerns.
>
> Regards,
> Amar
>
>
>
> ___
> maintainers mailing list
> maintain...@gluster.org
> https://lists.gluster.org/mailman/listinfo/maintainers
>
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Request For Opinions: what to do about the synthetic statfvs "tweak"?

[Raghavendra Talur]

On Wed 21 Mar, 2018, 3:57 PM Csaba Henk,  wrote:

> Hi list,
>
> We have an ancient hack that fuse not
> just passes on the statvfs data it's getting
> from the storage, but tweaks it by setting
> f_bsize / f_frsize to values of its own
> preference. [1]
>
> The supposed advantage is that f_bsize
> serves as a hint to applications for the
> preferred io size. (And regarding f_frsize --
> in Linux it's a historical workaround for
> certain bugs in userspace[2] that f_bsize
> and f_frsize are being kept equal.)
>
> However, this has the side effect of
> getting slightly different values for total
> and used/free space of a volume when
> accessed through libgfapi and when through
> fuse -- because these values are multiples
> of f_frsize, and libgfapi uses the native f_frsize
> of the backend (typically 4k), while fuse provides
> its synthetic f_frsize (typically 128k). So the
> libfgapi provided sizes are more granular.
>
> OTOH, I couldn't find any program in
> standard Unix userspace where the
> implementation commonly used in GNU/Linux
> would rely on the f_bsize hint. It does not
> mean there is none -- and then there is of course
> the vast space of non-standard userland.
>
> Possibiliities are:
>
> 1) retire the whole tweak and just pass on
>what we get from storage [3]
>

I prefer the above. It makes libgfapi and fuse consistent.


2) keep the f_bsize tweak, but drop the
>of f_frsize == f_bsize legacy workaround
>and take f_frsize from storage
> 3) keep everything as is, and put up with the
>fs stat divergence between transports
> +1) make behaviors of 1) to 3) tunable --
>   but I would not like to go this way in
>   the spirit of KISS, unless absolutely a
>   demand
>
> Developers: do you know of any scenario where
> we benefit from the f_bsize tweak?
>
> Users:
> - do you have any application that relies on f_bsize
>   and benefits from its custom value?
> - do you have any legacy application/stack
>   where the f_frsize == f_bsize workaround is
>   still needed (but GlusterFS  / RHGS is being kept
>   up to date, so a change in this regard would hit
>   your setup)?
>
> Thanks for your thoughts!
>
> Regards,
> Csaba
>
> [1]:
> https://github.com/gluster/glusterfs/blob/v4.0.0/xlators/mount/fuse/src/fuse-bridge.c#L3177-L3189
> [2]: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=11406
> [3] practically this will also imply f_frsize == f_bsize, because
>  Linux filesystems usually adhere to this convention
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Locating blocker bugs for a release (and what is a blocker anyway)

[Raghavendra Talur]

On Thu, Oct 12, 2017 at 6:09 PM, Shyam Ranganathan <srang...@redhat.com> wrote:
> On 10/12/2017 12:58 AM, Nithya Balachandran wrote:
>>
>>  > - When to mark the blocker bug as dependent on another bug:
>>  >
>>  > Blockers are meant to track *just* blockers for a major or minor
>> release,
>>  > not all bugs that are a part of the release. Hence when a bug is
>> actually a
>>  > blocker for the release, only then should the tracker be made
>> dependent on
>>  > that.
>>
>> I have used it a little differently. On the day of release, I add all
>> the bugs that are fixed in a release in "depends on" list of the
>> blocker bug. Essentially, I used it as a tracker bug than a blocker
>> bug.
>>
>> Wouldn't it be easier that every bug that one wants to get fixed in
>> next minor release is added to the tracker bug and is removed and
>> moved to next release if it did not make it? It would be less work for
>> maintainer.
>>
>> +1 . We could use the blocker flag to tag actual blockers.
>>
>
> So here are 2 things a contributor needs to do if all bugs are marked
> against the blocker.
>
> a) Backport it, no way to avoid this and the additional work of
> filing/cloning a bug against the release for which the backport is aimed at.
>
> b) Add it to the tracker
>
> (b) is not essential, it is additional work for the contributor. Further the
> bug may or may not make it to the release (as reflected above), and adds
> work for the release maintainer to cleanup the tracker and add those that
> are missed to the next release tracker.
>
> For a release maintainer (or anyone with a cloned repository), the ability
> to find bugs that are fixed in a release is relatively simple. It is as easy
> as executing this (for example),
> git log --format=email v3.12.1..v3.12.2 | grep -i ^bug: | awk ‘{print $2}’ |
> sort -u
>
> For users this list is presented in the release notes, which is part of the
> documentation, release announcement, and in github as well. Further, post a
> release bugs that were fixed as a part of the release are closed with a
> comment in bugzilla stating which release it made it to, and also the
> release announcement mail.
>
> The above user perspective is added, as there maybe a justification that a
> user could look at the tracker and figure out is a bug is fixed in the said
> release, but this again is not required, if the bug of interest is known
> then looking at that bug in bugzilla provides the information, if the query
> is more along what all are fixed, the release notes inform regarding the
> same.
>
> As a result, marking all bugs against the tracker, is more work for a
> contributor as well as the release maintainer. So why would we want to do
> this?

Yes, this is mainly a matter of preference. I don't have any objection
to using it as a blocker bug and adding only blockers to it.

Thanks,
Raghavendra Talur

>
> Shyam
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Locating blocker bugs for a release (and what is a blocker anyway)

[Raghavendra Talur]

On Wed, Oct 11, 2017 at 8:35 PM, Shyam Ranganathan  wrote:
> Hi,
>
> Recently I was in some conversations that mentioned having to hunt for the
> mail that announced the blocker bug for a release, when there is a need to
> mark a bug as a blocker for that release.
>
> This need not be done as here is the shortcut (if you did not know) on how
> to find the blocker,
>
> Use this URL:
> https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-
>
> Replacing  with the version for which you are attempting to find
> the blocker bug for.
>
> Example:
>   - https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.10.6
>   - https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.10.7
>   - https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.12.2
>
> Some FAQs:
>
> - When is the blocker bug created?
>
> Blocker bugs are created post branching a release, so if you went around
> hunting for the 3.13.0 blocker today, it does not exist.
>
> Blocker bugs for minor releases are created post closing the prior minor
> release, and migrating any blockers that did not make it in the prior
> release to the next minor release blocker (this is only(?) possible as a bug
> maybe marked as blocking a release post tagging the release and before
> actually announcing the release).
>
> - When to mark the blocker bug as dependent on another bug:
>
> Blockers are meant to track *just* blockers for a major or minor release,
> not all bugs that are a part of the release. Hence when a bug is actually a
> blocker for the release, only then should the tracker be made dependent on
> that.

I have used it a little differently. On the day of release, I add all
the bugs that are fixed in a release in "depends on" list of the
blocker bug. Essentially, I used it as a tracker bug than a blocker
bug.

Wouldn't it be easier that every bug that one wants to get fixed in
next minor release is added to the tracker bug and is removed and
moved to next release if it did not make it? It would be less work for
maintainer.

Talur


>
> - What is a blocker anyway?
>
> Blockers can be broadly classified as, regression of functionality due to
> code introduced in prior minor releases for the current version, bugs
> identified causing corruptions, bugs identified causing crashes
>
> If you believe a bug is a blocker, but still does not meet the criteria
> above, err on the safe option and call it a blocker, whose merit can then be
> discussed on the lists and appropriate action for the release taken.
>
> - Other questions or comments?
>
> HTH,
> Shyam
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [heketi-devel] Nos vemos!

[Raghavendra Talur]

On Mon, Sep 25, 2017 at 10:17 PM, Luis Pabon  wrote:
> Hi community,
>   It has been a great time working together on GlusterFS and Heketi these
> past few years. I remember starting on Heketi to enable GlusterFS for
> Manila, but now I will be moving on to work at Portworx and their
> containerized storage product.
>   I am passing leadership of the Heketi project to Michael Adam, and I now
> it is in good hands at Red Hat.
>
> Thank you all again for making Heketi awesome.
>
> - Luis
>
> PS: If you didn't know, Heketi is a Taino word (indigenous people of the
> Caribbean) which means One. Other Taino words are: Barbecue,  hurricane,
> hammock and many others.

Thanks for everything and Best of Luck!

Talur
>
>
>
> ___
> heketi-devel mailing list
> heketi-de...@gluster.org
> http://lists.gluster.org/mailman/listinfo/heketi-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question regarding to gluster and vfs

[Raghavendra Talur]

On Wed, Aug 16, 2017 at 5:52 PM, Ilan Schwarts <ila...@gmail.com> wrote:
> Hi,
> So this is a bit odd case.
> I have created 2 servers nodes (running CentOS 7.3)
> From Client machine (CentOS 7.2) I mount to one of the nodes (nfs) using:
> [root@CentOS7286-64 mnt]#  mount -t nfs
> L137B-GlusterFS-Node1.L137B-root.com:/volume1 /mnt/glustervianfs/
>
> When i created (touch) a file over the NFS:
> From Client Machine:
> [revivo@CentOS7286-64 glustervianfs]$ touch nfs3file
> [revivo@CentOS7286-64 glustervianfs]$ id revivo
> uid=2021(revivo) gid=2020(maccabi) groups=2020(maccabi),10(wheel)
>
> On Server machine:
> I monitor the file operations at VFS kernel level.
> I receive 1 event of file create, and 2 events of set attribute changes.
> What I see is that root creates the file (uid/gid of 0)
> And then root (also) use chown and chgrp to set security (attribute)
> of the new file.
>
> When i go to the glutser volume itself and ls -la,i do see the
> *correct* (2021 - revivo /2020 - revivo) uid/gid:
> [root@L137B-GlusterFS-Node1 volume1]# ls -lia
> total 24
> 11 drwxrwxrwx.  3 revivo maccabi 4096 Aug 10 12:13 .
>  2 drwxr-xr-x.  3 root   root4096 Aug  9 14:32 ..
> 12 drw---. 16 root   root4096 Aug 10 12:13 .glusterfs
> 31 -rw-r--r--.  2 revivo maccabi0 Aug 10 12:13 nfs3file
>
> Why on the VFS layer i get uid/gid - 0/0

As you have pointed out above, the file is created with 0:0
owner:group but subsequent operations change owner and group using
chown and chgrp. This is because the glusterfsd(brick daemon) process
always runs as root. I don't know the exact reason why setfsuid and
setfsgid are not used although the code exist.

Amar/Pranith/Raghavendra/Vijay,

Do you know why HAVE_SET_FSID is undefined in line
https://github.com/gluster/glusterfs/blob/master/xlators/storage/posix/src/posix.c#L65

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Change in test file, run only that test in regression job?

[Raghavendra Talur]

On Tue, Jul 11, 2017 at 1:51 PM, Atin Mukherjee <amukh...@redhat.com> wrote:
> If a patch is all about .t changes, running the whole regression suite
> instead of running those particular tests is not worth. This idea came up
> while Nithya, Nigel & I were having conversations  around how to fix
> regression failures faster if the change is only required at the test file.
>
> Does everyone in favour of this idea? If yes, can this be implemented Nigel?

+1
Jeff Darcy has implemented[1] ability to run only updated/added tests.
We just have to invoke run-tests.sh with -H option.

This does not determine if only tests have been added. It just makes
sure we exit after running updated/added tests and don't run other
tests.
Nigel, you will have to check for "test only change* in jenkins and
conditionally call run-tests.sh with -H


[1] https://review.gluster.org/#/c/13439/

Raghavendra Talur


>
> ~Atin
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Introducing minister

[Raghavendra Talur]

Adding gluster-devel

On Fri, Jun 23, 2017 at 5:52 PM, Raghavendra Talur <rta...@redhat.com> wrote:
> Hi All,
>
> Kubernetes and Openshift have amazing projects called minikube and
> minishift which make it very easy to setup those distributed systems
> for easy development. As the Gluster ecosystem grows, we have more
> external projects which require easy setup of multi node Gluster
> cluster.
>
> Hence, along those lines, I introduce to you...minister (mini +
> Glu"ster"). Please do check out the github page, it has a asciinema
> demo.
>
> Prerequisites:
> 1. you need docker to be installed and running on the machine.(add
> yourself to docker group to not require sudo for docker commands)
> 2. the script might prompt for sudo password for loop back devices
> creation and it requires sudo only for that part. Rest of the
> operations are not privileged.
>
> Warning:
> I have worked on it for about a week now and have tried my best to
> eliminate as many critical bugs as possible; but I would strongly
> suggest that you try it out on a test VM than on your workstation.
>
> The idea is to eventually integrate it with Glusto for easy test development.
>
> Comments, suggestions and patches welcome!
>
> [1] https://github.com/raghavendra-talur/minister
>
> Thanks,
> Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] brick multiplexing and memory consumption

[Raghavendra Talur]

On 21-Jun-2017 9:45 AM, "Jeff Darcy" <j...@pl.atyp.us> wrote:




On Tue, Jun 20, 2017, at 03:38 PM, Raghavendra Talur wrote:

Each process takes 795MB of virtual memory and resident memory is 10MB each.


Wow, that's even better than I thought.  I was seeing about a 3x difference
per brick (plus the fixed cost of a brick process) during development.
Your numbers suggest more than 10x.  Almost makes it seem worth the effort.
 ;)


:)


Just to be clear, I am not saying that brick multiplexing isn't working.
The aim is to prevent the glusterfsd process from getting OOM killed
because 200 bricks when multiplexed consume 20GB of virtual memory.


Yes, the OOM killer is more dangerous with multiplexing.  It likes to take
out the process that is the whole machine's reason for existence, which is
pretty darn dumb.  Perhaps we should use oom_adj/OOM_DISABLE to make it a
bit less dumb?


This is not so easy for container deployment models.


If it is found that the additional usage of 75MB of virtual memory per
every brick attach can't be removed/reduced, then the only solution would
be to fix issue 151 [1] by limiting multiplexed bricks.
[1] https://github.com/gluster/glusterfs/issues/151


This is another reason why limiting the number of brick processes is
preferable to limiting the number of bricks per process.  When we limit
bricks per process and wait until one is "full" before starting another,
then that first brick process remains a prime target for the OOM killer.
By "striping" bricks across N processes (where N ~= number of cores), none
of them become targets until we're approaching our system-wide brick limit
anyway.


+1, I now understand the reasoning behind limiting number of processes. I
was in the favor of limiting bricks per process before.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] brick multiplexing and memory consumption

[Raghavendra Talur]

On Tue, Jun 20, 2017 at 8:13 PM, Jeff Darcy <j...@pl.atyp.us> wrote:

>
>
>
> On Tue, Jun 20, 2017, at 08:45 AM, Raghavendra Talur wrote:
>
> Here is the data I gathered while debugging the considerable increase in
> memory consumption by brick process when brick multiplexing is on.
>
> before adding 14th brick to it: 3163 MB
> before glusterfs_graph_init is called   3171 (8  MB increase)
> io-stats init   3180 (9  MB increase)
> index  init 3181 (1  MB increase)
> bitrot-stub init3182 (1  MB increase)
> changelog  init 3206 (24 MB increase)
> posix  init 3230 (24 MB increase)
> glusterfs_autoscale_threads 3238 (8  MB increase)
> end of glusterfs_handle_attach
>
> Every brick attach is taking about 75 MB of virtual memory and it is
> consistent. Need help from respective xlators owners to confirm if init of
> those xlators really takes that much memory.
>
> This is all Virtual memory data, resident memory is very nicely at 40 MB
> after 14 bricks.
>
>
> Do you have the equivalent numbers for memory consumption of 14 bricks
> *without* multiplexing?
>


Each process takes 795MB of virtual memory and resident memory is 10MB each.
Just to be clear, I am not saying that brick multiplexing isn't working.
The aim is to prevent the glusterfsd process from getting OOM killed
because 200 bricks when multiplexed consume 20GB of virtual memory.
If it is found that the additional usage of 75MB of virtual memory per
every brick attach can't be removed/reduced, then the only solution would
be to fix issue 151 [1] by limiting multiplexed bricks.
[1] https://github.com/gluster/glusterfs/issues/151



>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] brick multiplexing and memory consumption

[Raghavendra Talur]

Hi,

Here is the data I gathered while debugging the considerable increase in
memory consumption by brick process when brick multiplexing is on.

before adding 14th brick to it: 3163 MB
before glusterfs_graph_init is called   3171 (8  MB increase)
io-stats init   3180 (9  MB increase)
index  init 3181 (1  MB increase)
bitrot-stub init3182 (1  MB increase)
changelog  init 3206 (24 MB increase)
posix  init 3230 (24 MB increase)
glusterfs_autoscale_threads 3238 (8  MB increase)
end of glusterfs_handle_attach

Every brick attach is taking about 75 MB of virtual memory and it is
consistent. Need help from respective xlators owners to confirm if init of
those xlators really takes that much memory.

This is all Virtual memory data, resident memory is very nicely at 40 MB
after 14 bricks.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Glusterfs 3.10.3 has been tagged

[Raghavendra Talur]

Glusterfs 3.10.3 has been tagged.

Packages for the various distributions will be available in a few days,
and with that a more formal release announcement will be made.

- Tagged code: https://github.com/gluster/glusterfs/tree/v3.10.3
- Release notes:
https://github.com/gluster/glusterfs/blob/release-3.10/doc/release-notes/3.10.3.md

Thanks,
Raghavendra Talur

NOTE: Tracker bug for 3.10.3 will be closed in a couple of days and
tracker for 3.10.4 will be opened, and an announcement for 3.10.4 will
be sent with the details
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gerrit submit type (was tests/bugs/gfapi/bug-1447266/bug-1447266.t)

[Raghavendra Talur]

On Mon, May 15, 2017 at 5:33 PM, Jeff Darcy <j...@pl.atyp.us> wrote:
>
>
> On Sun, May 14, 2017, at 11:39 PM, Nigel Babu wrote:
>> We use the "cherry-pick" submit type for glusterfs on Gerrit[1]. In the
>> past,
>> Poornima has pointed this out as well. I believe there was no interest in
>> changing the submit type[2] because the other submit types do not add
>> metadata to
>> the commit itself.
>>
>> [1]:
>> https://review.gluster.org/Documentation/project-configuration.html#submit_type
>> [2]:
>> http://lists.gluster.org/pipermail/gluster-devel/2016-September/050874.html
>
> I'm not convinced that submit type should matter.  Gerrit clearly can
> recognize that there is a dependency before the patch is submitted.  It
> does this to generate the "related changes" list, among other things.
> If it can do that, and it can enable/disable submission based on other
> factors like regression-test votes, it has all of the mechanisms it
> needs to enable/disable submission based on dependency state.  Submit
> type only comes into play after the decision has already been made to
> enable/allow submission.

Not true. I have looked into this last year when I sent out the
mail[1] asking fast-forward to be the submit type. I am now aware that
fast-forward is wrong method to use. With cherry pick method gerrit
does not use the information which it uses to show related changes to
*enforce* submit order. However, with rebase if necessary it might.
Nigel, would it be possible for you to setup a test server to try?

Thanks,
Raghavendra Talur



[1] http://lists.gluster.org/pipermail/gluster-devel/2016-January/047740.html
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Release 3.10.2: Scheduled for the 30th of April

[Raghavendra Talur]

Glusterfs 3.10.2 has been tagged.

Packages for the various distributions will be available in a few days,
and with that a more formal release announcement will be made.

- Tagged code: https://github.com/gluster/glusterfs/tree/v3.10.2
- Release notes:
https://github.com/gluster/glusterfs/blob/release-3.10/doc/release-notes/3.10.2.md

Thanks,
Raghavendra Talur

NOTE: Tracker bug for 3.10.2 will be closed in a couple of days and
tracker for 3.10.3 will be opened, and an announcement for 3.10.3 will
be sent with the details



On Wed, May 3, 2017 at 3:46 PM, Raghavendra Talur <rta...@redhat.com> wrote:
> I had previously announced that we would be releasing 3.10.2 today.
> This is to update the 3.10.2 release is now delayed. We are waiting
> for a bug[1] to be fixed.
> If you are waiting for 3.10.2 release for a particular bug fix, please
> let us know.
>
> I will update with expected release date by tomorrow.
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1447608
>
> Thanks,
> Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Is glusterd upgrade procedure idempotent?

[Raghavendra Talur]

When gluster is upgraded, we update the volfiles in the post install
part of package installations.

In container world, there is no such procedure. We kill the old
container and bring up a new container with new Gluster packages. IMO,

glusterd --xlator-option *.upgrade=on -N

command is a idempotent operation which can be run before starting
glusterd service in a container.
Am I right?

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Release 3.10.2: Scheduled for the 30th of April

[Raghavendra Talur]

I had previously announced that we would be releasing 3.10.2 today.
This is to update the 3.10.2 release is now delayed. We are waiting
for a bug[1] to be fixed.
If you are waiting for 3.10.2 release for a particular bug fix, please
let us know.

I will update with expected release date by tomorrow.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1447608

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Don't allow data loss via add-brick (was Re: Add single server)

[Raghavendra Talur]

On Tue, May 2, 2017 at 8:46 PM, Nithya Balachandran <nbala...@redhat.com> wrote:
>
>
> On 2 May 2017 at 16:59, Shyam <srang...@redhat.com> wrote:
>>
>> Talur,
>>
>> Please wait for this fix before releasing 3.10.2.
>>
>> We will take in the change to either prevent add-brick in
>> sharded+distrbuted volumes, or throw a warning and force the use of --force
>> to execute this.

Agreed, I have filed bug and marked as blocker for 3.10.2.
https://bugzilla.redhat.com/show_bug.cgi?id=1447608


>>
> IIUC, the problem is less the add brick operation and more the
> rebalance/fix-layout. It is those that need to be prevented (as someone
> could trigger those without an add-brick).

Yes, that problem seems to be with fix-layout/rebalance and not add-brick.
However, depending on how users have arranged their dir structure, a
add-brick without a fix-layout might be useless for them.

I also had a look at the code to see if I can do the cli/glusterd
change myself. However, sharding is enabled just as a xlator and not
added to glusterd_volinfo_t.
If someone from dht team could work with glusterd team here it would
fix the issue faster.

Action item on Nithya/Atin to assign bug 1447608 to someone. I will
wait for the fix for 3.10.2.

Thanks,
Raghavendra Talur

>
> Nithya
>>
>> Let's get a bug going, and not wait for someone to report it in bugzilla,
>> and also mark it as blocking 3.10.2 release tracker bug.
>>
>> Thanks,
>> Shyam
>>
>> On 05/02/2017 06:20 AM, Pranith Kumar Karampuri wrote:
>>>
>>>
>>>
>>> On Tue, May 2, 2017 at 9:16 AM, Pranith Kumar Karampuri
>>> <pkara...@redhat.com <mailto:pkara...@redhat.com>> wrote:
>>>
>>> Yeah it is a good idea. I asked him to raise a bug and we can move
>>> forward with it.
>>>
>>>
>>> +Raghavendra/Nitya who can help with the fix.
>>>
>>>
>>>
>>> On Mon, May 1, 2017 at 9:07 PM, Joe Julian <j...@julianfamily.org
>>> <mailto:j...@julianfamily.org>> wrote:
>>>
>>>
>>> On 04/30/2017 01:13 AM, lemonni...@ulrar.net
>>> <mailto:lemonni...@ulrar.net> wrote:
>>>
>>> So I was a little but luck. If I has all the hardware
>>> part, probably i
>>> would be firesd after causing data loss by using a
>>> software marked as stable
>>>
>>> Yes, we lost our data last year to this bug, and it wasn't a
>>> test cluster.
>>> We still hear from it from our clients to this day.
>>>
>>> Is known that this feature is causing data loss and
>>> there is no evidence or
>>> no warning in official docs.
>>>
>>> I was (I believe) the first one to run into the bug, it
>>> happens and I knew it
>>> was a risk when installing gluster.
>>> But since then I didn't see any warnings anywhere except
>>> here, I agree
>>> with you that it should be mentionned in big bold letters on
>>> the site.
>>>
>>> Might even be worth adding a warning directly on the cli
>>> when trying to
>>> add bricks if sharding is enabled, to make sure no-one will
>>> destroy a
>>> whole cluster for a known bug.
>>>
>>>
>>> I absolutely agree - or, just disable the ability to add-brick
>>> with sharding enabled. Losing data should never be allowed.
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
>>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>> <http://lists.gluster.org/mailman/listinfo/gluster-devel>
>>>
>>>
>>>
>>>
>>> --
>>> Pranith
>>>
>>> ___
>>> Gluster-users mailing list
>>> gluster-us...@gluster.org <mailto:gluster-us...@gluster.org>
>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>> <http://lists.gluster.org/mailman/listinfo/gluster-users>
>>>
>>>
>>>
>>>
>>> --
>>> Pranith
>>>
>>>
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Release 3.10.2: Scheduled for the 30th of April

[Raghavendra Talur]

On Mon, May 1, 2017 at 3:35 PM, Raghavendra Talur <rta...@redhat.com> wrote:
> We seem to have merged all the intended patches for 3.10.2 except for
> one. While we wait for [1] to be merged, this is a last chance for
> others to point out if any patch is missing. I have verified that no
> backports are missing in 3.10 when compared to 3.8.
>
> Please visit 
> https://review.gluster.org/#/projects/glusterfs,dashboards/dashboard:3-10-dashboard
> to check if any of your patches are pending merge.
>
> To make my work easier, if you could point out any major changes that
> happened for 3.10.2, please comment on
> https://review.gluster.org/#/c/17063/ .
>
> I am looking forward to make announcement on 3rd May and we should
> have builds/packages ready by then. We will be tagging by tomorrow and
> performing a final test with packages before making the announcement.

[1] https://review.gluster.org/#/c/17134/

>
> Thanks,
> Raghavendra Talur
>
> On Mon, Apr 17, 2017 at 10:31 AM, Raghavendra Talur <rta...@redhat.com> wrote:
>> Hi,
>>
>> It's time to prepare the 3.10.2 release, which falls on the 30th of
>> each month, and hence would be April-30th-2017 this time around.
>>
>> This mail is to call out the following,
>>
>> 1) Are there any pending *blocker* bugs that need to be tracked for
>> 3.10.2? If so mark them against the provided tracker [1] as blockers
>> for the release, or at the very least post them as a response to this
>> mail
>>
>> 2) Pending reviews in the 3.10 dashboard will be part of the release,
>> *iff* they pass regressions and have the review votes, so use the
>> dashboard [2] to check on the status of your patches to 3.10 and get
>> these going
>>
>> 3) I have made checks on what went into 3.8 post 3.10 release and if
>> these fixes are included in 3.10 branch, the status on this is *green*
>> as all fixes ported to 3.8, are ported to 3.10 as well
>>
>> 4) Empty release notes are posted here [3], if there are any specific
>> call outs for 3.10 beyond bugs, please update the review, or leave a
>> comment in the review, for me to pick it up
>>
>> Thanks,
>> Raghavendra Talur
>>
>> [1] Release bug tracker:
>> https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.10.2
>>
>> [2] 3.10 review dashboard:
>> https://review.gluster.org/#/projects/glusterfs,dashboards/dashboard:3-10-dashboard
>>
>> [3] Release notes WIP: https://review.gluster.org/#/c/17063/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Release 3.10.2: Scheduled for the 30th of April

[Raghavendra Talur]

We seem to have merged all the intended patches for 3.10.2 except for
one. While we wait for [1] to be merged, this is a last chance for
others to point out if any patch is missing. I have verified that no
backports are missing in 3.10 when compared to 3.8.

Please visit 
https://review.gluster.org/#/projects/glusterfs,dashboards/dashboard:3-10-dashboard
to check if any of your patches are pending merge.

To make my work easier, if you could point out any major changes that
happened for 3.10.2, please comment on
https://review.gluster.org/#/c/17063/ .

I am looking forward to make announcement on 3rd May and we should
have builds/packages ready by then. We will be tagging by tomorrow and
performing a final test with packages before making the announcement.

Thanks,
Raghavendra Talur

On Mon, Apr 17, 2017 at 10:31 AM, Raghavendra Talur <rta...@redhat.com> wrote:
> Hi,
>
> It's time to prepare the 3.10.2 release, which falls on the 30th of
> each month, and hence would be April-30th-2017 this time around.
>
> This mail is to call out the following,
>
> 1) Are there any pending *blocker* bugs that need to be tracked for
> 3.10.2? If so mark them against the provided tracker [1] as blockers
> for the release, or at the very least post them as a response to this
> mail
>
> 2) Pending reviews in the 3.10 dashboard will be part of the release,
> *iff* they pass regressions and have the review votes, so use the
> dashboard [2] to check on the status of your patches to 3.10 and get
> these going
>
> 3) I have made checks on what went into 3.8 post 3.10 release and if
> these fixes are included in 3.10 branch, the status on this is *green*
> as all fixes ported to 3.8, are ported to 3.10 as well
>
> 4) Empty release notes are posted here [3], if there are any specific
> call outs for 3.10 beyond bugs, please update the review, or leave a
> comment in the review, for me to pick it up
>
> Thanks,
> Raghavendra Talur
>
> [1] Release bug tracker:
> https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.10.2
>
> [2] 3.10 review dashboard:
> https://review.gluster.org/#/projects/glusterfs,dashboards/dashboard:3-10-dashboard
>
> [3] Release notes WIP: https://review.gluster.org/#/c/17063/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Release 3.10.2: Scheduled for the 30th of April

[Raghavendra Talur]

Hi,

It's time to prepare the 3.10.2 release, which falls on the 30th of
each month, and hence would be April-30th-2017 this time around.

This mail is to call out the following,

1) Are there any pending *blocker* bugs that need to be tracked for
3.10.2? If so mark them against the provided tracker [1] as blockers
for the release, or at the very least post them as a response to this
mail

2) Pending reviews in the 3.10 dashboard will be part of the release,
*iff* they pass regressions and have the review votes, so use the
dashboard [2] to check on the status of your patches to 3.10 and get
these going

3) I have made checks on what went into 3.8 post 3.10 release and if
these fixes are included in 3.10 branch, the status on this is *green*
as all fixes ported to 3.8, are ported to 3.10 as well

4) Empty release notes are posted here [3], if there are any specific
call outs for 3.10 beyond bugs, please update the review, or leave a
comment in the review, for me to pick it up

Thanks,
Raghavendra Talur

[1] Release bug tracker:
https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-3.10.2

[2] 3.10 review dashboard:
https://review.gluster.org/#/projects/glusterfs,dashboards/dashboard:3-10-dashboard

[3] Release notes WIP: https://review.gluster.org/#/c/17063/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] git pull failed!

[Raghavendra Talur]

Seems to be fixed now.

On Mon, Apr 17, 2017 at 5:37 AM, Zhengping Zhou
 wrote:
>  Did anyone encount this problem when excute "git pull", this output is :
>
>
> [zzp@42 gluster]$ git pull
> ssh_exchange_identification: Connection closed by remote host
> fatal: Could not read from remote repository.
>
> Please make sure you have the correct access rights
> and the repository exists.
>
>
> I have changed my private key and added the new public key to my
> configuration in gerrit system.
>
> But the problem still not fixed!
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Is anyone else having trouble authenticating with review.gluster.org over ssh?

[Raghavendra Talur]

On Sun, Apr 16, 2017 at 9:07 PM, Raghavendra Talur <rta...@redhat.com> wrote:
> I am not able to login even after specifying the key file
>
> $ ssh -T -vvv -i ~/.ssh/gluster raghavendra-ta...@git.gluster.org
> OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
> debug1: Reading configuration data /home/rtalur/.ssh/config
> debug1: Reading configuration data /etc/ssh/ssh_config
> debug3: /etc/ssh/ssh_config line 56: Including file
> /etc/ssh/ssh_config.d/05-redhat.conf depth 0
> debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
> debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 2: include
> /etc/crypto-policies/back-ends/openssh.txt matched no files
> debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 8: Applying options for *
> debug1: auto-mux: Trying existing master
> debug1: Control socket
> "/tmp/ssh_mux_git.gluster.org_22_raghavendra-talur" does not exist
> debug2: resolving "git.gluster.org" port 22
> debug2: ssh_connect_direct: needpriv 0
> debug1: Connecting to git.gluster.org [8.43.85.171] port 22.
> debug1: Connection established.
> debug1: identity file /home/rtalur/.ssh/gluster type 1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/rtalur/.ssh/gluster-cert type -1
> debug1: Enabling compatibility mode for protocol 2.0
> debug1: Local version string SSH-2.0-OpenSSH_7.4
> ssh_exchange_identification: Connection closed by remote host

Confirmed with Pranith that he is facing same issue.

Nigel/Misc,
Please have a look.


>
> Thanks,
> Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.10 release and testing improvements

[Raghavendra Talur]

On Thu, Feb 2, 2017 at 11:08 AM, Nithya Balachandran
<nbala...@redhat.com> wrote:
>
>
> On 2 February 2017 at 09:26, Nithya Balachandran <nbala...@redhat.com>
> wrote:
>>
>>
>>
>> On 1 February 2017 at 19:27, Raghavendra Talur <rta...@redhat.com> wrote:
>>>
>>> Hi all,
>>>
>>> As we approach release of 3.10, we should take a look at existing
>>> tests are being skipped because of issues. These two weeks should be
>>> right time for us to focus on tests as feature patches have been
>>> merged and is on pause. I have made a list of components and known bad
>>> tests against them[1]. You can also find the list of tests [2].
>>>
>>> I have filed only one issue in github[3] and that is the only blocker
>>> from testing perspective for this release.
>>>
>>>
>>> Thanks,
>>> Raghavendra Talur
>>>
>>> [1] Components and bad tests against them
>>> <->
>>> afr 1
>>> ec 1
>>> gfapi 2
>>> tier 7
>>> dht 1
>>> ec 1
>>> fuse 1
>>> glusterd 1
>>> quota 1
>>> snapshot 1
>>> stripe 2
>>> georep 2
>>> write-behind 1
>>> <->
>>>
>>> [2] List of skipped tests
>>> <--List of skipped tests--->
>>> tests/basic/afr/tarissue.t
>>> tests/basic/ec/ec-background-heals.t
>>> tests/basic/gfapi/bug1291259.t
>>> tests/basic/tier/bug-1214222-directories_missing_after_attach_tier.t
>>> tests/basic/tier/fops-during-migration.t
>>> tests/basic/tier/record-metadata-heat.t
>>> tests/basic/tier/tier-file-create.t
>>> tests/basic/tier/tier_lookup_heal.t
>>> tests/basic/tier/tier-snapshot.t
>>> tests/bugs/distribute/bug-1066798.t
>>
>>
>> This should work once https://review.gluster.org/16457 is merged. I will
>> remove this test post that.
>>
> The test passes consistently on my system with the latest master (and
> without this patch). Can we try running it on a regression setup to see if
> it works there?
>
> What is the best way to proceed here?

Now we don't have access to regression setups anymore. Way to go
forward is to delete the G_TESTDEF line in the test and send that as a
patch.

>
>> Thanks,
>> Nithya
>>
>>
>>>
>>> tests/bugs/ec/bug-1304988.t
>>> tests/bugs/fuse/bug-924726.t
>>> tests/bugs/gfapi/bug-1093594.t
>>> tests/bugs/glusterd/bug-1238706-daemons-stop-on-peer-cleanup.t
>>> tests/bugs/quota/bug-1235182.t
>>>
>>> tests/bugs/snapshot/bug-1140162-file-snapshot-features-encrypt-opts-validation.t
>>> tests/bugs/stripe/bug-1002207.t
>>> tests/bugs/stripe/bug-454.t
>>> tests/bugs/tier/bug-1286974.t
>>> tests/bugs/write-behind/bug-1279730.t
>>> tests/geo-rep/georep-basic-dr-rsync.t
>>> tests/geo-rep/georep-basic-dr-tarssh.t
>>> <->
>>>
>>>
>>> [1] https://github.com/gluster/glusterfs/issues/86
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] 3.10 release and testing improvements

[Raghavendra Talur]

Hi all,

As we approach release of 3.10, we should take a look at existing
tests are being skipped because of issues. These two weeks should be
right time for us to focus on tests as feature patches have been
merged and is on pause. I have made a list of components and known bad
tests against them[1]. You can also find the list of tests [2].

I have filed only one issue in github[3] and that is the only blocker
from testing perspective for this release.


Thanks,
Raghavendra Talur

[1] Components and bad tests against them
<->
afr 1
ec 1
gfapi 2
tier 7
dht 1
ec 1
fuse 1
glusterd 1
quota 1
snapshot 1
stripe 2
georep 2
write-behind 1
<->

[2] List of skipped tests
<--List of skipped tests--->
tests/basic/afr/tarissue.t
tests/basic/ec/ec-background-heals.t
tests/basic/gfapi/bug1291259.t
tests/basic/tier/bug-1214222-directories_missing_after_attach_tier.t
tests/basic/tier/fops-during-migration.t
tests/basic/tier/record-metadata-heat.t
tests/basic/tier/tier-file-create.t
tests/basic/tier/tier_lookup_heal.t
tests/basic/tier/tier-snapshot.t
tests/bugs/distribute/bug-1066798.t
tests/bugs/ec/bug-1304988.t
tests/bugs/fuse/bug-924726.t
tests/bugs/gfapi/bug-1093594.t
tests/bugs/glusterd/bug-1238706-daemons-stop-on-peer-cleanup.t
tests/bugs/quota/bug-1235182.t
tests/bugs/snapshot/bug-1140162-file-snapshot-features-encrypt-opts-validation.t
tests/bugs/stripe/bug-1002207.t
tests/bugs/stripe/bug-454.t
tests/bugs/tier/bug-1286974.t
tests/bugs/write-behind/bug-1279730.t
tests/geo-rep/georep-basic-dr-rsync.t
tests/geo-rep/georep-basic-dr-tarssh.t
<->


[1] https://github.com/gluster/glusterfs/issues/86
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Priority based ping packet for 3.10

[Raghavendra Talur]

On Tue, Jan 24, 2017 at 10:39 AM, Vijay Bellur  wrote:
>
>
> On Thu, Jan 19, 2017 at 8:06 AM, Jeff Darcy  wrote:
>>
>> > The more relevant question would be with TCP_KEEPALIVE and
>> > TCP_USER_TIMEOUT
>> > on sockets, do we really need ping-pong framework in Clients? We might
>> > need
>> > that in transport/rdma setups, but my question is concentrating on
>> > transport/rdma. In other words would like to hear why do we need
>> > heart-beat
>> > mechanism in the first place. One scenario might be a healthy socket
>> > level
>> > connection but an unhealthy brick/client (like a deadlocked one).
>>
>> This is an important case to consider.  On the one hand, I think it
>> answers
>> your question about TCP_KEEPALIVE.  What we really care about is whether a
>> brick's request queue is moving.  In other words, what's the time since
>> the
>> last reply from that brick, and does that time exceed some threshold?  On
>> a
>> busy system, we don't even need ping packets to know that.  We can just
>> use
>> responses on other requests to set/reset that timer.  We only need to send
>> ping packets when our *outbound* queue has remained empty for some
>> fraction
>> of our timeout.
>>
>> However, it's important that our measurements be *end to end* and not just
>> at the transport level.  This is particularly true with multiplexing,
>> where multiple bricks will share and contend on various resources.  We
>> should ping *through* client and server, with separate translators above
>> and below each.  This would give us a true end-to-end ping *for that
>> brick*, and also keep the code nicely modular.
>
>
> +1 to this. Having ping, pong xlators immediately above and below protocol
> translators would also address the problem of epoll threads getting blocked
> in gluster's xlator stacks in busy systems.

IMO, as Jeff mentioned for ping timeout that it is required only when
there are no operations happening, similarly, we can consider
frame-timeout as a mechanism of *end to end* keep alive. What extra
feature(s) do we get by adding a ping-pong xlator couple?

>
> Having said that, I do see value in Rafi's patch that prompted this thread.
> Would it not help to prioritize ping - pong traffic in all parts of the
> gluster stack including the send queue on the client?
>
> Regards,
> Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] static analysis updated

[Raghavendra Talur]

On Mon, Dec 19, 2016 at 11:36 PM, Jeff Darcy <jda...@redhat.com> wrote:
>> Thank you Kaleb. Shall we start somwhere in terms of automation?
>>
>> The cppcheck results look the shortest[1]. If we can commit to fixing all of
>> them in the next 1 month, I can kick off a non-voting smoke job. We'll make
>> it
>> vote after 1 month. I guss the cli and experimental xlator (and a few more
>> xlators, please check log) owners need to confirm that this is something we
>> can
>> target to fix. And commit to keeping fixed.
>
> Before we get to automation, shouldn't we have a discussion about what
> "defects" we should filter out?  For example, we already have a problem
> with compilers spitting out warnings about unused variables in generated
> code, and then promoting those warnings to errors.  Fixing those is more
> trouble than it's worth.  Static analyzers are going to produce even
> more reports of Things That Don't Really Matter, along with a few about
> Actual Serious Problems.  It's a characteristic of the genre.  If we
> don't make any explicit decisions about priorities, it will actually
> take us longer to fix all of the null-pointer errors and array overflows
> and memory leaks as people wade through a sea of lesser defects.

Nigel,
When you automate it, if you could have a function in the script to
filter out warnings that we don't care about then it will make it
easier to get to a consensus on what to filter.

For example:

cppcheck.sh


func ignore warnings()
{
for warnings in warning1 \
warning2 \
; do 

...
}

This way, developers will have a way to send patches on what to filter
and if we add enough warning types into the list, we could turn voting
on after a month.

Jeff,
would that work?

Thanks,
Raghavendra Talur

> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Ability to skip regression jobs?

[Raghavendra Talur]

On Tue, Dec 20, 2016 at 9:31 PM, Nigel Babu <nig...@redhat.com> wrote:
> Hello folks,
>
> There's a few conversations I've been having about reducing regressions
> workloads. Is it useful to have a mechanism to suggest in a commit message to
> skip regressions on a job? For instance, you'd say [ci-skip] and it would skip
> your commit.

We already have some form of this:
1. If it a Work in progress patch, don't give a +1 verified on the
patch and we won't run regression.
2. If it is a doc only change, we don't run regression.
3. If it is a distaf only change, we don't run regression.

If there is any genuine reason for a type of change to skip
regression, it should be configured on the jenkins side as we do for
changes above.

>
> My assumption is that we may have a genuine use-case where we don't want to 
> run
> regressions. For example, changes to packaging, or changes to some files in
> extras. We could maintain a larger skip list, but that approach doesn't scale
> very well.
Packaging and extras changes do need to be tested, an error in systemd
unit file or an error in where it is placed is as critical as a code
bug.

>
> On the other hand, opening this up means it's likely to be abused. So we may
> have to do good policing.
This will certainly happen. I do not recommend any mechanism which
could be used by patch owner.

>
> Anyone have ideas either way?
One way to reduce load is to figure out a way to award +1s to parent
patches when a dependent patch passes regression. It is recommended
that a change be split into as many smaller sets as possible.
Currently, if we send 4 patches that need to be serially merged, all 4
have to pass regressions. Instead, if Jenkins offered a way to block
merging of subset of patches and made it possible to take all 4 in in
one go, we could run tests only on top of these patches. The last time
I checked, such a feature did not exist in Jenkins.

Thanks,
Raghavendra Talur

>
> --
> nigelb
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] getting "Transport endpoint is not connected" in glusterfs mount log file.

[Raghavendra Talur]

On 11-Nov-2016 11:50, "Mohammed Rafi K C"  wrote:
>
> Hi Abhishek,
>
> Could you please see if you are bricks are healthy or not, may be you can
do a gluster volume status or you can look into the logs. If bricks are not
running can you please attach the bricks logs in /var/log/gluster/bricks/ .
>
>
> Rafi KC

Also,  please provide details of Gluster version and setup.
>
>
> On 11/11/2016 10:20 AM, ABHISHEK PALIWAL wrote:
>>
>> Hi,
>>
>> Its an urgent case.
>>
>> Atleast provide your views on this
>>
>> On Wed, Nov 9, 2016 at 11:08 AM, ABHISHEK PALIWAL <
abhishpali...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> We could see that sync is getting failed to sync the GlusterFS bricks
due to error trace "Transport endpoint is not connected "
>>>
>>> [2016-10-31 04:06:03.627395] E [MSGID: 114031]
[client-rpc-fops.c:1673:client3_3_finodelk_cbk] 0-c_glusterfs-client-9:
remote operation failed [Transport endpoint is not connected]
>>> [2016-10-31 04:06:03.628381] I [socket.c:3308:socket_submit_request]
0-c_glusterfs-client-9: not connected (priv->connected = 0)
>>> [2016-10-31 04:06:03.628432] W [rpc-clnt.c:1586:rpc_clnt_submit]
0-c_glusterfs-client-9: failed to submit rpc-request (XID: 0x7f5f Program:
GlusterFS 3.3, ProgVers: 330, Proc: 30) to rpc-transport
(c_glusterfs-client-9)
>>> [2016-10-31 04:06:03.628466] E [MSGID: 114031]
[client-rpc-fops.c:1673:client3_3_finodelk_cbk] 0-c_glusterfs-client-9:
remote operation failed [Transport endpoint is not connected]
>>> [2016-10-31 04:06:03.628475] I [MSGID: 108019]
[afr-lk-common.c:1086:afr_lock_blocking] 0-c_glusterfs-replicate-0: unable
to lock on even one child
>>> [2016-10-31 04:06:03.628539] I [MSGID: 108019]
[afr-transaction.c:1224:afr_post_blocking_inodelk_cbk]
0-c_glusterfs-replicate-0: Blocking inodelks failed.
>>> [2016-10-31 04:06:03.628630] W [fuse-bridge.c:1282:fuse_err_cbk]
0-glusterfs-fuse: 20790: FLUSH() ERR => -1 (Transport endpoint is not
connected)
>>> [2016-10-31 04:06:03.629149] E [rpc-clnt.c:362:saved_frames_unwind]
(-->
/usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8ab79f58]
(--> /usr/lib64/libgfrpc.so.0(saved_frames_unwind-0x1b7a0)[0x3fff8ab1dc90]
(--> /usr/lib64/libgfrpc.so.0(saved_frames_destroy-0x1b638)[0x3fff8ab1de10]
(-->
/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19af8)[0x3fff8ab1fb18]
(--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8ab20808]
) 0-c_glusterfs-client-9: forced unwinding frame type(GlusterFS 3.3)
op(LOOKUP(27)) called at 2016-10-31 04:06:03.624346 (xid=0x7f5a)
>>> [2016-10-31 04:06:03.629183] I [rpc-clnt.c:1847:rpc_clnt_reconfig]
0-c_glusterfs-client-9: changing port to 49391 (from 0)
>>> [2016-10-31 04:06:03.629210] W [MSGID: 114031]
[client-rpc-fops.c:2971:client3_3_lookup_cbk] 0-c_glusterfs-client-9:
remote operation failed. Path:
/loadmodules_norepl/CXC1725605_P93A001/cello/emasviews
(b0e5a94e-a432-4dce-b86f-a551555780a2) [Transport endpoint is not connected]
>>>
>>>
>>> Could you please tell us the reason why we are getting these trace and
how to resolve this.
>>>
>>> Logs are attached here please share your analysis.
>>>
>>> Thanks in advanced
>>>
>>> --
>>> Regards
>>> Abhishek Paliwal
>>
>>
>>
>>
>> --
>>
>>
>>
>>
>> Regards
>> Abhishek Paliwal
>>
>>
>> ___ Gluster-users mailing
list gluster-us...@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Please pause merging patches to 3.9 waiting for just one patch

[Raghavendra Talur]

On 10-Nov-2016 20:52, "Vijay Bellur"  wrote:
>
> On Thu, Nov 10, 2016 at 10:16 AM, Manikandan Selvaganesh
>  wrote:
> > Enabling/disabling quota or removing limits are the ways in which
quota.conf
> > is regenerated to the later version. It works properly. And as Pranith
said,
> > both enabling/disabling takes a lot of time to crawl(though now much
faster
> > with enhanced quota enable/disable process) which we cannot suggest the
> > users with a lot of quota configuration. Resetting the limit using
> > limit-usage does not work properly. I have tested the same. The
workaround
> > is based on the user setup here. I mean the steps he exactly used in
order
> > matters here. The workaround is not so generic. However, quota
> > enable/disable would regenerate the file on any case.
> >
> > IMO, this bug is critical. I am not sure though how often users would
hit
> > this - Updating from 3.6 to latest versions. From 3.7 to latest, its
fine,
> > this has nothing to do with this patch.
> >
>
> Given that we are done with the last release in 3.6.x, I think there
> would be users looking to upgrade.  My vote is to include the
> necessary patches in 3.9 and not let users go through unnatural
> workflows to get quota working again in 3.9.0.

+1, especially considering this is a ".0" release.

>
> Thanks,
> Vijay
> ___
> maintainers mailing list
> maintain...@gluster.org
> http://www.gluster.org/mailman/listinfo/maintainers
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Please pause merging patches to 3.9 waiting for just one patch

[Raghavendra Talur]

On Thu, Nov 10, 2016 at 7:12 AM, Pranith Kumar Karampuri
 wrote:
> hi,
>   The only problem left was EC taking more time. This should affect
> small files a lot more. Best way to solve it is using compound-fops. So for
> now I think going ahead with the release is best.
>
> We are waiting for Raghavendra Talur's http://review.gluster.org/#/c/15778
> before going ahead with the release. If we missed any other crucial patch
> please let us know.

This patch is now merged.

>
> Will make the release as soon as this patch is merged.
>
> --
> Pranith & Aravinda
>
> ___
> maintainers mailing list
> maintain...@gluster.org
> http://www.gluster.org/mailman/listinfo/maintainers
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Possible problem introduced by http://review.gluster.org/15573

[Raghavendra Talur]

On Mon, Oct 24, 2016 at 12:39 PM, Xavier Hernandez <xhernan...@datalab.es>
wrote:

> Hi Soumya,
>
>
> On 21/10/16 16:15, Soumya Koduri wrote:
>
>>
>>
>> On 10/21/2016 06:35 PM, Soumya Koduri wrote:
>>
>>> Hi Xavi,
>>>
>>> On 10/21/2016 12:57 PM, Xavier Hernandez wrote:
>>>
>>>> Looking at the code, I think that the added fd_unref() should only be
>>>> called if the fop preparation fails. Otherwise the callback already
>>>> unreferences the fd.
>>>>
>>>> Code flow:
>>>>
>>>> * glfs_fsync_async_common() takes an fd ref and calls STACK_WIND passing
>>>> that fd.
>>>> * Just after that a ref is released.
>>>> * When glfs_io_async_cbk() is called another ref is released.
>>>>
>>>> Note that if fop preparation fails, a single fd_unref() is called, but
>>>> on success two fd_unref() are called.
>>>>
>>>
>>> Sorry for the inconvenience caused. I think its patch#15573 hasn't
>>> caused the problem but has highlighted another ref leak in the code.
>>>
>>> From the code I see that glfs_io_async_cbk() does fd_unref (glfd->fd)
>>> but not the fd passed in STACK_WIND_COOKIE() of the fop.
>>>
>>> If I take any fop, for eg.,
>>> glfs_fsync_common() {
>>>
>>>fd = glfs_resolve_fd (glfd->fs, subvol, glfd);
>>>
>>>
>>> }
>>>
>>> Here in glfs_resolve_fd ()
>>>
>>> fd_t *
>>> __glfs_resolve_fd (struct glfs *fs, xlator_t *subvol, struct glfs_fd
>>> *glfd)
>>> {
>>> fd_t *fd = NULL;
>>>
>>> if (glfd->fd->inode->table->xl == subvol)
>>> return fd_ref (glfd->fd);
>>>
>>> Here we can see that  we are taking extra ref additional to the
>>>>>>>>>>>
>>>>>>>>>> ref already taken for glfd->fd. That means the caller of this
>>> function
>>> needs to fd_unref(fd) irrespective of subsequent fd_unref (glfd->fd).
>>>
>>> fd = __glfs_migrate_fd (fs, subvol, glfd);
>>> if (!fd)
>>> return NULL;
>>>
>>>
>>> if (subvol == fs->active_subvol) {
>>> fd_unref (glfd->fd);
>>> glfd->fd = fd_ref (fd);
>>> }
>>>
>>> I think the issue is here during graph_switch(). You have
>>>>>>>>>>>
>>>>>>>>>> mentioned as well that the crash happens post graph_switch. Maybe
>>> here
>>> we are missing an extra ref to be taken for fd additional to glfd->fd. I
>>> need to look through __glfs_migrate_fd() to confirm that. But these are
>>> my initial thoughts.
>>>
>>
>> Looking into this, I think we should fix glfs_io_async_cbk() not to
>> fd_unref(glfd->fd). glfd->fd should be active though out the lifetime of
>> glfd (i.e, until it is closed). Thoughts?
>>
>
> I don't know gfapi internals in deep, but at first sight I think this
> would be the right think to do. Assuming that glfd will keep a reference to
> the fd until it's destroyed, and that a glfd reference is taken during the
> lifetime of each request that needs it, the fd_unref() in
> glfd_io_async_cbk() seems unnecessary. I think it was there just to release
> the fd acquired if glfs_resolve_fd(), but it's better to place it where
> it's now.
>
> Another question is if we really need to take an additional reference in
> glfs_resolve_fd() ?
>

This answers the first question too, we don't need the additional ref in
glfs_resolve_fd() now that we have ref accounting in glfd. This confusion
is because earlier there was no ref accounting for glfd and only refs were
on fd_t.
No, we don't need this additional reference.


>
> Can an fd returned by this function live more time than the associated
> glfd in some circumstances ?
>
> Also could you please check if it is the second/subsequent fsync_async()
>> call which results in crash?
>>
>
> I'll try to test it as soon as possible, but this is on a server that we
> need to put in production very soon and we have decided to go with fuse for
> now. We'll have a lot of work to do this week. Once I have some free time
> I'll build a test environment to check it, probably next week.
>
>
I have not been able to test this out completely. Theoretically, I don't
see any possibility where fd can outlive a glfd that is poin

Re: [Gluster-devel] relative ordering of writes to same file from two different fds

[Raghavendra Talur]

On Mon, Sep 26, 2016 at 1:05 PM, Ryan Ding <ryan.d...@open-fs.com> wrote:

>
> > On Sep 23, 2016, at 8:59 PM, Jeff Darcy <jda...@redhat.com> wrote:
> >
> >>> write-behind: implement causal ordering and other cleanup
> >>
> >>> Rules of causal ordering implemented:¬
> >>
> >>> - If request A arrives after the acknowledgement (to the app,¬
> >>
> >>> i.e, STACK_UNWIND) of another request B, then request B is¬
> >>
> >>> said to have 'caused' request A.¬
> >>
> >>
> >> With the above principle, two write requests (p1 and p2 in example
> above)
> >> issued by _two different threads/processes_ there need _not always_ be a
> >> 'causal' relationship (whether there is a causal relationship is purely
> >> based on the "chance" that write-behind chose to ack one/both of them
> and
> >> their timing of arrival).
> >
> > I think this is an issue of terminology.  While it's not *certain* that B
> > (or p1) caused A (or p2), it's *possible*.  Contrast with the case where
> > they overlap, which could not possibly happen if the application were
> > trying to ensure order.  In the distributed-system literature, this is
> > often referred to as a causal relationship even though it's really just
> > the possibility of one, because in most cases even the possibility means
> > that reordering would be unacceptable.
> >
> >> So, current write-behind is agnostic to the
> >> ordering of p1 and p2 (when done by two threads).
> >>
> >> However if p1 and p2 are issued by same thread there is _always_ a
> causal
> >> relationship (p2 being caused by p1).
> >
> > See above.  If we feel bound to respect causal relationships, we have to
> > be pessimistic and assume that wherever such a relationship *could* exist
> > it *does* exist.  However, as I explained in my previous message, I don't
> > think it's practical to provide such a guarantee across multiple clients,
> > and if we don't provide it across multiple clients then it's not worth
> > much to provide it on a single client.  Applications that require such
> > strict ordering shouldn't use write-behind, or should explicitly flush
> > between writes.  Otherwise they'll break unexpectedly when parts are
> > distributed across multiple nodes.  Assuming that everything runs on one
> > node is the same mistake POSIX makes.  The assumption was appropriate
> > for an earlier era, but not now for a decade or more.
>
> We can separate this into 2 question:
> 1. should it be a causal relationship in local application ?
> 2. should it be a causal relationship in a distribute application ?
> I think the answer to #2 is ’NO’. this is an issue that distribute
> application should resolve. the way to resolve it is either use distribute
> lock we provided or use their own way (fsync is required in such condition).
>
True.


> I think the answer to #1 is ‘YES’. because buffer io should not involve
> new data consistency problem than no-buffer io. it’s very common that a
> local application will assume underlying file system to be.
> further more, compatible to linux page cache will always to be a better
> practice way, because there is a lot local applications that has already
> rely on its semantics.
>

I agree. If my understanding, this is the same model that write-behind uses
as of today if we don't consider the patch proposed. Write-behind orders
all causal operations on the inode(file object) irrespective of the FD used.
This particular patch brings a small modification where it lets the FSYNC
and FLUSH FOPs bypass the order as long as they are not on the same FD as
the pending WRITE FOP.

Thanks,
Raghavendra Talur


> Thanks,
> Ryan
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] relative ordering of writes to same file from two different fds

[Raghavendra Talur]

On Wed, Sep 21, 2016 at 6:32 PM, Ric Wheeler <ricwhee...@gmail.com> wrote:

> On 09/21/2016 08:06 AM, Raghavendra Gowdappa wrote:
>
>> Hi all,
>>
>> This mail is to figure out the behavior of write to same file from two
>> different fds. As Ryan quotes in one of comments,
>>
>> 
>>
>> I think it’s not safe. in this case:
>> 1. P1 write to F1 use FD1
>> 2. after P1 write finish, P2 write to the same place use FD2
>> since they are not conflict with each other now, the order the 2 writes
>> send to underlying fs is not determined. so the final data may be P1’s or
>> P2’s.
>> this semantics is not the same with linux buffer io. linux buffer io will
>> make the second write cover the first one, this is to say the final data is
>> P2’s.
>> you can see it from linux NFS (as we are all network filesystem)
>> fs/nfs/file.c:nfs_write_begin(), nfs will flush ‘incompatible’ request
>> first before another write begin. the way 2 request is determine to be
>> ‘incompatible’ is that they are from 2 different open fds.
>> I think write-behind behaviour should keep the same with linux page cache.
>>
>> 
>>
>
> I think that how this actually would work is that both would be written to
> the same page in the page cache (if not using buffered IO), so as long as
> they do not happen at the same time, you would get the second P2 copy of
> data each time.
>

I apologize if my understanding is wrong but IMO this is exactly what we do
in write-behind too. The cache is inode based and ensures that writes are
ordered irrespective of the FD used for the write.


Here is the commit message which brought the change
-
write-behind: implement causal ordering and other cleanup


Rules of causal ordering implemented:¬






 - If request A arrives after the acknowledgement (to the app,¬

   i.e, STACK_UNWIND) of another request B, then request B is¬

   said to have 'caused' request A.¬



- (corollary) Two requests, which at any point of time, are¬

   unacknowledged simultaneously in the system can never 'cause'¬

   each other (wb_inode->gen is based on this)¬



 - If request A is caused by request B, AND request A's region¬

   has an overlap with request B's region, then then the fulfillment¬

   of request A is guaranteed to happen after the fulfillment of B.¬



 - FD of origin is not considered for the determination of causal¬

   ordering.¬



 - Append operation's region is considered the whole file.¬



 Other cleanup:¬



 - wb_file_t not required any more.¬



 - wb_local_t not required any more.¬



 - O_RDONLY fd's operations now go through the queue to make sure¬

   writes in the requested region get fulfilled be
---

Thanks,
Raghavendra Talur


>
> Same story for using O_DIRECT - that write bypasses the page cache and
> will update the data directly.
>
> What might happen in practice though is that your applications might use
> higher level IO routines and they might buffer data internally. If that
> happens, there is no ordering that is predictable.
>
> Regards,
>
> Ric
>
>
>
>> However, my understanding is that filesystems need not maintain the
>> relative order of writes (as it received from vfs/kernel) on two different
>> fds. Also, if we have to maintain the order it might come with increased
>> latency. The increased latency can be because of having "newer" writes to
>> wait on "older" ones. This wait can fill up write-behind buffer and can
>> eventually result in a full write-behind cache and hence not able to
>> "write-back" newer writes.
>>
>> * What does POSIX say about it?
>> * How do other filesystems behave in this scenario?
>>
>>
>> Also, the current write-behind implementation has the concept of
>> "generation numbers". To quote from comment:
>>
>> 
>>
>>  uint64_t gen;/* Liability generation number. Represents
>>  the current 'state' of liability. Every
>>  new addition to the liability list bumps
>>  the generation number.
>>
>>
>> a newly arrived request
>> is only required
>>  to perform causal checks against the
>> entries
>>  in the liability list which were present
>>  at the time of its addition. the
&g

Re: [Gluster-devel] [Heketi] Block store related API design discussion

[Raghavendra Talur]

On Mon, Sep 12, 2016 at 11:30 PM, Prasanna Kalever 
wrote:

> Hi all,
>
> This mail is open for discussion on gluster block store integration with
> heketi and its REST API interface design constraints.
>
>
>  ___ Volume Request ...
> |
> |
> PVC claim -> Heketi --->|
> |
> |
> |
> |
> |__ BlockCreate
> |   |
> |   |__ BlockInfo
> |   |
> |___ Block Request (APIS)-> |__ BlockResize
> |
> |__ BlockList
> |
> |__ BlockDelete
>
> Heketi will have block API and volume API, when user submit a Persistent
> volume claim, Kubernetes provisioner based on the storage class(from PVC)
> talks to heketi for storage, heketi intern calls block or volume API's
> based on request.
>

This is probably wrong. It won't be Heketi calling block or volume APIs. It
would be Kubernetes calling block or volume API *of* Heketi.


> With my limited understanding, heketi currently creates clusters from
> provided nodes, creates volumes and handover them to the user.
> For block related API's, it has to deal with files right ?
>
> Here is how block API's look like in short-
> Create: heketi has to create file in the volume and export it as a iscsi
> target device and hand it over to user.
> Info: show block stores information across all the clusters, connection
> info, size etc.
> resize: resize the file in the volume, refresh connections from initiator
> side
> List: List the connections
> Delete: logout the connections and delete the file in the gluster volume
>
> Couple of questions:
> 1. Should Block API have sub API's such as FileCreate, FileList,
> FileResize, File delete and etc then get it used in Block API as they
> mostly deal with files.
>

IMO, Heketi should not expose any File related API. It should only have
APIs to service request for block devices; how the block devices are
created and modified is an implementation detail.


> 2. How do we create the actual file in the volume, meaning using FUSE
> mount (which may involve an extra process running) or gfapi, again if gfapi
> should we go with c API's, python bindings or go bindings ?
>
3. Should we get targetcli related (LUN exporting) setup done from heketi
> or do we seek help from gdeploy for this ?
>

I would prefer to either have it in Heketi or in Kubernetes. If the API in
Heketi promises just the creation of block device, then the rest of the
implementation should be in Kubernetes(the export part). If the API in
Heketi promises create and export both, I would say Heketi should have the
implementation within itself.


>
>
> Thoughts?
>
> Note: nothing is fixed as put in this mail, its all just part of initial
> discussions.
>
>
> Cheers,
> --
> Prasanna
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Request to provide PASS flags to a patch in gerrit

[Raghavendra Talur]

Hi All,

We have a test [1] which is causing hangs in NetBSD. We have not been able
to debug the issue yet.
It could be because the bash script does not comply with posix guidelines
or that there is a bug in the brick code.

However, as we have 3.9 merge deadline tomorrow this is causing the test
pipeline to grow a lot and needing manual intervention.
I recommend we disable this test for now. I request Kaushal to provide pass
flags to the patch [2] for faster merge.


[1] ./tests/features/lock_revocation.t
[2] http://review.gluster.org/#/c/15374/


Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.9. feature freeze status check

[Raghavendra Talur]

On Fri, Aug 26, 2016 at 9:38 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> hi,
>   Now that we are almost near the feature freeze date (31st of Aug),
> want to get a sense if any of the status of the features.
>
> Please respond with:
> 1) Feature already merged
> 2) Undergoing review will make it by 31st Aug
> 3) Undergoing review, but may not make it by 31st Aug
> 4) Feature won't make it for 3.9.
>
> I added the features that were not planned(i.e. not in the 3.9 roadmap
> page) but made it to the release and not planned but may make it to release
> at the end of this mail.
> If you added a feature on master that will be released as part of 3.9.0
> but forgot to add it to roadmap page, please let me know I will add it.
>
> Here are the features planned as per the roadmap:
> 1) Throttling
> Feature owner: Ravishankar
>
> 2) Trash improvements
> Feature owners: Anoop, Jiffin
>
> 3) Kerberos for Gluster protocols:
> Feature owners: Niels, Csaba
>
> 4) SELinux on gluster volumes:
> Feature owners: Niels, Manikandan
>
> 5) Native sub-directory mounts:
> Feature owners: Kaushal, Pranith
>
> 6) RichACL support for GlusterFS:
> Feature owners: Rajesh Joseph
>
> 7) Sharemodes/Share reservations:
> Feature owners: Raghavendra Talur, Poornima G, Soumya Koduri, Rajesh
> Joseph, Anoop C S
>

Feature won't make it for 3.9.
Will target for next release.


>
> 8) Integrate with external resource management software
> Feature owners: Kaleb Keithley, Jose Rivera
>
> 9) Python Wrappers for Gluster CLI Commands
> Feature owners: Aravinda VK
>
> 10) Package and ship libgfapi-python
> Feature owners: Prashant Pai
>
> 11) Management REST APIs
> Feature owners: Aravinda VK
>
> 12) Events APIs
> Feature owners: Aravinda VK
>
> 13) CLI to get state representation of a cluster from the local glusterd
> pov
> Feature owners: Samikshan Bairagya
>
> 14) Posix-locks Reclaim support
> Feature owners: Soumya Koduri
>
> 15) Deprecate striped volumes
> Feature owners: Vijay Bellur, Niels de Vos
>
> 16) Improvements in Gluster NFS-Ganesha integration
> Feature owners: Jiffin Tony Thottan, Soumya Koduri
>
> *The following need to be added to the roadmap:*
>
> Features that made it to master already but were not palnned:
> 1) Multi threaded self-heal in EC
> Feature owner: Pranith (Did this because serkan asked for it. He has 9PB
> volume, self-healing takes a long time :-/)
>
> 2) Lock revocation (Facebook patch)
> Feature owner: Richard Wareing
>
> Features that look like will make it to 3.9.0:
> 1) Hardware extension support for EC
> Feature owner: Xavi
>
> 2) Reset brick support for replica volumes:
> Feature owner: Anuradha
>
> 3) Md-cache perf improvements in smb:
> Feature owner: Poornima
>
> --
> Pranith
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Automated Gluster setups using Vagrant and gdeploy

[Raghavendra Talur]

Hi all,

I have blogged about using vagrant and gdeploy to create clusters of
Gluster easily.
Please have a look at
http://blog.raghavendratalur.in/2016/08/automated-gluster-cluster-creation-for.html


Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Reducing regression runs (hopefully)

[Raghavendra Talur]

>
> >
> > On the more ambitious side, I think we could also optimize around the
> idea
> > that some parts of the system aren't even present for some tests.  For
> > example, most of our tests don't even use GFAPI and won't be affected by
> a
> > GFAPI-only change.  Conversely, GFAPI tests won't be affected by a
> FUSE-only
> > change.  AFR, EC, and JBR code and tests are mutually exclusive in much
> the
> > same way.  We have the burn-in test to catch "stragglers" and git to
> revert
> > any patch with surprising effects, so IMO (at least on master) we should
> be
> > pretty aggressive about pruning out tests that provide little value.
>
> Raghavendra has been working on a patchset that does a better job of
> segregating tests by module. I'll let him explain the specifics.
>

 tests directory uses two different taxonomies today. One is by type of
tests where you find bugs/ performance/ basic/ etc. The other of type of
feature where you have features/ experimental/ bitrot/ georep/ etc. I am
working on moving all into first category where we have

1. functional/ basic
2. regression
3. performance (example: measure time for kernel untar with and without
patch)
4. integration (tests with samba/qemu/nfs-ganesha)
5. stress (ambitious for now, mentioned for completeness)
6. upgrade/update (we should not be breaking updates)


> My vision in this regard is something like this:
> * A patchset gets Verified +1.
> * A meta job is kicked off which determines regression jobs to run.
>   If the patch only touches GFAPI, we kick off the GFAPI regression tests.
> If
>   it touches multiple modules, we kick off the tests for these specific
>   modules.
>
As explained in the other reply this is not that trivial. Most of the
components do interact with each other and may break the functionality. It
is only modules which provide alternate functionality like AFR, EC and JBR
that can be tested exclusive manner.



> * The results for each platform are aggregated under our current labels
> similar
>   to how we aggregate results of multiple tests into one label for smoke.
>

This is possible.


>
> Am I being too ambitious here? Is there something I'm missing?
>
> --
> nigelb
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Reducing regression runs (hopefully)

[Raghavendra Talur]

On Mon, Jul 25, 2016 at 6:08 PM, Jeff Darcy  wrote:

> > I have a few proposals to reduce this turn around time:
> >
> > 1. We do not clear the Verified tag. This means if you want to re-run
> >regressions you have to manually trigger it. If your patch is rebased
> on
> >top
> >of another patch, you may have to retrigger failing regressions
> manually.
> > 2. We do give automatic +1 for regressions if the change is *only* in
> >`extras/`, to MAINTAINERS file and other no-op changes. Please
> correct me
> >here. I think the changes here do not affect regressions. If I'm
> wrong and
> >they do, I'd love to know which files do affect regressions. I've
> taken
> >the
> >MAINTAINERS file as an example, I'm also curious to know what other
> no-op
> >changes can be made.
>
> I think you're on the right track here, Nigel.  :)  I'd also add that
> changes
> to one .t file shouldn't require that any others be run (the same is not
> true
> of .rc files though).
>

+1


>
> On the more ambitious side, I think we could also optimize around the idea
> that some parts of the system aren't even present for some tests.  For
> example, most of our tests don't even use GFAPI and won't be affected by a
> GFAPI-only change.  Conversely, GFAPI tests won't be affected by a
> FUSE-only
> change.  AFR, EC, and JBR code and tests are mutually exclusive in much the
> same way.  We have the burn-in test to catch "stragglers" and git to revert
> any patch with surprising effects, so IMO (at least on master) we should be
> pretty aggressive about pruning out tests that provide little value.
>

 Yes. We introduced #G_TESTDEF_* keys in our .t files with the same idea.
Currently, we have two keys
a.  G_TESTDEF_TEST_STATUS_CENTOS6
b.  G_TESTDEF_TEST_STATUS_NETBSD7

Value for these keys is used by our framework to determine[1] whether to
run the test or not. It is fairly easy to add more keys. To take the same
example of a GFAPI change,
G_TESTDEF_TESTS_COMPONENTS="gfapi,dht,afr,"
G_TESTDEF_NO_IMPACT_COMPONENTS="fuse"

Combination of values from these two keys would decide whether to trigger
the test for a change in a component or not.

If you wish to provide such feature and implement in any different way,
ensure that such feature should be usable by the developer on his/her
laptop. Hence, it is better if it is developed in the framework than in
jenkins/gerrit configuration.


[1] http://review.gluster.org/#/c/13393/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Reducing regression runs (hopefully)

[Raghavendra Talur]

>
>
>
> 1. We do not clear the Verified tag. This means if you want to re-run
>regressions you have to manually trigger it. If your patch is rebased
> on top
>of another patch, you may have to retrigger failing regressions
> manually.
>
Yes, matching the settings for Verified flag to that of regressions is
right thing to do.
Users can use the rechech netbsd/centos/smoke option to retrigger specific
tests.


> 2. We do give automatic +1 for regressions if the change is *only* in
>`extras/`, to MAINTAINERS file and other no-op changes. Please correct
> me
>here. I think the changes here do not affect regressions. If I'm wrong
> and
>they do, I'd love to know which files do affect regressions. I've taken
> the
>MAINTAINERS file as an example, I'm also curious to know what other
> no-op
>changes can be made.
>

Unfortunately, the directory structure isn't that organised. What you say
holds true for MAINTAINERS file, tests/distaf/* and doc/*. We skip
regressions for changes in distaf/* and doc/* today. Extras folder has code
though.

Given below is a simple grep which shows files that reside in extras but
get installed as part of make/packages. Some of the scripts found in this
list are tested in the regression tests.

45:extras/peer_add_secret_pub
214:extras/Makefile
215:extras/glusterd.vol
216:extras/cliutils/Makefile
217:extras/init.d/Makefile
218:extras/init.d/glusterd.plist
219:extras/init.d/glusterd-Debian
220:extras/init.d/glusterd-Redhat
221:extras/init.d/glusterd-FreeBSD
222:extras/init.d/glusterd-SuSE
223:extras/ganesha/Makefile
224:extras/ganesha/config/Makefile
225:extras/ganesha/scripts/Makefile
226:extras/ganesha/ocf/Makefile
227:extras/systemd/Makefile
228:extras/systemd/glusterd.service
229:extras/systemd/glustereventsd.service
230:extras/run-gluster.tmpfiles
231:extras/benchmarking/Makefile
232:extras/hook-scripts/Makefile
233:extras/ocf/Makefile
234:extras/ocf/glusterd
235:extras/ocf/volume
236:extras/LinuxRPM/Makefile
237:extras/geo-rep/Makefile
238:extras/geo-rep/schedule_georep.py
239:extras/firewalld/Makefile
240:extras/hook-scripts/add-brick/Makefile
241:extras/hook-scripts/add-brick/pre/Makefile
242:extras/hook-scripts/add-brick/post/Makefile
243:extras/hook-scripts/start/Makefile
244:extras/hook-scripts/start/post/Makefile
245:extras/hook-scripts/set/Makefile
246:extras/hook-scripts/set/post/Makefile
247:extras/hook-scripts/stop/Makefile
248:extras/hook-scripts/stop/pre/Makefile
249:extras/hook-scripts/reset/Makefile
250:extras/hook-scripts/reset/post/Makefile
251:extras/hook-scripts/reset/pre/Makefile
252:extras/snap_scheduler/Makefile
720:# only install scripts from extras/geo-rep when enabled
722:  GEOREP_EXTRAS_SUBDIR=geo-rep
724:AC_SUBST(GEOREP_EXTRAS_SUBDIR)
2076:install -D -p -m 0644 extras/glusterd-sysconfig \
2137:install -D -p -m 0644 extras/glusterfs-logrotate \
2144:install -D -p -m 0644 extras/glusterfs-georep-logrotate \
2170:install -p -m 0744 -D extras/command-completion/gluster.bash \
2521:%doc extras/clear_xattrs.sh
2933:- Include extras/clear_xattrs.sh in the glusterfs-server sub-package
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] What tranpsort type in glfs_set_volfile_server() exactly mean ?

[Raghavendra Talur]

On Mon, Jul 18, 2016 at 4:35 PM, Raghavendra Talur <rta...@redhat.com>
wrote:

>
>
> On Mon, Jul 18, 2016 at 4:10 PM, Prasanna Kalever <pkale...@redhat.com>
> wrote:
>
>> Hey Team,
>>
>>
>> My understanding is that @transport argumet in
>> glfs_set_volfile_server() is meant for specifying transport used in
>> fetching volfile server,
>>
>
> Yes, @transport arg here is transport to use for fetching volfile.
>
>
>> IIRC which currently supports tcp and unix only...
>>
> Yes, this is correct too.
>
>
>>
>> The doc here
>> https://github.com/gluster/glusterfs/blob/master/api/src/glfs.h
>> +166 shows the rdma as well, which is something I cannot digest.
>>
> This is doc written with assumption that rdma would work too.
>
>
>>
>>
>> Can someone correct me ?
>>
>> Have we ever supported volfile fetch over rdma ?
>>
>
> I think no. To test, you would have to set rdma as only transport option
> in glusterd.vol and see what happens in volfile fetch.
>

Rafi, Prasanna and I looked at the code and verified that volfile is
fetched over tcp in the above case.

Here is the complete scenario
1. option: unix, volfile is fetched over unix.
2. option:tcp, volfile is fetched over socket.
3. option:rdma, volfile is fetched over socket with no warning about no
support for rdma.

This gives an illusion to user that volfile is also fetched over rdma,
whereas we used tcp connection for it. I would recommend deprecating the
volfile-server-transport option's overloaded usecase of deciding IO
transport in 3.9 version of Gluster and having a different option name for
it.

Thanks,
Raghavendra Talur


>
>
> IMO, fetching volfile over rdma is an overkill and would not be required.
> RDMA should be kept only for IO operations.
>
> We should just remove it from the docs.
>
> Thanks,
> Raghavendra Talur
>
>
>>
>> Thanks,
>> --
>> Prasanna
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Need feedback on vagrant based Gluster cluster creator

[Raghavendra Talur]

Hi All,

Here is a patch for quick cluster creation of gluster.
http://review.gluster.org/#/c/14930/

I need feedback on the features/design/implementation etc.

I am currently planning to move these files to extras/devel-tools/ unless
there is any other suggestion.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] What tranpsort type in glfs_set_volfile_server() exactly mean ?

[Raghavendra Talur]

On Mon, Jul 18, 2016 at 4:10 PM, Prasanna Kalever <pkale...@redhat.com>
wrote:

> Hey Team,
>
>
> My understanding is that @transport argumet in
> glfs_set_volfile_server() is meant for specifying transport used in
> fetching volfile server,
>

Yes, @transport arg here is transport to use for fetching volfile.


> IIRC which currently supports tcp and unix only...
>
Yes, this is correct too.


>
> The doc here
> https://github.com/gluster/glusterfs/blob/master/api/src/glfs.h
> +166 shows the rdma as well, which is something I cannot digest.
>
This is doc written with assumption that rdma would work too.


>
>
> Can someone correct me ?
>
> Have we ever supported volfile fetch over rdma ?
>

I think no. To test, you would have to set rdma as only transport option in
glusterd.vol and see what happens in volfile fetch.


IMO, fetching volfile over rdma is an overkill and would not be required.
RDMA should be kept only for IO operations.

We should just remove it from the docs.

Thanks,
Raghavendra Talur


>
> Thanks,
> --
> Prasanna
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] tests/basic/op_errnos.t failure

[Raghavendra Talur]

Nigel/Misc,

Could you please look into this?
slave29 does not seem to have a xfs formatted backend for tests.

Thanks,
Raghavendra Talur

On Tue, Jul 12, 2016 at 6:41 PM, Avra Sengupta <aseng...@redhat.com> wrote:

> Atin,
>
> I am not sure about the docker containers, but both the failures you
> mentioned are in slave29, which as Talur explained is missing the
> appropriate backend filesystem. Owing to this, op-errno.t is just the tip
> of the iceberg, and every other test that uses lvm will fail in this
> particular slave will fail too.
>
> Talur,
> Thanks for looking into it. It is indeed strange this. I checked the dmesg
> and the /var/log/messages in this slave and I couldn't find any relevant
> log.
>
>
> On 07/12/2016 05:29 PM, Raghavendra Talur wrote:
>
> I checked the machine.
>
> Here is the df -hT output
> [jenkins@slave29 ~]$ cat /etc/fstab
> # Accessible filesystems, by reference, are maintained under '/dev/disk'
> # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
> #
> /dev/xvda1  /   ext3
>  defaults,noatime,barrier=0 1 1
> tmpfs   /dev/shmtmpfs   defaults0 0
> devpts  /dev/ptsdevpts  gid=5,mode=620  0 0
> sysfs   /syssysfs   defaults0 0
> proc/proc   procdefaults0 0
> #/dev/xvdc1 noneswapsw  0 0
>
>
> We don't see a xfs device mounted at /d and / is of type ext3 which does
> not support fallocate. The uptime of the machine is 73 days though. I don't
> know how the /d xfs partition vanished.
>
> On Tue, Jul 12, 2016 at 4:54 PM, Atin Mukherjee < <amukh...@redhat.com>
> amukh...@redhat.com> wrote:
>
>>
>> https://build.gluster.org/job/rackspace-regression-2GB-triggered/22156/consoleFull
>> - another failure
>>
>> On Tue, Jul 12, 2016 at 4:42 PM, Atin Mukherjee < <amukh...@redhat.com>
>> amukh...@redhat.com> wrote:
>>
>>>
>>>
>>> On Tue, Jul 12, 2016 at 4:36 PM, Avra Sengupta < <aseng...@redhat.com>
>>> aseng...@redhat.com> wrote:
>>>
>>>> Hi Atin,
>>>>
>>>> Please check the testcase result in the console. It clearly states the
>>>> reason of the failure. A quick search of 30815, as shown in the testcase
>>>> shows that the error that is generated is a thinp issue, and we can see
>>>> fallocate failing and lvm not properly being setup in the environment.
>>>>
>>>
>>> While this is valid for my docker containers, I am just wondering why
>>> did this happen in jenkins slave?
>>>
>>>
>>>> Regards,
>>>> Avra
>>>>
>>>> P.S Here are the logs from the console stating so.
>>>>
>>>> *02:50:34* [09:50:34] Running tests in file 
>>>> ./tests/basic/op_errnos.t*02:50:41* fallocate: 
>>>> /d/backends/patchy_snap_vhd: fallocate failed: Operation not 
>>>> supported*02:50:41* losetup: /d/backends/patchy_snap_vhd: warning: file 
>>>> smaller than 512 bytes, the loop device maybe be useless or invisible for 
>>>> system tools.*02:50:41*   Device /d/backends/patchy_snap_loop not found 
>>>> (or ignored by filtering).*02:50:41*   Device /d/backends/patchy_snap_loop 
>>>> not found (or ignored by filtering).*02:50:41*   Unable to add physical 
>>>> volume '/d/backends/patchy_snap_loop' to volume group 
>>>> 'patchy_snap_vg_1'.*02:50:41*   Volume group "patchy_snap_vg_1" not 
>>>> found*02:50:41*   Cannot process volume group patchy_snap_vg_1*02:50:42*   
>>>> Volume group "patchy_snap_vg_1" not found*02:50:42*   Cannot process 
>>>> volume group patchy_snap_vg_1*02:50:42* /dev/patchy_snap_vg_1/brick_lvm: 
>>>> No such file or directory*02:50:42* Usage: mkfs.xfs*02:50:42* /* blocksize 
>>>> */  [-b log=n|size=num]*02:50:42* /* data subvol */ [-d 
>>>> agcount=n,agsize=n,file,name=xxx,size=num,*02:50:42*   
>>>>  (sunit=value,swidth=value|su=num,sw=num),*02:50:42*   
>>>>   sectlog=n|sectsize=num*02:50:42* /* inode size */   [-i 
>>>> log=n|perblock=n|size=num,maxpct=n,attr=0|1|2,*02:50:42*   
>>>>  projid32bit=0|1]*02:50:42* /* log subvol */ [-l 
>>>> agnum=n,internal,size=num,logdev=xxx,version=n*02:50:42*   
>>>>  sunit=value|su=num,sectl

[Gluster-devel] Weekly Community Meeting - 25/May/2016

[Raghavendra Talur]

The meeting minutes for this weeks meeting are available at


Minutes:
https://meetbot.fedoraproject.org/gluster-meeting/2016-05-25/weekly_community_meeting_25may2016.2016-05-25-12.02.html
Minutes (text):
https://meetbot.fedoraproject.org/gluster-meeting/2016-05-25/weekly_community_meeting_25may2016.2016-05-25-12.02.txt
Log:
https://meetbot.fedoraproject.org/gluster-meeting/2016-05-25/weekly_community_meeting_25may2016.2016-05-25-12.02.log.html


Next weeks meeting will be held at the same time.

Thanks,
Raghavendra Talur


Meeting summary
---
* Rollcall  (rastar, 12:02:23)

* Next weeks meeting host  (rastar, 12:04:48)

* GlusterFS 4.0  (rastar, 12:09:09)

* GlusterFS 3.8  (rastar, 12:17:31)
  * LINK:

https://github.com/gluster/glusterfs/blob/release-3.8/doc/release-notes/3.8.0.md
(ndevos, 12:18:22)
  * HELP: need assistance from feature owners and maintainers to get the
release-notes in order  (ndevos, 12:18:54)
  * LINK:

https://bugzilla.redhat.com/showdependencytree.cgi?id=glusterfs-3.8.0_resolved=1
(ndevos, 12:22:13)

* GlusterFS 3.7  (rastar, 12:28:59)

* GlusterFS 3.6  (rastar, 12:31:21)
  * LINK: http://review.gluster.org/#/c/14007/
http://review.gluster.org/#/c/14403/
http://review.gluster.org/#/c/14418/  (vangelis, 12:34:51)
  * ACTION: rastar to looks at 3.6 builds failures on BSD  (rastar,
12:35:40)

* GlusterFS 3.5  (rastar, 12:41:48)

* NFS Ganesha and Gluster  (rastar, 12:44:19)

* Samba and Gluster  (rastar, 12:48:28)

* Saravanakmr to add documentation on how to add blogs  (rastar,
  12:51:56)
  * LINK:

http://gluster.readthedocs.io/en/latest/Contributors-Guide/Adding-your-blog/
(ndevos, 12:52:58)

* Open Floor  (rastar, 12:53:44)
  * LINK: http://review.gluster.org/#/c/14399/   (post-factum, 12:53:59)

Meeting ended at 13:04:24 UTC.




Action Items

* rastar to looks at 3.6 builds failures on BSD




Action Items, by person
---
* rastar
  * rastar to looks at 3.6 builds failures on BSD
* **UNASSIGNED**
  * (none)




People Present (lines said)
---
* rastar (90)
* ndevos (39)
* atinm (16)
* partner (14)
* jdarcy (11)
* kkeithley (8)
* post-factum (7)
* shyam (7)
* vangelis (5)
* zodbot (4)
* Saravanakmr (3)
* glusterbot (3)
* anoopcs (2)
* skoduri (2)
* jiffin (2)
* nigelb (1)
* karthik___ (1)
* msvbhat (1)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Weekly Community Meeting - 18/May/2016

[Raghavendra Talur]

The meeting minutes for this weeks meeting are available at

Minutes:
https://meetbot.fedoraproject.org/gluster-meeting/2016-05-18/weekly_community_meeting_18may2016.2016-05-18-12.06.html
Minutes (text):
https://meetbot-raw.fedoraproject.org/gluster-meeting/2016-05-18/weekly_community_meeting_18may2016.2016-05-18-12.06.txt
Log:
https://meetbot.fedoraproject.org/gluster-meeting/2016-05-18/weekly_community_meeting_18may2016.2016-05-18-12.06.log.html

Next weeks meeting will be held at the same time.

Thanks,
Raghavendra Talur




Meeting summary
---
* Rollcall  (rastar, 12:07:08)

* Next weeks meeting host  (rastar, 12:09:28)

* kshlm & csim to set up faux/pseudo user email for gerrit, bugzilla,
  github  (rastar, 12:10:55)
  * ACTION: kshlm/csim/nigelb to set up faux/pseudo user email for
gerrit, bugzilla, github  (rastar, 12:13:19)

* amye to check on some blog posts being distorted on blog.gluster.org,
  josferna's post in particular  (rastar, 12:13:34)
  * ACTION: aravinda/amye to check on some blog posts being distorted on
blog.gluster.org, josferna's post in particular  (rastar, 12:15:09)

* pranithk1 sends out a summary of release requirements, with some ideas
  (rastar, 12:15:27)
  * ACTION: hagarth to announce release strategy after getting inputs
from amye  (rastar, 12:21:50)

* kshlm to check with reported of 3.6 leaks on backport need  (rastar,
  12:22:37)
  * ACTION: kshlm to check with reported of 3.6 leaks on backport need
(rastar, 12:23:12)

* GlusterFS 3.7  (rastar, 12:23:38)

* GlusterFS 3.6  (rastar, 12:33:06)

* GlusterFS 3.5  (rastar, 12:37:50)

* GlusterFS 3.8  (rastar, 12:39:15)
  * LINK:
http://thread.gmane.org/gmane.comp.file-systems.gluster.maintainers/727
(ndevos, 12:40:11)
  * FYI (reminder) we don't (can't) ship in EPEL because RHS/RHGS
client-side pkgs are in RHEL.  We took a decision to not provide
el[567] 3.8 pkgs on download.gluster.org because they will be in the
CentOS Storage SIG  (kkeithley, 12:46:23)

* GlusterFS 4.0  (rastar, 12:49:04)

* NFS Ganesha and Gluster updates  (rastar, 12:52:53)

* Samba and Gluster  (rastar, 13:00:22)

* open floor  (rastar, 13:08:26)
  * HELP: packagers for lio/tcmu-runner with libgfapi wanted to get the
packages in the CentOS Storage SIG  (ndevos, 13:11:04)
  * IDEA: thought for the day: devs should occasionally do a build on a
32-bit system. It's a good way to catch log/printf format string
mistakes  (kkeithley, 13:11:41)
  * LINK: https://www.gluster.org/events/   (ndevos, 13:14:50)
  * ACTION: Saravanakmr to add documentation on how to add blogs
(rastar, 13:17:44)

Meeting ended at 13:25:03 UTC.




Action Items

* kshlm/csim/nigelb to set up faux/pseudo user email for gerrit,
  bugzilla, github
* aravinda/amye to check on some blog posts being distorted on
  blog.gluster.org, josferna's post in particular
* hagarth to announce release strategy after getting inputs from amye
* kshlm to check with reported of 3.6 leaks on backport need
* Saravanakmr to add documentation on how to add blogs




Action Items, by person
---
* amye
  * aravinda/amye to check on some blog posts being distorted on
blog.gluster.org, josferna's post in particular
  * hagarth to announce release strategy after getting inputs from amye
* hagarth
  * hagarth to announce release strategy after getting inputs from amye
* Saravanakmr
  * Saravanakmr to add documentation on how to add blogs
* **UNASSIGNED**
  * kshlm/csim/nigelb to set up faux/pseudo user email for gerrit,
bugzilla, github
  * kshlm to check with reported of 3.6 leaks on backport need




People Present (lines said)
---
* rastar (92)
* ndevos (65)
* kkeithley (38)
* hagarth (36)
* post-factum (35)
* ira (15)
* Saravanakmr (10)
* amye (6)
* jdarcy (5)
* obnox_ (3)
* zodbot (3)
* anoopcs (1)
* misc (1)
* atinm (1)
* obnox (1)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Suggest a feature redirect

[Raghavendra Talur]

On Thu, May 19, 2016 at 12:46 PM, Aravinda <avish...@redhat.com> wrote:

>
> regards
> Aravinda
>
> On 05/19/2016 12:37 PM, Raghavendra Talur wrote:
>
>
>
> On Thu, May 19, 2016 at 12:10 PM, Vijay Bellur <vbel...@redhat.com> wrote:
>
>> On 05/19/2016 02:38 AM, Amye Scavarda wrote:
>>
>>>
>>>
>>> On Thu, May 19, 2016 at 12:06 PM, Vijay Bellur < <vbel...@redhat.com>
>>> vbel...@redhat.com
>>> <mailto:vbel...@redhat.com>> wrote:
>>>
>>> On Thu, May 19, 2016 at 2:21 AM, Raghavendra Talur
>>> <rta...@redhat.com <mailto:rta...@redhat.com>> wrote:
>>> >
>>> >
>>> > On Wed, May 18, 2016 at 2:43 PM, Amye Scavarda <a...@redhat.com
>>> <mailto:a...@redhat.com>> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Wed, May 18, 2016 at 1:46 PM, Humble Devassy Chirammal
>>> >> <humble.deva...@gmail.com <mailto:humble.deva...@gmail.com>>
>>> wrote:
>>> >>>
>>> >>> Hi Amye,
>>> >>>
>>> >>> afaict, the feature proposal has start from glusterfs-specs
>>> >>> (http://review.gluster.org/#/q/project:glusterfs-specs) project
>>> under
>>> >>> review.gluster.org < <http://review.gluster.org>
>>> http://review.gluster.org>
>>> >>>
>>> >>> --Humble
>>> >>>
>>> >>>
>>> >>> On Wed, May 18, 2016 at 12:15 PM, Amye Scavarda <a...@redhat.com
>>> <mailto:a...@redhat.com>> wrote:
>>> >>>>
>>> >>>> Currently on the gluster.org < <http://gluster.org>
>>> http://gluster.org> website,
>>> we're directing the 'Suggest a
>>> >>>> feature' to the old mediawiki site for 'Features'.
>>> >>>>
>>> http://www.gluster.org/community/documentation/index.php/Features
>>> >>>>
>>> >>>> In an ideal world, this would go somewhere else. Where should
>>> this go?
>>> >>>> Let me know and I'll fix it.
>>> >>>> - amye
>>> >>>>
>>> >>>> --
>>> >>>> Amye Scavarda | <a...@redhat.com>a...@redhat.com >> <a...@redhat.com>a...@redhat.com> |
>>> Gluster Community Lead
>>> >>>>
>>> >>>> ___
>>> >>>> Gluster-devel mailing list
>>> >>>> Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
>>> >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>> >>>
>>> >>>
>>> >> Said another way, if you wanted to be able to have someone
>>> contribute a
>>> >> feature idea, what would be the best way?
>>> >> Bugzilla? A google form? An email into the -devel list?
>>> >>
>>> >
>>> > If it is more of a request from a user it should a bugzilla RFE.
>>> If the user
>>> > requested on -devel list without a bug, then one of the community
>>> devs
>>> > should file a bug for it.
>>> > If it is a proposal for a feature with design
>>> details/implementation ideas
>>> > then a patch to glusterfs-specs is encouraged.
>>>
>>>
>>> In any case opening a discussion on gluster-devel would be ideal to
>>> get the attention of all concerned.
>>>
>>> -Vijay
>>>
>>>
>>> So should this stay on the website in the first place?
>>> - amye
>>>
>>>
>> Yes, the relevant workflow needs to be in the website. In the absence of
>> that, newcomers to the community might find it difficult to understand our
>> workflow.
>
>
> Currently it is
> 
> 
> 
> Suggest a feature!
> 
> 
>
> which points to old wiki.
>
> Would it be better to have
> Suggest a feature
> there?
>
> Makes sense. Is it possible to send mail to this list without
> registration? or moderator can verify the feature mail from unsubscribed
> user and allow?
>

I did not think of this. I will let someone who knows how mailman works
reply.


>
> Thanks,
> Raghavendra Talur
>
>>
>>
>> -Vijay
>>
>>
>>
>>
>
>
> ___
> Gluster-devel mailing 
> listGluster-devel@gluster.orghttp://www.gluster.org/mailman/listinfo/gluster-devel
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Suggest a feature redirect

[Raghavendra Talur]

On Thu, May 19, 2016 at 12:10 PM, Vijay Bellur <vbel...@redhat.com> wrote:

> On 05/19/2016 02:38 AM, Amye Scavarda wrote:
>
>>
>>
>> On Thu, May 19, 2016 at 12:06 PM, Vijay Bellur <vbel...@redhat.com
>> <mailto:vbel...@redhat.com>> wrote:
>>
>> On Thu, May 19, 2016 at 2:21 AM, Raghavendra Talur
>> <rta...@redhat.com <mailto:rta...@redhat.com>> wrote:
>> >
>> >
>> > On Wed, May 18, 2016 at 2:43 PM, Amye Scavarda <a...@redhat.com
>> <mailto:a...@redhat.com>> wrote:
>> >>
>> >>
>> >>
>> >> On Wed, May 18, 2016 at 1:46 PM, Humble Devassy Chirammal
>> >> <humble.deva...@gmail.com <mailto:humble.deva...@gmail.com>>
>> wrote:
>> >>>
>> >>> Hi Amye,
>> >>>
>> >>> afaict, the feature proposal has start from glusterfs-specs
>> >>> (http://review.gluster.org/#/q/project:glusterfs-specs) project
>> under
>> >>> review.gluster.org <http://review.gluster.org>
>> >>>
>> >>> --Humble
>> >>>
>> >>>
>> >>> On Wed, May 18, 2016 at 12:15 PM, Amye Scavarda <a...@redhat.com
>> <mailto:a...@redhat.com>> wrote:
>> >>>>
>> >>>> Currently on the gluster.org <http://gluster.org> website,
>> we're directing the 'Suggest a
>> >>>> feature' to the old mediawiki site for 'Features'.
>> >>>>
>> http://www.gluster.org/community/documentation/index.php/Features
>> >>>>
>> >>>> In an ideal world, this would go somewhere else. Where should
>> this go?
>> >>>> Let me know and I'll fix it.
>> >>>> - amye
>> >>>>
>> >>>> --
>> >>>> Amye Scavarda | a...@redhat.com <mailto:a...@redhat.com> |
>> Gluster Community Lead
>> >>>>
>> >>>> ___
>> >>>> Gluster-devel mailing list
>> >>>> Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
>> >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>> >>>
>> >>>
>> >> Said another way, if you wanted to be able to have someone
>> contribute a
>> >> feature idea, what would be the best way?
>> >> Bugzilla? A google form? An email into the -devel list?
>> >>
>> >
>> > If it is more of a request from a user it should a bugzilla RFE.
>> If the user
>> > requested on -devel list without a bug, then one of the community
>> devs
>> > should file a bug for it.
>> > If it is a proposal for a feature with design
>> details/implementation ideas
>> > then a patch to glusterfs-specs is encouraged.
>>
>>
>> In any case opening a discussion on gluster-devel would be ideal to
>> get the attention of all concerned.
>>
>> -Vijay
>>
>>
>> So should this stay on the website in the first place?
>> - amye
>>
>>
> Yes, the relevant workflow needs to be in the website. In the absence of
> that, newcomers to the community might find it difficult to understand our
> workflow.


Currently it is



Suggest a feature!



which points to old wiki.

Would it be better to have
Suggest a feature
there?

Thanks,
Raghavendra Talur

>
>
> -Vijay
>
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Suggest a feature redirect

[Raghavendra Talur]

On Wed, May 18, 2016 at 2:43 PM, Amye Scavarda  wrote:

>
>
> On Wed, May 18, 2016 at 1:46 PM, Humble Devassy Chirammal <
> humble.deva...@gmail.com> wrote:
>
>> Hi Amye,
>>
>> afaict, the feature proposal has start from glusterfs-specs (
>> http://review.gluster.org/#/q/project:glusterfs-specs) project under
>> review.gluster.org
>>
>> --Humble
>>
>>
>> On Wed, May 18, 2016 at 12:15 PM, Amye Scavarda  wrote:
>>
>>> Currently on the gluster.org website, we're directing the 'Suggest a
>>> feature' to the old mediawiki site for 'Features'.
>>> http://www.gluster.org/community/documentation/index.php/Features
>>>
>>> In an ideal world, this would go somewhere else. Where should this go?
>>> Let me know and I'll fix it.
>>> - amye
>>>
>>> --
>>> Amye Scavarda | a...@redhat.com | Gluster Community Lead
>>>
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>> Said another way, if you wanted to be able to have someone contribute a
> feature idea, what would be the best way?
> Bugzilla? A google form? An email into the -devel list?
>
>
If it is more of a request from a user it should a bugzilla RFE. If the
user requested on -devel list without a bug, then one of the community devs
should file a bug for it.
If it is a proposal for a feature with design details/implementation ideas
then a patch to glusterfs-specs is encouraged.


> - amye
>
> --
> Amye Scavarda | a...@redhat.com | Gluster Community Lead
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Idea: Alternate Release process

[Raghavendra Talur]

On Thu, May 19, 2016 at 11:39 AM, Kaushal M <kshlms...@gmail.com> wrote:

> On Thu, May 19, 2016 at 11:35 AM, Kaushal M <kshlms...@gmail.com> wrote:
> > On Thu, May 19, 2016 at 11:29 AM, Raghavendra Talur <rta...@redhat.com>
> wrote:
> >>
> >>
> >> On Thu, May 19, 2016 at 11:13 AM, Kaushal M <kshlms...@gmail.com>
> wrote:
> >>>
> >>> I'm in favour of a stable release every 2 months and an LTS once a
> >>> year (option 2).
> >>
> >>
> >> +1
> >>
> >>>
> >>>
> >>> As Oleksander already suggested, I'm in favour of having well defined
> >>> merge windows, freeze dates and testing period.
> >>> (A slightly modified timeline from Oleksander's proposal follows)
> >>> For every 2 month window,
> >>> - 1 month of development (merge window). New features will not be
> >>> accepted post this period.
> >>> - At the end of the development period a release-candidate 1 is tagged.
> >>> - A 1 month testing/bug-fixing window follows. This time is for
> >>> testing and fixing bugs found.
> >>>  Feature development and reviews can happen during this period on
> >>> gerrit, but nothing gets merged. Merges will be held till the next
> >>> merge window.
> >>
> >>
> >> This means the branching is not done at the end of merge window and the
> >> patches will not be merged even on the master branch. Is that right?
> >
> > There will be no branching at all (except for the LTS release). We
> > have just 2 branches,
> > one LTS and one stable.
> > All features (even those partially developed) can get merged in during
> > the merge window.
> > During the stability window, the partially developed and unstable
> > features get disabled.
> > This requires us to properly implement a feature flags framework,
> > that allows features to be selectively compiled.
>
> Just to be more complete, if any urgent fixes are required for any
> released version,
> an emergency branch will be created from the tag for the release,
> which will only
> contain the required fixes. These fixes will have to land on the
> stable branch before
> being backported to a emergency branch.
>

I would prefer if this was relaxed a bit. Rather than placing a requirement
of a bug fix in a released branch to be backported it to LTS, I would
prefer having it in *next* release is ok(i.e merging in stable branch).


>
> >
> >>
> >>> - At least 2 more release-candidates will be release once every
> fortnight
> >>> - The final release-candidate becomes the release if it passes all our
> >>> tests.
> >>>
> >>> One of the 6 releases of the year will become a LTS release.
> >>> The 2 month window for this release will mainly be targeted at making
> >>> the release stable.
> >>> New features should be minimal for this release.
> >>
> >>
> >> I really like this point. Taking 3.8 as a LTS release this would mean
> new
> >> features for 3.14 release would not be encouraged.
> >>
> >>>
> >>>
> >>> During every 2 month window, the LTS release will get any required bug
> >>> fixes and stability improvements backported.
> >>> For fix to be backported it needs to be present in a stable release.
> >>
> >>
> >> +1
> >>
> >>>
> >>>
> >>> ~kaushal
> >>>
> >>> On Wed, May 18, 2016 at 11:46 PM, Atin Mukherjee <amukh...@redhat.com>
> >>> wrote:
> >>> > A bit late but better than never. My vote is for option 2.
> >>> >
> >>> > ~Atin
> >>> >
> >>> > On 05/18/2016 07:19 PM, Vijay Bellur wrote:
> >>> >> [Adding gluster-users]
> >>> >>
> >>> >> I would like to wrap this poll by the next community meeting on 25th
> >>> >> May. Can you please weigh in with your opinions on the options
> >>> >> provided by Aravinda?
> >>> >>
> >>> >> Thanks!
> >>> >> Vijay
> >>> >>
> >>> >>
> >>> >> On Fri, May 13, 2016 at 4:16 AM, Aravinda <avish...@redhat.com>
> wrote:
> >>> >>> Hi,
> >>> >>>
> >>> >>> Based on the discussion in last community meeting and previous
> >>> >>> discussions,
> >>>

Re: [Gluster-devel] [Gluster-users] Idea: Alternate Release process

[Raghavendra Talur]

On Thu, May 19, 2016 at 11:13 AM, Kaushal M  wrote:

> I'm in favour of a stable release every 2 months and an LTS once a
> year (option 2).
>

+1


>
> As Oleksander already suggested, I'm in favour of having well defined
> merge windows, freeze dates and testing period.
> (A slightly modified timeline from Oleksander's proposal follows)
> For every 2 month window,
> - 1 month of development (merge window). New features will not be
> accepted post this period.
> - At the end of the development period a release-candidate 1 is tagged.
> - A 1 month testing/bug-fixing window follows. This time is for
> testing and fixing bugs found.
>  Feature development and reviews can happen during this period on
> gerrit, but nothing gets merged. Merges will be held till the next
> merge window.
>

This means the branching is not done at the end of merge window and the
patches will not be merged even on the master branch. Is that right?

- At least 2 more release-candidates will be release once every fortnight
> - The final release-candidate becomes the release if it passes all our
> tests.
>
> One of the 6 releases of the year will become a LTS release.
> The 2 month window for this release will mainly be targeted at making
> the release stable.
> New features should be minimal for this release.
>

I really like this point. Taking 3.8 as a LTS release this would mean new
features for 3.14 release would not be encouraged.


>
> During every 2 month window, the LTS release will get any required bug
> fixes and stability improvements backported.
> For fix to be backported it needs to be present in a stable release.
>

+1


>
> ~kaushal
>
> On Wed, May 18, 2016 at 11:46 PM, Atin Mukherjee 
> wrote:
> > A bit late but better than never. My vote is for option 2.
> >
> > ~Atin
> >
> > On 05/18/2016 07:19 PM, Vijay Bellur wrote:
> >> [Adding gluster-users]
> >>
> >> I would like to wrap this poll by the next community meeting on 25th
> >> May. Can you please weigh in with your opinions on the options
> >> provided by Aravinda?
> >>
> >> Thanks!
> >> Vijay
> >>
> >>
> >> On Fri, May 13, 2016 at 4:16 AM, Aravinda  wrote:
> >>> Hi,
> >>>
> >>> Based on the discussion in last community meeting and previous
> discussions,
> >>>
> >>> 1. Too frequent releases are difficult to manage.(without dedicated
> release
> >>> manager)
> >>> 2. Users wants to see features early for testing or POC.
> >>> 3. Backporting patches to more than two release branches is pain
> >>>
> >>> Enclosed visualizations to understand existing release and support
> cycle and
> >>> proposed alternatives.
> >>>
> >>> - Each grid interval is 6 months
> >>> - Green rectangle shows supported release or LTS
> >>> - Black dots are minor releases till it is supported(once a month)
> >>> - Orange rectangle is non LTS release with minor releases(Support ends
> when
> >>> next version released)
> >>>
> >>> Enclosed following images
> >>> 1. Existing Release cycle and support plan(6 months release cycle, 3
> >>> releases supported all the time)
> >>> 2. Proposed alternative 1 - One LTS every year and non LTS stable
> release
> >>> once in every 2 months
> >>> 3. Proposed alternative 2 - One LTS every year and non LTS stable
> release
> >>> once in every 3 months
> >>> 4. Proposed alternative 3 - One LTS every year and non LTS stable
> release
> >>> once in every 4 months
> >>> 5. Proposed alternative 4 - One LTS every year and non LTS stable
> release
> >>> once in every 6 months (Similar to existing but only alternate one will
> >>> become LTS)
> >>>
> >>> Please do vote for the proposed alternatives about release intervals
> and LTS
> >>> releases. You can also vote for the existing plan.
> >>>
> >>> Do let me know if I missed anything.
> >>>
> >>> regards
> >>> Aravinda
> >>>
> >>> On 05/11/2016 12:01 AM, Aravinda wrote:
> >>>
> >>> I couldn't find any solution for the backward incompatible changes. As
> you
> >>> mentioned this model will not work for LTS.
> >>>
> >>> How about adopting this only for non LTS releases? We will not have
> backward
> >>> incompatibility problem since we need not release minor updates to non
> LTS
> >>> releases.
> >>>
> >>> regards
> >>> Aravinda
> >>>
> >>> On 05/05/2016 04:46 PM, Aravinda wrote:
> >>>
> >>>
> >>> regards
> >>> Aravinda
> >>>
> >>> On 05/05/2016 03:54 PM, Kaushal M wrote:
> >>>
> >>> On Thu, May 5, 2016 at 11:48 AM, Aravinda  wrote:
> >>>
> >>> Hi,
> >>>
> >>> Sharing an idea to manage multiple releases without maintaining
> >>> multiple release branches and backports.
> >>>
> >>> This idea is heavily inspired by the Rust release model(you may feel
> >>> exactly same except the LTS part). I think Chrome/Firefox also follows
> >>> the same model.
> >>>
> >>> http://blog.rust-lang.org/2014/10/30/Stability.html
> >>>
> >>> Feature Flag:
> >>> --
> >>> Compile time variable to prevent compiling featurerelated code when
> >>> 

Re: [Gluster-devel] Reducing the size of the glusterdocs git repository

[Raghavendra Talur]

On Thu, May 19, 2016 at 9:46 AM, Aravinda  wrote:

> +1 for Prashanth's approach.
>

+1

> regards
> Aravinda
>
> On 05/18/2016 03:55 PM, Kaushal M wrote:
>
> On Wed, May 18, 2016 at 3:29 PM, Humble Devassy 
> Chirammal  wrote:
>
>
>
> On Wed, May 18, 2016 at 3:26 PM, Amye Scavarda  
>  wrote:
>
>  All in favor of the 'everyone working on this should clone again'
> approach.
>
> Both approaches require cloning again, so which on are we voting for?
>
> 1. Prashanth's approach
> - modify existing repo
> - users re clone
> - push force to their forks on github
>
> 2. Misc's approach
>   - create a new minified repo
>   - users clone/fork the new repo
>
> I don't mind either.
>
>
> +2 on the same thought. :)
>
>  ___
> Gluster-devel mailing 
> listGluster-devel@gluster.orghttp://www.gluster.org/mailman/listinfo/gluster-devel
>
> ___
> Gluster-devel mailing 
> listGluster-devel@gluster.orghttp://www.gluster.org/mailman/listinfo/gluster-devel
>
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] tests/performance/open-behind.t fails on NetBSD

[Raghavendra Talur]

On Sun, May 8, 2016 at 10:33 PM, Emmanuel Dreyfus <m...@netbsd.org> wrote:

> Joseph Fernandes <josfe...@redhat.com> wrote:
>
> > ./tests/performance/open-behind.t is failing continuously on 3.7.11
>
> This is the fate of non enforced tests. It may be a good idea to
> invetigate it: perhaps NetBSD gets a reliable failure for a rare bug
> that is not NetBSD specific. We already saw such situations.
>

Emmanuel is right. It might be a valid failure that is only reproducible on
NetBSD.
I found that a bug was already filed on master. I have cloned it for 3.7.

Bug:https://bugzilla.redhat.com/show_bug.cgi?id=1334204
Patch to mark test as bad in 3.7: http://review.gluster.org/14256

Thanks,
Raghavendra Talur




>
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Show and Tell sessions for Gluster 4.0

[Raghavendra Talur]

On Mon, May 9, 2016 at 11:55 AM, Atin Mukherjee  wrote:

> In the view of keeping the visibility of the work completed vs work in
> progress and getting some fruitful feedback from the community we are
> thinking of having a hangout/bluejeans session for 30-45 minutes once in
> every month. The sessions will be concentrating on discussing the work
> done over the last month and demoing few pieces of Gluster 4.0 projects
> and what's the expectation for the coming month(s).
>
> There are couple of options we have w.r.t scheduling these sessions.
>
> 1. We use the weekly community meeting slot once in a month
> or 2. Have a dedicated separate slot (probably @ 12:00 UTC, last
> Thursday of each month)
>

I would prefer dedicated time slot.


>
> I'd request you to vote for any of these two and based on that we can
> move forward.
>
> Thanks,
> Atin
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] xlator option setting in gfapi

[Raghavendra Talur]

On Mon, May 9, 2016 at 8:45 AM, Raghavendra Gowdappa <rgowd...@redhat.com>
wrote:

> Hi Poornima/Raghavendra,
>
> This mail is an initiation of a thread to discuss how to make xlator
> options setting in gfapi synchronous (so, that caller will know the result)
> or providing a cbk to let the application know about the status.
>
> My very naive attempt of code-reading showed me that
> pub_glfs_set_xlator_option just adds the option to cmd_args.xlator_options.
> Can you please explain how/when these values are communicated to glusterd
> to change volfile? From there we can carry forward the conversation.
>
>
Raghavendra,

glfs_set_xlator_option is equivalent to --xlator-option of mount.glusterfs
of FUSE. This feature is not intended to apply the setting to the volume
permanently, rather it is specific to the mount and only valid for its
lifetime.
The current architecture of Gluster graph takes these options only in
cmd_args and then these values are given preference over whatever comes in
volfile from Glusterd. In essence, these settings can be used to override
the volume settings for a particular mount.

If the requirement is to set a volume wide option then glusterd or one of
the REST interfaces for Glusterd should be used.

I will also reply to the other thread with possible methods on overcoming
the problem with write-behind and qemu.

Thanks,
Raghavendra Talur


> regards,
> Raghavendra
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] State of 3.7 regression

[Raghavendra Talur]

Out of the three bugs, one is because of a mistake in backport that I did.
I will mark the other two as bad tests in 3.7 branch and re-trigger my
patch so that we can merge it.


On Tue, May 3, 2016 at 7:08 PM, Atin Mukherjee <atin.mukherje...@gmail.com>
wrote:

> Strange, there is no delta in tests/bugs/glusterd/bug-1314649-group-virt.t
> between master and 3.7 and I ran it locally and it passed.
>
> Gaurav,
>
> Mind to grep through the logs and find out the cause of the failure?
>
>
This is my mistake, I did a mistake in conflict resolution which deleted
the virt group volume set file.


> Thanks,
> Atin
>
> On Tue, May 3, 2016 at 10:54 AM, Raghavendra Talur <rta...@redhat.com>
> wrote:
>
>> Hi All,
>>
>> I had sent a patch to backport all the test infra and test fixes to 3.7
>> branch from master.[1]
>> Here are the details for test failures
>>
>> 1. Netbsd
>>
>> 01:32:03 1 test(s) failed
>> 01:32:03 ./tests/basic/afr/durability-off.t
>> 01:32:03
>> 01:32:03 1 test(s) generated core
>> 01:32:03 ./tests/bitrot/br-state-check.t
>>
>> More details can be found here[2]
>>
>>
>> 2. Centos Regression
>>
>> 05:06:09
>> 05:06:09 1 test(s) failed
>> 05:06:09 ./tests/bugs/glusterd/bug-1314649-group-virt.t
>> 05:06:09
>>
>> More details can be found here [3]
>>
>> I request the maintainers to look at the failures and fix them so that we
>> can successfully backport test fixes.
>>
>>
>> Thanks,
>> Raghavendra Talur
>>
>>
>>
>>
>>
>> [1]: http://review.gluster.org/#/c/13683/
>> [2]:
>> https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16489/console
>> [3]:
>> https://build.gluster.org/job/rackspace-regression-2GB-triggered/20348/console
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] State of 3.7 regression

[Raghavendra Talur]

Hi All,

I had sent a patch to backport all the test infra and test fixes to 3.7
branch from master.[1]
Here are the details for test failures

1. Netbsd

01:32:03 1 test(s) failed
01:32:03 ./tests/basic/afr/durability-off.t
01:32:03
01:32:03 1 test(s) generated core
01:32:03 ./tests/bitrot/br-state-check.t

More details can be found here[2]


2. Centos Regression

05:06:09
05:06:09 1 test(s) failed
05:06:09 ./tests/bugs/glusterd/bug-1314649-group-virt.t
05:06:09

More details can be found here [3]

I request the maintainers to look at the failures and fix them so that we
can successfully backport test fixes.


Thanks,
Raghavendra Talur





[1]: http://review.gluster.org/#/c/13683/
[2]:
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/16489/console
[3]:
https://build.gluster.org/job/rackspace-regression-2GB-triggered/20348/console
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Weekly Gluster Community meeting 20-Apr-2016

[Raghavendra Talur]

The meeting logs for this weeks meeting are available at the following links


   - Minutes: 
https://meetbot.fedoraproject.org/gluster-meeting/2016-04-20/gluster_community_meeting_20-apr-2016.2016-04-20-12.01.html
   - Minutes (text):
https://meetbot.fedoraproject.org/gluster-meeting/2016-04-20/gluster_community_meeting_20-apr-2016.2016-04-20-12.01.txt
   - Log: 
https://meetbot.fedoraproject.org/gluster-meeting/2016-04-20/gluster_community_meeting_20-apr-2016.2016-04-20-12.01.log.html

Next week's meeting will be held at the same time.

Thanks,

Raghavendra Talur



Meeting summary
---
* Roll call  (rastar, 12:02:16)

* Next weeks meeting host?  (rastar, 12:06:46)
  * rastar to host next meeting  (rastar, 12:08:12)

* Last weeks AIs  (rastar, 12:08:21)

* kshlm & csim to set up faux/pseudo user email for gerrit, bugzilla,
  github  (rastar, 12:08:45)
  * ACTION: kshlm & csim to set up faux/pseudo user email for gerrit,
bugzilla, github  (rastar, 12:09:36)

* jdarcy to provide a genral Gluster-4.0 status update  (rastar,
  12:09:54)
  * ACTION: jdarcy to provide a genral Gluster-4.0 status update
(rastar, 12:11:25)

* hagarth to take forward discussion on release and support strategies
  (onto mailing lists or another IRC meeting)  (rastar, 12:11:57)
  * pranith sent a mail to maintain...@gluster.org regarding the same
(rastar, 12:13:16)
  * post-factum says hagarth talked about some voting among devs
(rastar, 12:13:49)
  * ACTION: hagarth to take forward discussion on release and support
strategies (onto mailing lists or another IRC meeting)  (rastar,
12:14:09)

* jdarcy will get more technical and community leads to participate in
  the weekly meeting  (rastar, 12:14:30)
  * ACTION: jdarcy will get more technical and community leads to
participate in the weekly meeting  (rastar, 12:15:59)

* GlusterFS-3.7  (rastar, 12:16:38)
  * kshlm announced 3.7.11  (rastar, 12:17:51)
  * LINK:
https://www.mail-archive.com/gluster-users@gluster.org/msg24508.html
(rastar, 12:17:59)
  * LINK: http://termbin.com/13oa   (post-factum, 12:19:23)

* GlusterFS 4.0  (rastar, 12:24:50)
  * dht2 updated design docs will be up for review some time next week
(rastar, 12:29:33)

* GlusterFS 3.8  (rastar, 12:33:10)
  * Feature Freeze is April 30 2016  (rastar, 12:33:30)
  * msvbhat has sent mails on how to use distaf  (rastar, 12:35:01)

*   (rastar, 12:37:51)

* Open Floor  (rastar, 12:37:56)
  * ACTION: amye to check on some blog posts being distored on
blog.gluster.org  (rastar, 12:46:49)

Meeting ended at 12:58:16 UTC.




Action Items

* kshlm & csim to set up faux/pseudo user email for gerrit, bugzilla,
  github
* jdarcy to provide a genral Gluster-4.0 status update
* hagarth to take forward discussion on release and support strategies
  (onto mailing lists or another IRC meeting)
* jdarcy will get more technical and community leads to participate in
  the weekly meeting
* amye to check on some blog posts being distored on blog.gluster.org




Action Items, by person
---
* amye
  * amye to check on some blog posts being distored on blog.gluster.org
* hagarth
  * hagarth to take forward discussion on release and support strategies
(onto mailing lists or another IRC meeting)
* **UNASSIGNED**
  * kshlm & csim to set up faux/pseudo user email for gerrit, bugzilla,
github
  * jdarcy to provide a genral Gluster-4.0 status update
  * jdarcy will get more technical and community leads to participate in
the weekly meeting




People Present (lines said)
---
* rastar (74)
* hagarth (19)
* msvbhat (17)
* amye (11)
* overclk (10)
* Saravanakmr (10)
* post-factum (8)
* zodbot (4)
* anoopcs (3)
* glusterbot (2)
* aravindavk (1)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Which test generates the core?

[Raghavendra Talur]

On Thu, Apr 14, 2016 at 10:13 AM, Atin Mukherjee 
wrote:

> I am pretty sure that earlier we used to log the culprit test generating
> the core. But now I don't see that same behavior in these runs [1] [2]
>
> [1]
>
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/19709/consoleFull
> [2]
>
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/19720/consoleFull
>
>
These runs are from 3.7 release branch and not all the test framework
improvements have been backported there.
I have a patch [1] which backports every change under tests/ dir but it is
failing because some of the required code changes have not been backported.

For this particular crash that you are observing, the test which generates
the core is ./tests/basic/tier/tier-file-create.t. How to determine? Well,
based on core name which has PID and then look into messages or logs to see
what was happening during that time.

The cause of the crash has been documented well in the bug [2] and a
partial fix [3] by Pranith is available.

Pranith,
With respect to dict_foreach problem you mention, won't taking a lock in
dict_foreach solve the problem?



[1] http://review.gluster.org/#/c/13683/
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1315560
[3] http://review.gluster.org/#/c/13680/2


~Atin
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

[Raghavendra Talur]

On Wed, Mar 16, 2016 at 11:39 AM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

>
>
> On 03/16/2016 11:31 AM, Raghavendra Talur wrote:
>
> Hi,
>
> Lot many fixes to tests were found to be not back ported to 3.7 and other
> release branches.
> This causes tests to fail only in those branches and leaves the
> maintainers puzzled.
>
> Also, this seems to be the case with back porting code fixes too.
>
> I copied all the changes to tests/ dir on master to tests/ dir on 3.7
> branch and posted a patch at http://review.gluster.org/#/c/13683/ .
>
> This is failing for ./tests/bugs/distribute/bug-860663.t test :
> [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
> tar: Removing leading `/' from member names
> ./tests/bugs/distribute/bug-860663.t ..
> 1..15
> ok 1, LINENUM:23
> ok 2, LINENUM:24
> ok 3, LINENUM:26
> ok 4, LINENUM:27
> ok 5, LINENUM:30
> ok 6, LINENUM:32
> ok 7, LINENUM:35
> not ok 8 , LINENUM:40
> FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance patchy
> fix-layout start
> ok 9, LINENUM:42
> ok 10, LINENUM:43
> ok 11, LINENUM:45
> ok 12, LINENUM:47
> ok 13, LINENUM:50
> ok 14, LINENUM:51
> ok 15, LINENUM:55
> Failed 1/15 subtests
>
> Given that it is a simple rebalance command that is failing I am assuming
> that a critical patch has not been back ported to 3.7, correct me if I am
> wrong.
>
> I request every developer to take responsibility of back porting patches.
>
>
> Corollary question: Our test-framework is now capable of disabling tests
> for certain OS, certain branch etc. I would like to propose that we stop
> having tests in main git repo. This will remove need to back port test only
> fixes.
>
>
> Some times what I do is to enhance existing test to handle extra cases
> based on new code that is added on master. Until the code-fix is not
> backported to lower versions, the tests are not valid. Should we mark such
> tests disabled when we do enhancements to .t files?
>

Yes, this case would pose a problem and hence any new code change which
introduces a feature should be tested by a new test(not by modifying
existing test) and bug fix should be immediately backported along with test
fix.


>
> Pranith
>
>
>
> Thanks,
> Raghavendra Talur
>
>
>
> ___
> Gluster-devel mailing 
> listGluster-devel@gluster.orghttp://www.gluster.org/mailman/listinfo/gluster-devel
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

[Raghavendra Talur]

On Wed, Mar 16, 2016 at 11:59 PM, Atin Mukherjee <atin.mukherje...@gmail.com
> wrote:

> -Atin
> Sent from one plus one
>
> On 16-Mar-2016 11:32 am, "Raghavendra Talur" <rta...@redhat.com> wrote:
> >
> > Hi,
> >
> > Lot many fixes to tests were found to be not back ported to 3.7 and
> other release branches.
> > This causes tests to fail only in those branches and leaves the
> maintainers puzzled.
> >
> > Also, this seems to be the case with back porting code fixes too.
> >
> > I copied all the changes to tests/ dir on master to tests/ dir on 3.7
> branch and posted a patch at http://review.gluster.org/#/c/13683/ .
> >
> > This is failing for ./tests/bugs/distribute/bug-860663.t test :
> > [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
> > tar: Removing leading `/' from member names
> > ./tests/bugs/distribute/bug-860663.t ..
> > 1..15
> > ok 1, LINENUM:23
> > ok 2, LINENUM:24
> > ok 3, LINENUM:26
> > ok 4, LINENUM:27
> > ok 5, LINENUM:30
> > ok 6, LINENUM:32
> > ok 7, LINENUM:35
> > not ok 8 , LINENUM:40
> > FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance
> patchy fix-layout start
> > ok 9, LINENUM:42
> > ok 10, LINENUM:43
> > ok 11, LINENUM:45
> > ok 12, LINENUM:47
> > ok 13, LINENUM:50
> > ok 14, LINENUM:51
> > ok 15, LINENUM:55
> > Failed 1/15 subtests
> >
> > Given that it is a simple rebalance command that is failing I am
> assuming that a critical patch has not been back ported to 3.7, correct me
> if I am wrong.
> >
> > I request every developer to take responsibility of back porting patches.
> >
> >
> > Corollary question: Our test-framework is now capable of disabling tests
> for certain OS, certain branch etc. I would like to propose that we stop
> having tests in main git repo. This will remove need to back port test only
> fixes.
> Where are you going to host these .t files then?


Proposal was a separate git repo for tests. It seems very unlikely to
maintain it separately though.
Basically what I am looking for is making tests/ dir independent of git
branch but still part of the git repo.


>
> >
> >
> > Thanks,
> > Raghavendra Talur
> >
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

[Raghavendra Talur]

On Mar 17, 2016 7:50 AM, "Pranith Kumar Karampuri" <pkara...@redhat.com>
wrote:
>
>
>
> On 03/16/2016 11:46 PM, Raghavendra Talur wrote:
>>
>>
>>
>> On Wed, Mar 16, 2016 at 11:39 AM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:
>>>
>>>
>>>
>>> On 03/16/2016 11:31 AM, Raghavendra Talur wrote:
>>>>
>>>> Hi,
>>>>
>>>> Lot many fixes to tests were found to be not back ported to 3.7 and
other release branches.
>>>> This causes tests to fail only in those branches and leaves the
maintainers puzzled.
>>>>
>>>> Also, this seems to be the case with back porting code fixes too.
>>>>
>>>> I copied all the changes to tests/ dir on master to tests/ dir on 3.7
branch and posted a patch at http://review.gluster.org/#/c/13683/ .
>>>>
>>>> This is failing for ./tests/bugs/distribute/bug-860663.t test :
>>>> [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
>>>> tar: Removing leading `/' from member names
>>>> ./tests/bugs/distribute/bug-860663.t ..
>>>> 1..15
>>>> ok 1, LINENUM:23
>>>> ok 2, LINENUM:24
>>>> ok 3, LINENUM:26
>>>> ok 4, LINENUM:27
>>>> ok 5, LINENUM:30
>>>> ok 6, LINENUM:32
>>>> ok 7, LINENUM:35
>>>> not ok 8 , LINENUM:40
>>>> FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance
patchy fix-layout start
>>>> ok 9, LINENUM:42
>>>> ok 10, LINENUM:43
>>>> ok 11, LINENUM:45
>>>> ok 12, LINENUM:47
>>>> ok 13, LINENUM:50
>>>> ok 14, LINENUM:51
>>>> ok 15, LINENUM:55
>>>> Failed 1/15 subtests
>>>>
>>>> Given that it is a simple rebalance command that is failing I am
assuming that a critical patch has not been back ported to 3.7, correct me
if I am wrong.
>>>>
>>>> I request every developer to take responsibility of back porting
patches.
>>>>
>>>>
>>>> Corollary question: Our test-framework is now capable of disabling
tests for certain OS, certain branch etc. I would like to propose that we
stop having tests in main git repo. This will remove need to back port test
only fixes.
>>>
>>>
>>> Some times what I do is to enhance existing test to handle extra cases
based on new code that is added on master. Until the code-fix is not
backported to lower versions, the tests are not valid. Should we mark such
tests disabled when we do enhancements to .t files?
>>
>>
>> Yes, this case would pose a problem and hence any new code change which
introduces a feature should be tested by a new test(not by modifying
existing test) and bug fix should be immediately backported along with test
fix.
>
>
> As Atin mentioned in another mail, it is better to have code + test files
which test code together.

Makes sense,  so proactive back porting of fixes is what we require.

>
> Pranith
>
>>
>>>
>>>
>>> Pranith
>>>>
>>>>
>>>>
>>>> Thanks,
>>>> Raghavendra Talur
>>>>
>>>>
>>>>
>>>> ___
>>>> Gluster-devel mailing list
>>>> Gluster-devel@gluster.org
>>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>>
>>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

[Raghavendra Talur]

On Wed, Mar 16, 2016 at 11:31 AM, Raghavendra Talur <rta...@redhat.com>
wrote:

> Hi,
>
> Lot many fixes to tests were found to be not back ported to 3.7 and other
> release branches.
> This causes tests to fail only in those branches and leaves the
> maintainers puzzled.
>
> Also, this seems to be the case with back porting code fixes too.
>
> I copied all the changes to tests/ dir on master to tests/ dir on 3.7
> branch and posted a patch at http://review.gluster.org/#/c/13683/ .
>
> This is failing for ./tests/bugs/distribute/bug-860663.t test :
> [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
> tar: Removing leading `/' from member names
> ./tests/bugs/distribute/bug-860663.t ..
> 1..15
> ok 1, LINENUM:23
> ok 2, LINENUM:24
> ok 3, LINENUM:26
> ok 4, LINENUM:27
> ok 5, LINENUM:30
> ok 6, LINENUM:32
> ok 7, LINENUM:35
> not ok 8 , LINENUM:40
> FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance patchy
> fix-layout start
> ok 9, LINENUM:42
> ok 10, LINENUM:43
> ok 11, LINENUM:45
> ok 12, LINENUM:47
> ok 13, LINENUM:50
> ok 14, LINENUM:51
> ok 15, LINENUM:55
> Failed 1/15 subtests
>
> Given that it is a simple rebalance command that is failing I am assuming
> that a critical patch has not been back ported to 3.7, correct me if I am
> wrong.
>

In this particular case, the fix has already been posted on 3.7 branch and
just not merged yet. Link http://review.gluster.org/#/c/13537/3
Thanks Sakshi .


>
> I request every developer to take responsibility of back porting patches.
>
>
> Corollary question: Our test-framework is now capable of disabling tests
> for certain OS, certain branch etc. I would like to propose that we stop
> having tests in main git repo. This will remove need to back port test only
> fixes.
>
>
> Thanks,
> Raghavendra Talur
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gerrit trigger connection to jenkins not working

[Raghavendra Talur]

Root cause: I see the following error when I do "test connection"
Connection error : com.jcraft.jsch.JSchException:
java.net.NoRouteToHostException: No route to host


On Wed, Mar 16, 2016 at 10:57 AM, Raghavendra Talur <rta...@redhat.com>
wrote:

> Hi All,
>
> I see that jenkins isn't getting any triggers from gerrit. This includes
> patch set update trigger, so basically no patch is automatically tested. I
> am looking into it. Will update in this thread.
>
> For anything critical and urgent, please mail in gluster-devel asking
> jenkins maintainers to trigger build from you.
>
> Thanks,
> Raghavendra Talur
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Gerrit trigger connection to jenkins not working

[Raghavendra Talur]

Hi All,

I see that jenkins isn't getting any triggers from gerrit. This includes
patch set update trigger, so basically no patch is automatically tested. I
am looking into it. Will update in this thread.

For anything critical and urgent, please mail in gluster-devel asking
jenkins maintainers to trigger build from you.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How to cope with spurious regression failures

[Raghavendra Talur]

On Thu, Mar 3, 2016 at 7:12 AM, Raghavendra Talur <rta...@redhat.com> wrote:

>
>
> On Wed, Feb 10, 2016 at 8:29 PM, Emmanuel Dreyfus <m...@netbsd.org> wrote:
>
>> On Wed, Feb 10, 2016 at 07:30:24PM +0530, Raghavendra Talur wrote:
>> > Any comments before I merge the patch
>> http://review.gluster.org/#/c/13393/ ?
>>
>> The proposal has the merit of adressing the multi-OS case, but fails
>> to address future OS addition. If it does not matter to change the
>> naming the day we add another OS, it is fine for me. Otherwise, I
>> adivse using a 8 bit hex value that will be fine for a long time:
>>
>> K01-test.t kills for Linux
>> K02-test.t kills for NetBSD
>> K03-test.t kills for both Linux and NetBSD
>> K04-test.t kills for new OS 1 we would add later.
>> K08-test.t kills for new OS 2 we would add later.
>> K10-test.t kills for new OS 3 we would add later.
>> K19-tets.t kills for Linux, new OS 2 and new OS 3...
>>
>> Of course if we add more than 8 OS in regression that is not enough,
>> but you can start with a 16 bit value if you prefer :-)
>>
>>
> OK, I have updated the patch and replied to Manu's question and Jeff's
> question on the patch itself.
> The tests passed on the first run itself, except for the NetBSD with
> "another test running on slave" error.
> I consider passing on first run itself as a great improvement compared to
> our current state.
>
> Please review the patch: http://review.gluster.org/#/c/13393/
>

Next patch set uploaded, hopefully this is final one.
Prasanna had a comment that he did not like encoding of info in test name.
Also, parsing will become difficult if we ever want other data to be
encoded in file name.

This new patch set puts all the data in the test file as a comment.
At the most basic level it is a csv of key,value pairs.

Please have a look, http://review.gluster.org/#/c/13393/


>
> --
>> Emmanuel Dreyfus
>> m...@netbsd.org
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Code-Review+2 and Verified+1 cause multiple retriggers on Jenkins

[Raghavendra Talur]

On Fri, Mar 4, 2016 at 6:13 PM, Raghavendra Talur <rta...@redhat.com> wrote:

>
>
> On Thu, Feb 4, 2016 at 7:13 PM, Niels de Vos <nde...@redhat.com> wrote:
>
>> On Thu, Feb 04, 2016 at 04:15:16PM +0530, Raghavendra Talur wrote:
>> > On Thu, Feb 4, 2016 at 4:13 PM, Niels de Vos <nde...@redhat.com> wrote:
>> >
>> > > On Thu, Feb 04, 2016 at 03:34:05PM +0530, Raghavendra Talur wrote:
>> > > > Hi,
>> > > >
>> > > > We recently changed the jenkins builds to be triggered on the
>> following
>> > > > triggers.
>> > > >
>> > > > 1. Verified+1
>> > > > 2. Code-review+2
>> > > > 3. recheck (netbsd|centos|smoke)
>> > > >
>> > > > There is a bug in 1 and 2.
>> > > >
>> > > > Multiple triggers of 1 or 2 would result in re-runs even when not
>> > > intended.
>> > > >
>> > > > I would like to replace 1 and 2 with a comment "run-all-regression"
>> or
>> > > > something like that.
>> > > > Thoughts?
>> > >
>> > > Maybe starting regressions on Code-Review +1 (or +2) only?
>> > >
>> >
>> > Multiple code-reviews would do multiple triggers. Won't work.
>>
>> How can we make this to work, without the need of providing magic
>> comments?
>>
>
> I investigated but couldn't find a way to make it work. Discussed with
> Kaushal and we feel it should be ok to go with a "check all" comment for
> initial regression run and deprecate Code-Review+2 and Verified+1 triggers.
>
> I would like to go ahead and do it as the build queue is increasing again
> just because of Code-Review+2's given just before a patch is merged; they
> don't serve any purpose.
>

I have for now just removed trigger for code-review.
Trigger for verified+1 remains as is.
No new trigger on comments have been added.


>> Niels
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Code-Review+2 and Verified+1 cause multiple retriggers on Jenkins

[Raghavendra Talur]

On Thu, Feb 4, 2016 at 7:13 PM, Niels de Vos <nde...@redhat.com> wrote:

> On Thu, Feb 04, 2016 at 04:15:16PM +0530, Raghavendra Talur wrote:
> > On Thu, Feb 4, 2016 at 4:13 PM, Niels de Vos <nde...@redhat.com> wrote:
> >
> > > On Thu, Feb 04, 2016 at 03:34:05PM +0530, Raghavendra Talur wrote:
> > > > Hi,
> > > >
> > > > We recently changed the jenkins builds to be triggered on the
> following
> > > > triggers.
> > > >
> > > > 1. Verified+1
> > > > 2. Code-review+2
> > > > 3. recheck (netbsd|centos|smoke)
> > > >
> > > > There is a bug in 1 and 2.
> > > >
> > > > Multiple triggers of 1 or 2 would result in re-runs even when not
> > > intended.
> > > >
> > > > I would like to replace 1 and 2 with a comment "run-all-regression"
> or
> > > > something like that.
> > > > Thoughts?
> > >
> > > Maybe starting regressions on Code-Review +1 (or +2) only?
> > >
> >
> > Multiple code-reviews would do multiple triggers. Won't work.
>
> How can we make this to work, without the need of providing magic
> comments?
>

I investigated but couldn't find a way to make it work. Discussed with
Kaushal and we feel it should be ok to go with a "check all" comment for
initial regression run and deprecate Code-Review+2 and Verified+1 triggers.

I would like to go ahead and do it as the build queue is increasing again
just because of Code-Review+2's given just before a patch is merged; they
don't serve any purpose.


>
> Niels
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How to cope with spurious regression failures

[Raghavendra Talur]

On Mar 3, 2016 7:42 AM, "Emmanuel Dreyfus" <m...@netbsd.org> wrote:
>
> Raghavendra Talur <rta...@redhat.com> wrote:
>
> > The tests passed on the first run itself, except for the NetBSD with
> > "another test running on slave" error.
>
> Was the previous test on the slave canceled?

Yes,  because I updated from patch set 2 to 3 and tests for 2 were running
on the same slave.

Job for patch set 2
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/14816/

Job for patch set 3
https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/14817/

>
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How to cope with spurious regression failures

[Raghavendra Talur]

On Wed, Feb 10, 2016 at 8:29 PM, Emmanuel Dreyfus <m...@netbsd.org> wrote:

> On Wed, Feb 10, 2016 at 07:30:24PM +0530, Raghavendra Talur wrote:
> > Any comments before I merge the patch
> http://review.gluster.org/#/c/13393/ ?
>
> The proposal has the merit of adressing the multi-OS case, but fails
> to address future OS addition. If it does not matter to change the
> naming the day we add another OS, it is fine for me. Otherwise, I
> adivse using a 8 bit hex value that will be fine for a long time:
>
> K01-test.t kills for Linux
> K02-test.t kills for NetBSD
> K03-test.t kills for both Linux and NetBSD
> K04-test.t kills for new OS 1 we would add later.
> K08-test.t kills for new OS 2 we would add later.
> K10-test.t kills for new OS 3 we would add later.
> K19-tets.t kills for Linux, new OS 2 and new OS 3...
>
> Of course if we add more than 8 OS in regression that is not enough,
> but you can start with a 16 bit value if you prefer :-)
>
>
OK, I have updated the patch and replied to Manu's question and Jeff's
question on the patch itself.
The tests passed on the first run itself, except for the NetBSD with
"another test running on slave" error.
I consider passing on first run itself as a great improvement compared to
our current state.

Please review the patch: http://review.gluster.org/#/c/13393/

--
> Emmanuel Dreyfus
> m...@netbsd.org
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Gerrit search bookmarks that work in firefox too

[Raghavendra Talur]

Hi All,

I use these two gerrit bookmarks for easy search, just wanted to send on
the mailing list as many were interested in it.


*Patches that have passed all tests:*
http://review.gluster.org/#/q/status:open+label:CentOS-regression%2B1+AND+label:NetBSD-regression%2B1+AND+label:smoke%2B1


*Patches that have passed all tests and are waiting for my review:*
http://review.gluster.org/#/q/status:open+label:CentOS-regression%2B1+AND+label:NetBSD-regression%2B1+AND+label:smoke%2B1+AND+reviewer:self

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How to cope with spurious regression failures

[Raghavendra Talur]

Any comments before I merge the patch http://review.gluster.org/#/c/13393/ ?

On Mon, Feb 8, 2016 at 3:15 PM, Raghavendra Talur <rta...@redhat.com> wrote:

>
>
> On Tue, Jan 19, 2016 at 8:33 PM, Emmanuel Dreyfus <m...@netbsd.org> wrote:
>
>> On Tue, Jan 19, 2016 at 07:08:03PM +0530, Raghavendra Talur wrote:
>> > a. Allowing re-running to tests to make them pass leads to complacency
>> with
>> > how tests are written.
>> > b. A test is bad if it is not deterministic and running a bad test has
>> *no*
>> > value. We are wasting time even if the test runs for a few seconds.
>>
>> I agree with your vision for the long term, but my proposal address the
>> short term situation. But we could use the retry approahc to fuel your
>> blacklist approach:
>>
>> We could immagine a system where the retry feature would cast votes on
>> individual tests: each time we fail once and succeed on retry, cast
>> a +1 unreliable for the test.
>>
>> After a few days, we will have a wall of shame for unreliable tests,
>> which could either be fixed or go to the blacklist.
>>
>> I do not know what software to use to collect and display the results,
>> though. Should we have a gerrit change for each test?
>>
>
> This should be the process of adding tests to bad tests list. However, I
> have run out of time on this one.
> If someone would like to implement go ahead. I don't see myself trying
> this any soon.
>
>
>>
>> --
>> Emmanuel Dreyfus
>> m...@netbsd.org
>
>
>
> Thanks for the inputs.
>
> I have refactored run-tests.sh to use retry option.
> If run-tests.sh is started with -r flag, failed tests would be run once
> again and won't be considered as failed if they pass. Note: Adding -r flag
> to jenkins config is not done yet.
>
> I have also implemented a better version of blacklist which complies with
> requirements from Manu on granularity of bad tests to be OS.
> Here is the patch: http://review.gluster.org/#/c/13393/
>
>
>
>
>
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Gluster test infrastructure and current challenges

[Raghavendra Talur]

Hey folks,

Gluster test infrastructure needs a bit of attention. We have committed a
lot of code in the last couple of years, but we have not scaled our
infrastructure at the same rate. Lately, the signs have become alarming and
it calls for attention.

We have scheduled a hangout[1] to educate new Devs about
1. Current Gluster test framework - TAP, prove, include.rc.
2. Challenges
 a. Regression takes a lot of time to run
 b. We have so many non-deterministic tests
 c. Identifying cause  for test failure takes time
 d. NetBSD debugging knowledge in the community is scarce.
 e. DISTAF: Multinode testing is not integrated with regression yet.
 f. More types of tests to have: upgrade, performance, integration.
 g. unit tests

Note that agenda is mainly to showcase the current challenges through a
medium more informative than email/irc. It is not to discuss possible
solutions; that should be done over email on the gluster-devel mailing list.

Hence this hangout is scheduled keeping in view convenience of devs from
IST or around. We will have the session recorded for everyone to view at
their own convenience. Also we could do the same session for different time
zones.

[1] https://plus.google.com/events/c25hcj9llrvvp3dcidqk26sjrhs

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster test infrastructure and current challenges

[Raghavendra Talur]

On Tue, Feb 9, 2016 at 9:12 PM, Amye Scavarda <a...@redhat.com> wrote:

>
>
> On Tue, Feb 9, 2016 at 3:41 PM, Raghavendra Talur <rta...@redhat.com>
> wrote:
>
>> Hey folks,
>>
>> Gluster test infrastructure needs a bit of attention. We have committed a
>> lot of code in the last couple of years, but we have not scaled our
>> infrastructure at the same rate. Lately, the signs have become alarming and
>> it calls for attention.
>>
>> We have scheduled a hangout[1] to educate new Devs about
>> 1. Current Gluster test framework - TAP, prove, include.rc.
>> 2. Challenges
>>  a. Regression takes a lot of time to run
>>  b. We have so many non-deterministic tests
>>  c. Identifying cause  for test failure takes time
>>  d. NetBSD debugging knowledge in the community is scarce.
>>  e. DISTAF: Multinode testing is not integrated with regression yet.
>>  f. More types of tests to have: upgrade, performance, integration.
>>  g. unit tests
>>
>> Note that agenda is mainly to showcase the current challenges through a
>> medium more informative than email/irc. It is not to discuss possible
>> solutions; that should be done over email on the gluster-devel mailing list.
>>
>> Hence this hangout is scheduled keeping in view convenience of devs from
>> IST or around. We will have the session recorded for everyone to view at
>> their own convenience. Also we could do the same session for different time
>> zones.
>>
>
> This is for tomorrow, Feb 10th? Or am I misreading this event?
> With this short of notice, will we be able to get people to attend?
>
> Thanks!
>  -amye
>
>


Originally this was supposed to be a record and upload to youtube kind of
presentation.
We changed it to hangout thinking it could help being a realtime broadcast.

Considering it now, I feel we add no value to the event making it a hangout
if it is just a demo.
We need a hangout/irc meeting for discussion.

I will cancel the hangout for now and upload a demo somewhere with the
agenda mentioned above.(It would just explain current state)
We can discuss in community meeting today for a better date and time to
hold a meeting to discuss solutions for challenges that we mention in the
demo.

Thanks for the input Amye.

NOTE: The hangout has been cancelled. We will send invite for the
rescheduled session later.


>>
>> [1] https://plus.google.com/events/c25hcj9llrvvp3dcidqk26sjrhs
>>
>> Thanks,
>> Raghavendra Talur
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
> Amye Scavarda | a...@redhat.com | Gluster Community Lead
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How to cope with spurious regression failures

[Raghavendra Talur]

On Tue, Jan 19, 2016 at 8:33 PM, Emmanuel Dreyfus <m...@netbsd.org> wrote:

> On Tue, Jan 19, 2016 at 07:08:03PM +0530, Raghavendra Talur wrote:
> > a. Allowing re-running to tests to make them pass leads to complacency
> with
> > how tests are written.
> > b. A test is bad if it is not deterministic and running a bad test has
> *no*
> > value. We are wasting time even if the test runs for a few seconds.
>
> I agree with your vision for the long term, but my proposal address the
> short term situation. But we could use the retry approahc to fuel your
> blacklist approach:
>
> We could immagine a system where the retry feature would cast votes on
> individual tests: each time we fail once and succeed on retry, cast
> a +1 unreliable for the test.
>
> After a few days, we will have a wall of shame for unreliable tests,
> which could either be fixed or go to the blacklist.
>
> I do not know what software to use to collect and display the results,
> though. Should we have a gerrit change for each test?
>

This should be the process of adding tests to bad tests list. However, I
have run out of time on this one.
If someone would like to implement go ahead. I don't see myself trying this
any soon.


>
> --
> Emmanuel Dreyfus
> m...@netbsd.org



Thanks for the inputs.

I have refactored run-tests.sh to use retry option.
If run-tests.sh is started with -r flag, failed tests would be run once
again and won't be considered as failed if they pass. Note: Adding -r flag
to jenkins config is not done yet.

I have also implemented a better version of blacklist which complies with
requirements from Manu on granularity of bad tests to be OS.
Here is the patch: http://review.gluster.org/#/c/13393/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Retrigger request: https://build.gluster.org/job/rackspace-regression-2GB-triggered/18066/

[Raghavendra Talur]

On Sun, Feb 7, 2016 at 2:51 PM, Nithya Balachandran 
wrote:

> Hi,
>
> Can someone please retrigger this regression? It has been aborted in the
> middle of the run.
>
>
Du had re-triggered centos and it is running now.
Netbsd had failed for the same reason.
I have re-triggered.

We are seeing timeouts between slave and jenkins.


> Thanks,
> Nithya
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Code-Review+2 and Verified+1 cause multiple retriggers on Jenkins

[Raghavendra Talur]

Hi,

We recently changed the jenkins builds to be triggered on the following
triggers.

1. Verified+1
2. Code-review+2
3. recheck (netbsd|centos|smoke)

There is a bug in 1 and 2.

Multiple triggers of 1 or 2 would result in re-runs even when not intended.

I would like to replace 1 and 2 with a comment "run-all-regression" or
something like that.
Thoughts?


Thanks
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Code-Review+2 and Verified+1 cause multiple retriggers on Jenkins

[Raghavendra Talur]

On Thu, Feb 4, 2016 at 4:13 PM, Niels de Vos <nde...@redhat.com> wrote:

> On Thu, Feb 04, 2016 at 03:34:05PM +0530, Raghavendra Talur wrote:
> > Hi,
> >
> > We recently changed the jenkins builds to be triggered on the following
> > triggers.
> >
> > 1. Verified+1
> > 2. Code-review+2
> > 3. recheck (netbsd|centos|smoke)
> >
> > There is a bug in 1 and 2.
> >
> > Multiple triggers of 1 or 2 would result in re-runs even when not
> intended.
> >
> > I would like to replace 1 and 2 with a comment "run-all-regression" or
> > something like that.
> > Thoughts?
>
> Maybe starting regressions on Code-Review +1 (or +2) only?
>

Multiple code-reviews would do multiple triggers. Won't work.


>
> Niels
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Different version of run-tests.sh in jenkin slaves?

[Raghavendra Talur]

On Thu, Jan 28, 2016 at 11:17 AM, Atin Mukherjee 
wrote:

> Are we running a different version of run-tests.sh in jenkin slaves. The
> reason of suspection is beacuse in last couple of runs [1] & [2] in
> NetBSD I am seeing no failures apart from bad tests but the regression
> voted failure and I can not make out any valid reason out of it.
>
> [1]
>
> https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/13756/consoleFull
> [2]
>
> https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/13755/consoleFull



I checked the slave machine now.
regression.sh file is different but the run-tests.sh script is same.

A wild guess here, is it possible that the core generation takes time and
when we check for a core right after a test is run it is not present yet?
Does anyone know how to work around that?


>
> ~Atin
> ___
> Gluster-infra mailing list
> gluster-in...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-infra
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Different version of run-tests.sh in jenkin slaves?

[Raghavendra Talur]

Ok, RCA:

In NetBSD cores are being generated in /d/backends/*/*.core
run-tests.sh looks only for "/core*" when looking for cores.

So, at the end of test run when regression.sh looks for core everywhere, it
finds one and errors out.

Should think of a solution which is generic. Will update.


On Thu, Jan 28, 2016 at 11:37 AM, Raghavendra Talur <rta...@redhat.com>
wrote:

>
>
> On Thu, Jan 28, 2016 at 11:17 AM, Atin Mukherjee <amukh...@redhat.com>
> wrote:
>
>> Are we running a different version of run-tests.sh in jenkin slaves. The
>> reason of suspection is beacuse in last couple of runs [1] & [2] in
>> NetBSD I am seeing no failures apart from bad tests but the regression
>> voted failure and I can not make out any valid reason out of it.
>>
>> [1]
>>
>> https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/13756/consoleFull
>> [2]
>>
>> https://build.gluster.org/job/rackspace-netbsd7-regression-triggered/13755/consoleFull
>
>
>
> I checked the slave machine now.
> regression.sh file is different but the run-tests.sh script is same.
>
> A wild guess here, is it possible that the core generation takes time and
> when we check for a core right after a test is run it is not present yet?
> Does anyone know how to work around that?
>
>
>>
>> ~Atin
>> ___
>> Gluster-infra mailing list
>> gluster-in...@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-infra
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Different version of run-tests.sh in jenkin slaves?

[Raghavendra Talur]

On Thu, Jan 28, 2016 at 12:10 PM, Atin Mukherjee <amukh...@redhat.com>
wrote:

>
>
> On 01/28/2016 12:00 PM, Raghavendra Talur wrote:
> > Ok, RCA:
> >
> > In NetBSD cores are being generated in /d/backends/*/*.core
> > run-tests.sh looks only for "/core*" when looking for cores.
> >
> > So, at the end of test run when regression.sh looks for core everywhere,
> > it finds one and errors out.
> So does that mean we never analyzed any core reported by NetBSD
> regression failure? That's strange.
>

I wonder when these changes were done.

Manu,

Where do I find config in NetBSD which decides which location to dump core
in?
Any particular reason you added /d/backends/*/*.core to list of path to
search for core?

>
> > Should think of a solution which is generic. Will update.
> >
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Patches for supporting lseek() with SEEK_DATA/SEEK_HOLE available for review

[Raghavendra Talur]

On Wed, Jan 27, 2016 at 5:34 PM, Niels de Vos  wrote:

> Hi all,
>
> yesterday I have posted the latest version of a series that implements
> support for lseek() with SEEK_DATA/SEEK_HOLE. This functionality can
> improve the handling of sparse files immensely.
>
> Please review the changes that involve components if your interest:
>
>
> http://review.gluster.org/#/q/status:open+project:glusterfs+branch:master+topic:bug-1220173
>
> Once these changes made it in, we can detect holes with glfs_lseek() and
> can improve NFS-Ganesha, QEMU, Samba and glusterfs-coreutils. There are
> probably other projects where we should contribute improvements for
> SEEK_DATA/SEEK_HOLE (like 'cp' from coreutils?).
>
> Ravi already got his Linux kernel FUSE patch merged, kernel-4.5 will
> support lseek() through FUSE as well:
>
>
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/fs/fuse?id=0b5da8db145bfd44266ac964a2636a0cf8d7c286
>
> Eventhough this is a fairly basic (though relatively new) filesystem
> functionality, I want to document it in a feature page in our
> glusterfs-specs repository:
>
>   https://github.com/gluster/glusterfs-specs
>
> Once the document is posted in Gerrit, I'll send a reply to this email
> to get some more reviews.
>
> Thanks,
> Niels
>
>
> Note to other maintainers:
>
> There is some order of merging changes needed. A new FOP has been
> added, and some patches depend on the new GF_FOP_SEEK or other core
> bits.
>
> Change-Id: I4b74fce8b0bad2f45291fd2c2b9e243c4f4a1aa9
>core: add seek() FOP
>
> Change-Id: I0c15153beb27de73d5844b6f692175750fc28f60
>syncop: add seek() FOP
>
> Change-Id: I060768a8a4b9b1c80f4a24c0f17d630f7f028690
>protocol: implement seek() FOP
>
> Change-Id: I100b741d9bfacb799df318bb081f2497c0664927
>afr: add seek() FOP
>
> Change-Id: Iaa23ba81df4ee78ddaab1f96b3d926a563b4bb3d
>cluster/ec: add seek() FOP
>
> Change-Id: I5c272855a21501ac31e1a5f4b68ed7245582c17c
>shard: add seek() FOP as not supported
>
> Change-Id: I5d15533c53fd710497f97c3cb4a8ea29fba47271
>posix: implement seek() FOP
>
> Change-Id: Ic86919d28cf639b561114dc1440c6ea4bc6f7307
>upcall: add seek() FOP
>
> Change-Id: I142dde11923244809b03fcca8cd4c2f7d5ff3929
>gfapi: add support for SEEK_HOLE and SEEK_DATA in
>
> Change-Id: I8d0573ed8b2ea5ce976ad140a24be7974dbad0e3
>gfapi: add test for glfs_lseek() SEEK_DATA/SEEK_HOLE
>
> Change-Id: I58c74e161638b2d4ce12fc91a206fdc1b96de14d
>fuse: update fuse_kernel.h to version 23
>
> Change-Id: I12496d788e59461a3023ddd30e0ea3179007f77e
>fuse: add support for SEEK_HOLE and SEEK_DATA through lseek()
>
> Change-Id: Ifdee3c793e33f9d763940130e8d01a61eae5498a
>tests: add seek program for testing SEEK_DATA/SEEK_HOLE over
>
> Change-Id: I9cfed795737609120eafe86f9287d79057b14fe2
>stripe: add seek() FOP as not supported
>


Thanks for the order.

Today, shard patch[1] for seek() had got merged before we could look at
this mail.
The revert patch for that is merged now[2]. I request you to send another
patch for the same.


[1] http://review.gluster.org/#/c/13290/
[2] http://review.gluster.org/#/c/13301/




>
> ___
> maintainers mailing list
> maintain...@gluster.org
> http://www.gluster.org/mailman/listinfo/maintainers
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Simple way to re-trigger jenkins jobs

[Raghavendra Talur]

Hi,

Prashanth informed about a simple way to let every contributor retrigger
the regression runs that openstack uses.

We have implemented same for gluster. Trigger comments are:

recheck netbsd
recheck smoke
recheck centos

You just have to reply on the patch set with one of the following comments
above to trigger the job again.

With this we will have time to follow proper process for getting jenkins
account for developers.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Smoke tests run on the builder in RH DC (at least)

[Raghavendra Talur]

On Mon, Jan 25, 2016 at 11:29 PM, Michael Scherer 
wrote:

> Hi,
>
> so today, after fixing one last config item, the smoke test jobs run
> fine on the Centos 6 builder in the RH DC, which build things as non
> root, then start the tests, then reboot the server.
>

Awesome!


>
> Now, I am looking at the fedora one, but once this one is good, I will
> likely reinstall a few builders as a test, and go on Centos 7 builder.
>

This is what I had to do to get Fedora working. Ansible lines are shown
where applicable.

1. change ownership for python site packages: difference is in version 2.7
when compared to 2.6 of CentOS
file: path=/usr/lib/python2.7/site-packages/gluster/ state=directory
owner=jenkins group=root

2. Had to give jenkins write permission on /usr/lib/systemd/system/
for installing glusterd service file.




> I was also planning to look at jenkins job builder for the jenkins, but
> no time yet. Will be after jenkins migration to a new host (which is
> still not planned, unlike gerrit where we should be attempting to find a
> time for that)
>
>
> --
> Michael Scherer
> Sysadmin, Community Infrastructure and Platform, OSAS
>
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Tips and Tricks for Gluster Developer

[Raghavendra Talur]

I don't like installing the bits under /usr/local so I configure and
compile them to install in the same place as a Fedora rpm would.
Here is my compile command


./autogen.sh

CFLAGS="-g -O0 -Werror -Wall -Wno-error=cpp -Wno-error=maybe-uninitialized"
./configure \
--prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--sysconfdir=/etc \
--datadir=/usr/share \
--includedir=/usr/include \
--libdir=/usr/lib64 \
--libexecdir=/usr/libexec \
--localstatedir=/var \
--sharedstatedir=/var/lib \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--libdir=/usr/lib64 \
--enable-debug

make install




On Fri, Jan 22, 2016 at 7:43 PM, Raghavendra Talur <rta...@redhat.com>
wrote:

> HI All,
>
> I am sure there are many tricks hidden under sleeves of many Gluster
> developers.
> I realized this when speaking to new developers. It would be good have a
> searchable thread of such tricks.
>
> Just reply back on this thread with the tricks that you have and I promise
> I will collate them and add them to developer guide.
>
>
> Looking forward to be amazed!
>
> Thanks,
> Raghavendra Talur
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Jenkins accounts for all devs.

[Raghavendra Talur]

On Fri, Jan 22, 2016 at 2:41 PM, Michael Scherer <msche...@redhat.com>
wrote:

> Le vendredi 22 janvier 2016 à 11:31 +0530, Ravishankar N a écrit :
> > On 01/14/2016 12:16 PM, Kaushal M wrote:
> > > On Thu, Jan 14, 2016 at 10:33 AM, Raghavendra Talur <rta...@redhat.com>
> wrote:
> > >>
> > >> On Thu, Jan 14, 2016 at 10:32 AM, Ravishankar N <
> ravishan...@redhat.com>
> > >> wrote:
> > >>> On 01/08/2016 12:03 PM, Raghavendra Talur wrote:
> > >>>> P.S: Stop using the "universal" jenkins account to trigger jenkins
> build
> > >>>> if you are not a maintainer.
> > >>>> If you are a maintainer and don't have your own jenkins account
> then get
> > >>>> one soon!
> > >>>>
> > >>> I would request for a jenkins account for non-maintainers too, at
> least
> > >>> for the devs who are actively contributing code (as opposed to random
> > >>> one-off commits from persons). That way, if the regression failure is
> > >>> *definitely* not in my patch (or) is a spurious failure (or) is
> something
> > >>> that I need to take a netbsd slave offline to debug etc.,  I don't
> have to
> > >>> be blocked on the Maintainer. Since the accounts are anyway tied to
> an
> > >>> individual, it should be easy to spot if someone habitually
> re-trigger
> > >>> regressions without any initial debugging.
> > >>>
> > >> +1
> > > We'd like to give everyone accounts. But the way we're providing
> > > accounts now gives admin accounts to all. This is not very secure.
> > >
> > > This was one of the reasons misc setup freeipa.gluster.org, to provide
> > > controlled accounts for all. But it hasn't been used yet. We would
> > > need to integrate jenkins and the slaves with freeipa, which would
> > > give everyone easy access.
> >
> > Hi Michael,
> > Do you think it is possible to have this integration soon so that all
> > contributors can re-trigger/initiate builds by themselves?
>
> The thing that is missing is still the same, how do we consider that
> someone is a contributor. IE, do we want people just say "add me" and
> get root access to all our jenkins builder (because that's also what go
> with jenkins way of restarting a build for now) ?
>
> I did the technical stuff, but so far, no one did the organisational
> part of giving a criteria for who has access to what. Without clear
> process, I can't do much.
>


+ndevos +vijay

Something like "should have contributed 10 patches to Gluster and be
supported by at least 1 maintainer" would do?



--
> Michael Scherer
> Sysadmin, Community Infrastructure and Platform, OSAS
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Tips and Tricks for Gluster Developer

[Raghavendra Talur]

HI All,

I am sure there are many tricks hidden under sleeves of many Gluster
developers.
I realized this when speaking to new developers. It would be good have a
searchable thread of such tricks.

Just reply back on this thread with the tricks that you have and I promise
I will collate them and add them to developer guide.


Looking forward to be amazed!

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] heal hanging

[Raghavendra Talur]

On Jan 22, 2016 7:27 AM, "Pranith Kumar Karampuri" <pkara...@redhat.com>
wrote:
>
>
>
> On 01/22/2016 07:19 AM, Pranith Kumar Karampuri wrote:
>>
>>
>>
>> On 01/22/2016 07:13 AM, Glomski, Patrick wrote:
>>>
>>> We use the samba glusterfs virtual filesystem (the current version
provided on download.gluster.org), but no windows clients connecting
directly.
>>
>>
>> Hmm.. Is there a way to disable using this and check if the CPU% still
increases? What getxattr of "glusterfs.get_real_filename " does is
to scan the entire directory looking for strcasecmp(,
). If anything matches then it will return the
. But the problem is the scan is costly. So I wonder if
this is the reason for the CPU spikes.
>
> +Raghavendra Talur, +Poornima
>
> Raghavendra, Poornima,
> When are these getxattrs triggered? Did you guys see any
brick CPU spikes before? I initially thought it could be because of big
directory heals. But this is happening even when no self-heals are
required. So I had to move away from that theory.

These getxattrs are triggered when a SMB client performs a path based
operation. It is necessary then that some client was connected.

The last fix to go in that code for 3.6 was
http://review.gluster.org/#/c/10403/.

I am not able to determine which release of 3.6 it made into. Will update.

Also we would need version of Samba installed. Including the vfs plugin
package.

There is a for loop of strcmp involved here which does take a lot of CPU.
It should be for short bursts though and is expected and harmless.

>
> Pranith
>
>>
>> Pranith
>>>
>>>
>>> On Thu, Jan 21, 2016 at 8:37 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:
>>>>
>>>> Do you have any windows clients? I see a lot of getxattr calls for
"glusterfs.get_real_filename" which lead to full readdirs of the
directories on the brick.
>>>>
>>>> Pranith
>>>>
>>>> On 01/22/2016 12:51 AM, Glomski, Patrick wrote:
>>>>>
>>>>> Pranith, could this kind of behavior be self-inflicted by us deleting
files directly from the bricks? We have done that in the past to clean up
an issues where gluster wouldn't allow us to delete from the mount.
>>>>>
>>>>> If so, is it feasible to clean them up by running a search on the
.glusterfs directories directly and removing files with a reference count
of 1 that are non-zero size (or directly checking the xattrs to be sure
that it's not a DHT link).
>>>>>
>>>>> find /data/brick01a/homegfs/.glusterfs -type f -not -empty -links -2
-exec rm -f "{}" \;
>>>>>
>>>>> Is there anything I'm inherently missing with that approach that will
further corrupt the system?
>>>>>
>>>>>
>>>>> On Thu, Jan 21, 2016 at 1:02 PM, Glomski, Patrick <
patrick.glom...@corvidtec.com> wrote:
>>>>>>
>>>>>> Load spiked again: ~1200%cpu on gfs02a for glusterfsd. Crawl has
been running on one of the bricks on gfs02b for 25 min or so and users
cannot access the volume.
>>>>>>
>>>>>> I re-listed the xattrop directories as well as a 'top' entry and
heal statistics. Then I restarted the gluster services on gfs02a.
>>>>>>
>>>>>> === top ===
>>>>>> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+
COMMAND
>>>>>>  8969 root  20   0 2815m 204m 3588 S 1181.0  0.6 591:06.93
glusterfsd
>>>>>>
>>>>>> === xattrop ===
>>>>>> /data/brick01a/homegfs/.glusterfs/indices/xattrop:
>>>>>> xattrop-41f19453-91e4-437c-afa9-3b25614de210
xattrop-9b815879-2f4d-402b-867c-a6d65087788c
>>>>>>
>>>>>> /data/brick02a/homegfs/.glusterfs/indices/xattrop:
>>>>>> xattrop-70131855-3cfb-49af-abce-9d23f57fb393
xattrop-dfb77848-a39d-4417-a725-9beca75d78c6
>>>>>>
>>>>>> /data/brick01b/homegfs/.glusterfs/indices/xattrop:
>>>>>> e6e47ed9-309b-42a7-8c44-28c29b9a20f8
xattrop-5c797a64-bde7-4eac-b4fc-0befc632e125
>>>>>> xattrop-38ec65a1-00b5-4544-8a6c-bf0f531a1934
xattrop-ef0980ad-f074-4163-979f-16d5ef85b0a0
>>>>>>
>>>>>> /data/brick02b/homegfs/.glusterfs/indices/xattrop:
>>>>>> xattrop-7402438d-0ee7-4fcf-b9bb-b561236f99bc
xattrop-8ffbf5f7-ace3-497d-944e-93ac85241413
>>>>>>
>>>>>> /data/brick01a/homegfs/.glusterfs/indices/xattrop:
>>>>>> xa

Re: [Gluster-devel] Linux regression tests are hanging too

[Raghavendra Talur]

On Tue, Jan 19, 2016 at 5:20 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> Result: PASS
> Build timed out (after 300 minutes). Marking the build as failed.
> Build was aborted
> Finished: FAILURE
>
>
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/17664/console
>
> Pranith
>

This is due to build timeout. I had set it to 300 minute (5 hours).
Recently though, some patch has caused a major performance regression.
Tests which used to pass in 120 seconds are taking more than 200.
This has increased test run time from 200 minutes to greater than 300.

I have increased timeout value to 500 minutes now and it should not get
aborted.

Someone really needs to look into the regression though. Pick any test from
the console output above and run a .t file that took over 20 ms.

Thanks,
Raghavendra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How to cope with spurious regression failures

[Raghavendra Talur]

On Tue, Jan 19, 2016 at 5:21 PM, Atin Mukherjee <amukh...@redhat.com> wrote:

>
>
> On 01/19/2016 10:45 AM, Emmanuel Dreyfus wrote:
> > Hi
> >
> > Spurious regression failures make developers frustrated. One submits a
> > change and gets completely unrelated failures. The only way out is to
> > retrigger regression until it passes, a boring and time-wasting task.
> > Sometimes after 4 or 5 failed runs, the submitter realize there is a
> > real issue and look at it, which is a waste of time and resources.
> >
> > The fact that we run regression on multiple platforms makes the
> > situation worse. If you have 10% of chances to hit a spurious failure on
> > Linux and a 20% chances to hit a spurious failure on NetBSD (random
> > number chosen), that means you get roughtly a failure for four
> > submissions (random prediction, as I used random input numbers, but you
> > get the idea)
> >
> > Two solutions are proposed:
> >
> > 1) do not run unreliable tests, as proposed by Raghavendra Talur:
> > http://review.gluster.org/13173
> >
> > I have nothing against the idea, but I voted down the change because it
> > fails to address the need for different test blacklists on different
> > platforms: we do not have the same unreliable tests on Linux and NetBSD.
>

Why I prefer having this solution:
a. Allowing re-running to tests to make them pass leads to complacency with
how tests are written.
b. A test is bad if it is not deterministic and running a bad test has *no*
value. We are wasting time even if the test runs for a few seconds.
c. I propose another method to overcome the technical difficulty of having
blacklists for different platforms. We could have "[K[a-z]*-]*" as prefix
of tests where [a-z]* could be L or N signify that the test is bad on Linux
and NetBSD respectively. The run-tests.sh script can be made intelligent
enough to determine host OS and skip them.



> >
> > 2) add a regression option to retry a failed test once, and to validate
> > the regression if second attempt passes, as I proposed:
> > http://review.gluster.org/13245
> >
> > The idea is basicaly to automatically do what every submitter has been
> > doing: retry without a thought when regression fails. The benefit of
> > this approach is also that it gives us a better view of what test failed
> > because of the change, and what test failed because it was unreliable.
> >
> > The retry feature is optionnal and triggered by using the -r flag to
> > run-tests.sh. I intend to use it on NetBSD regression to reduce the
> > number of failures that annoy people. It could be used on Linux
> > regression too, though I do not plan to touch that on my own.
> +1 to option 2
> >
> > Please people tell us what approach you prefer.
> >
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Core from gNFS process

[Raghavendra Talur]

On Sat, Jan 16, 2016 at 2:25 AM, Vijay Bellur  wrote:

> On 01/15/2016 08:38 AM, Soumya Koduri wrote:
>
>>
>>
>>
>> On 01/15/2016 06:52 PM, Soumya Koduri wrote:
>>
>>>
>>>
>>> On 01/14/2016 08:41 PM, Vijay Bellur wrote:
>>>
 On 01/14/2016 04:11 AM, Jiffin Tony Thottan wrote:

>
>
> On 14/01/16 14:28, Jiffin Tony Thottan wrote:
>
>> Hi,
>>
>> The core generated when encryption xlator is enabled
>>
>> [2016-01-14 08:13:15.740835] E
>> [crypt.c:4298:master_set_master_vol_key] 0-test1-crypt: FATAL: missing
>> master key
>> [2016-01-14 08:13:15.740859] E [MSGID: 101019]
>> [xlator.c:429:xlator_init] 0-test1-crypt: Initialization of volume
>> 'test1-crypt' failed, review your volfile again
>> [2016-01-14 08:13:15.740890] E [MSGID: 101066]
>> [graph.c:324:glusterfs_graph_init] 0-test1-crypt: initializing
>> translator failed
>> [2016-01-14 08:13:15.740904] E [MSGID: 101176]
>> [graph.c:670:glusterfs_graph_activate] 0-graph: init failed
>> [2016-01-14 08:13:15.741676] W [glusterfsd.c:1231:cleanup_and_exit]
>> (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x307) [0x40d287]
>> -->/usr/sbin/glusterfs(glusterfs_process_volfp+0x117) [0x4086c7]
>> -->/usr/sbin/glusterfs(cleanup_and_exit+0x4d) [0x407e1d] ) 0-:
>> received signum (0), shutting down
>>
>>
>>
> Forgot to mention this last mail,  for crypt xlator needs master key
> before enabling the translator which cause the issue
> --
>

 Irrespective of the problem, the nfs process should not crash. Can we
 check why there is a memory corruption during cleanup_and_exit()?

 That's right. This issue was reported quite a few times earlier in
>>> gluster-devel and it is not specific to gluster-nfs process. As updated
>>> in [1], we have raised bug1293594[2] against lib-gcc team to further
>>> investigate this.
>>>
>>
> The segmentation fault in gcc is while attempting to print a backtrace
> upon glusterfs receiving a SIGSEGV. It would be good to isolate the reason
> for the initial SIGSEGV whose signal handler causes the further crash.


I wasn't able to check this today. Will check tomorrow.


>
>
> -Vijay
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster Build system's voting for regression run

[Raghavendra Talur]

On Wed, Jan 13, 2016 at 4:25 PM, Kaleb Keithley  wrote:

> > From: "Niels de Vos" 
> >
> > On Wed, Jan 13, 2016 at 09:56:54AM +0530, Atin Mukherjee wrote:
> > > Hi Raghavendra,
> > >
> > > I noticed that for the below patches Gluster Build system hasn't voted
> > > and hence the verified flag doesn't have an ack from it even if the
> > > regression has passed. I think something is fishy in the script now,
> > > mind having a look?
> > >
> > > http://review.gluster.org/#/c/13222/
> > > http://review.gluster.org/#/c/13210/
> >
> > I've manually approved these now. No idea what caused the voting to have
> > failed. The votes are done through an ssh command in the regression test
> > itself, it is not a property of the test job (which is an option, but
> > gives us less flexibility).
> >
> > Maybe someone can figure out what went wrong? I could not see anything
> > in the console log that suggested a problem.
>
> Did the other tests (smoke, rpmbuild, version-and-branch) run after the
> regression?
>
> The logic is broken and if the other tests are delayed and will erase the
> regression +1Verified.
>

Is this still applicable after removing the voting from smoke, rpmbuild and
other such projects?
IIRC, we removed voting lines from these scripts because of the race reason.


>
> --
>
> Kaleb
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster Build system's voting for regression run

[Raghavendra Talur]

On Wed, Jan 13, 2016 at 2:35 PM, Niels de Vos  wrote:

> On Wed, Jan 13, 2016 at 09:56:54AM +0530, Atin Mukherjee wrote:
> > Hi Raghavendra,
> >
> > I noticed that for the below patches Gluster Build system hasn't voted
> > and hence the verified flag doesn't have an ack from it even if the
> > regression has passed. I think something is fishy in the script now,
> > mind having a look?
> >
> > http://review.gluster.org/#/c/13222/
> > http://review.gluster.org/#/c/13210/
>
> I've manually approved these now. No idea what caused the voting to have
> failed. The votes are done through an ssh command in the regression test
> itself, it is not a property of the test job (which is an option, but
> gives us less flexibility).
>
> Maybe someone can figure out what went wrong? I could not see anything
> in the console log that suggested a problem.
>

I checked the console logs and jenkins log from UI. I did not see any error
being reported.
The commands use by NetBSD and Linux Regression are same too with different
user names.

I guess something is wrong with how Gerrit interprets it.
Someone please check Gerrit logs, I don't know where to check.


>
> HTH,
> Niels
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Core from gNFS process

[Raghavendra Talur]

Hi Jiffin and Soumya,

Ravishankar told me about core generated by gNFS process
during 
./tests/bugs/snapshot/bug-1140162-file-snapshot-features-encrypt-opts-validation.t.

Here is console output:
https://build.gluster.org/job/rackspace-regression-2GB-triggered/17525/console

And here is the backtrace for convenience

(gdb) thread apply all bt

Thread 9 (LWP 12499):
#0  0x7f622f4fda0e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
./lib64/libpthread.so.0
#1  0x7f6230258a61 in syncenv_task (proc=0x7f621c0332f0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:603
#2  0x7f6230258d08 in syncenv_processor (thdata=0x7f621c0332f0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:695
#3  0x7f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
#4  0x7f622ee6393d in clone () from ./lib64/libc.so.6

Thread 8 (LWP 12497):
#0  0x7f622edc2e2c in vfprintf () from ./lib64/libc.so.6
#1  0x7f622edea752 in vsnprintf () from ./lib64/libc.so.6
#2  0x7f6230243f70 in gf_vasprintf (string_ptr=0x7f6220a66ba8,
format=0x7f62302aeacd "[%s] %s [%s:%d:%s] %d-%s: ", arg=0x7f6220a66a70)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/mem-pool.c:219
#3  0x7f62302440ad in gf_asprintf (string_ptr=0x7f6220a66ba8,
format=0x7f62302aeacd "[%s] %s [%s:%d:%s] %d-%s: ")
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/mem-pool.c:239
#4  0x7f623021d387 in _gf_log (domain=0x7f621c00cde0 "d_exit+0x87)
[0x407cdf]",
file=0x7f622272b468 ,
function=0x7f622272d130 , line=2895,
level=GF_LOG_INFO, fmt=0x7f622272c690 )
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/logging.c:2216
#5  0x7f6222725d99 in ?? ()
#6  0x0005 in ?? ()
#7  0x in ?? ()

Thread 7 (LWP 12460):
#0  0x7f622f4fda0e in pthread_cond_timedwait@@GLIBC_2.3.2 () from
./lib64/libpthread.so.0
#1  0x7f6230258a61 in syncenv_task (proc=0x241b210)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:603
#2  0x7f6230258d08 in syncenv_processor (thdata=0x241b210)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/syncop.c:695
#3  0x7f622f4f9a51 in start_thread () from ./lib64/libpthread.so.0
#4  0x7f622ee6393d in clone () from ./lib64/libc.so.6

Thread 6 (LWP 12476):
#0  0x7f622f5002e4 in __lll_lock_wait () from ./lib64/libpthread.so.0
#1  0x7f622f4fb588 in _L_lock_854 () from ./lib64/libpthread.so.0
#2  0x7f622f4fb457 in pthread_mutex_lock () from ./lib64/libpthread.so.0
#3  0x7f623021ca6c in _gf_msg (domain=0x4117ef ,
---Type  to continue, or q  to quit---
file=0x411468 ,
function=0x4125d0 <__FUNCTION__.18918> , line=1231,
level=GF_LOG_WARNING, errnum=0, trace=1, msgid=100032,
fmt=0x411bb0 )
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/logging.c:2055
#4  0x00407cdf in cleanup_and_exit (signum=0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:1231
#5  0x00409ee4 in glusterfs_process_volfp (ctx=0x23f6010,
fp=0x7f621c001400)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd.c:2202
#6  0x0040e71d in mgmt_getspec_cbk (req=0x7f621c001a4c,
iov=0x7f621c001a8c, count=1,
myframe=0x7f621c00135c)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/glusterfsd/src/glusterfsd-mgmt.c:1640
#7  0x7f622ffe242a in rpc_clnt_handle_reply (clnt=0x244e450,
pollin=0x7f621c0026c0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-clnt.c:759
#8  0x7f622ffe28c8 in rpc_clnt_notify (trans=0x244e8d0,
mydata=0x244e480,
event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f621c0026c0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-clnt.c:900
#9  0x7f622ffdeb5a in rpc_transport_notify (this=0x244e8d0,
event=RPC_TRANSPORT_MSG_RECEIVED,
data=0x7f621c0026c0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-lib/src/rpc-transport.c:541
#10 0x7f62257d0dcb in socket_event_poll_in (this=0x244e8d0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2231
#11 0x7f62257d1321 in socket_event_handler (fd=9, idx=1,
data=0x244e8d0, poll_in=1, poll_out=0,
poll_err=0)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/rpc/rpc-transport/socket/src/socket.c:2344
#12 0x7f623027a1c0 in event_dispatch_epoll_handler
(event_pool=0x2414ce0, event=0x7f62241a8e70)
at
/home/jenkins/root/workspace/rackspace-regression-2GB-triggered/libglusterfs/src/event-epoll.c:571
#13 0x7f623027a5ae in event_dispatch_epoll_worker 

Re: [Gluster-devel] [Gluster-Maintainers] IMPORTANT: New jenkins triggering method

[Raghavendra Talur]

On Tue, Jan 12, 2016 at 7:59 PM, Atin Mukherjee <atin.mukherje...@gmail.com>
wrote:

> -Atin
> Sent from one plus one
> On Jan 12, 2016 7:41 PM, "Niels de Vos" <nde...@redhat.com> wrote:
> >
> > On Tue, Jan 12, 2016 at 07:21:37PM +0530, Raghavendra Talur wrote:
> > > We have now changed the gerrit-jenkins workflow as follows:
> > >
> > > 1. Developer works on a new feature/bug fix and tests it locally(run
> > > run-tests.sh completely).
> > > 2. Developer sends the patch to gerrit using rfc.sh.
> > >
> > > +++Note that no regression runs have started automatically for this
> patch
> > > at this point.+++
> > >
> > > 3. Developer marks the patch as +1 verified on gerrit as a promise of
> > > having tested the patch completely. For cases where patches don't have
> a +1
> > > verified from the developer, maintainer has the following options
> > > a. just do the code-review and award a +2 code review.
> > > b. pull the patch locally and test completely and award a +1 verified.
> > > Both the above actions would result in triggering of regression runs
> for
> > > the patch.
> >
> > Would it not help if anyone giving +1 code-review starts the regression
> > tests too? When developers ask me to review, I prefer to see reviews
> > done by others first, and any regression failures should have been fixed
> > by the time I look at the change.
> When this idea was originated (long back) I was in favour of having
> regression triggered on a +1, however verified flag set by the developer
> would still trigger the regression. Being a maintainer I would always
> prefer to look at a patch when its verified  flag is +1 which means the
> regression result would also be available.
>


Niels requested in IRC that it is good have a mechanism of getting all
patches that have already passed all regressions before starting review.
Here is what I found
a. You can use the search string
status:open label:Verified+1,user=build AND label:Verified+1,user=nb7build
b. You can bookmark this link and it will take you directly to the page
with list of such patches.

http://review.gluster.org/#/q/status:open+label:Verified%252B1%252Cuser%253Dbuild+AND+label:Verified%252B1%252Cuser%253Dnb7build


>
> >
> > Niels
> >
> > >
> > > 4. Once the regression runs complete, verification verdict is passed
> onto
> > > gerrit by jenkins
> > > and maintainer can proceed with the merge using submit button.
> > >
> > > Thanks,
> > > Raghavendra Talur
> >
> > > ___
> > > maintainers mailing list
> > > maintain...@gluster.org
> > > http://www.gluster.org/mailman/listinfo/maintainers
> >
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster Build system's voting for regression run

[Raghavendra Talur]

On Wed, Jan 13, 2016 at 6:32 PM, Niels de Vos <nde...@redhat.com> wrote:

> On Wed, Jan 13, 2016 at 04:42:12PM +0530, Raghavendra Talur wrote:
> > On Wed, Jan 13, 2016 at 4:25 PM, Kaleb Keithley <kkeit...@redhat.com>
> wrote:
> >
> > > > From: "Niels de Vos" <nde...@redhat.com>
> > > >
> > > > On Wed, Jan 13, 2016 at 09:56:54AM +0530, Atin Mukherjee wrote:
> > > > > Hi Raghavendra,
> > > > >
> > > > > I noticed that for the below patches Gluster Build system hasn't
> voted
> > > > > and hence the verified flag doesn't have an ack from it even if the
> > > > > regression has passed. I think something is fishy in the script
> now,
> > > > > mind having a look?
> > > > >
> > > > > http://review.gluster.org/#/c/13222/
> > > > > http://review.gluster.org/#/c/13210/
> > > >
> > > > I've manually approved these now. No idea what caused the voting to
> have
> > > > failed. The votes are done through an ssh command in the regression
> test
> > > > itself, it is not a property of the test job (which is an option, but
> > > > gives us less flexibility).
> > > >
> > > > Maybe someone can figure out what went wrong? I could not see
> anything
> > > > in the console log that suggested a problem.
> > >
> > > Did the other tests (smoke, rpmbuild, version-and-branch) run after the
> > > regression?
> > >
> > > The logic is broken and if the other tests are delayed and will erase
> the
> > > regression +1Verified.
> > >
> >
> > Is this still applicable after removing the voting from smoke, rpmbuild
> and
> > other such projects?
> > IIRC, we removed voting lines from these scripts because of the race
> reason.
>
> It seems to be disabled, yes. But I do not know the reason for it, or
> when and where this was discussed/announced. It surely was surprising me
> because the FreeBSD failure might have been noticed much earlier.
>
> Got any pointers?
>

I don't remember if I came to know this through email or IRC and I am not
able to find using google.
My guess is that it was done about a year ago. Kaushal might know more
about it.


>
> Thanks,
> Niels
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] IMPORTANT: New jenkins triggering method

[Raghavendra Talur]

We have now changed the gerrit-jenkins workflow as follows:

1. Developer works on a new feature/bug fix and tests it locally(run
run-tests.sh completely).
2. Developer sends the patch to gerrit using rfc.sh.

+++Note that no regression runs have started automatically for this patch
at this point.+++

3. Developer marks the patch as +1 verified on gerrit as a promise of
having tested the patch completely. For cases where patches don't have a +1
verified from the developer, maintainer has the following options
a. just do the code-review and award a +2 code review.
b. pull the patch locally and test completely and award a +1 verified.
Both the above actions would result in triggering of regression runs for
the patch.

4. Once the regression runs complete, verification verdict is passed onto
gerrit by jenkins
and maintainer can proceed with the merge using submit button.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Tests failing on Fedora

[Raghavendra Talur]

Here is the list of tests that fail on Fedora 23 machine for our master
branch.
Considering that most of them pass on upstream CentOS regression machines,
it indicates that our tests are not portable enough.

Test Summary Report
---
tests/basic/afr/sparse-file-self-heal.t
(Wstat: 0 Tests: 68 Failed: 2)
  Failed tests:  36, 68
tests/basic/afr/split-brain-healing.t
(Wstat: 0 Tests: 56 Failed: 9)
  Failed tests:  18-26
tests/basic/mount-nfs-auth.t
 (Wstat: 0 Tests: 89 Failed: 19)
  Failed tests:  16, 19-20, 24-25, 28, 31, 34-35, 38, 44-45
47, 51-53, 55, 60-61
tests/basic/rpm.t
(Wstat: 0 Tests: 7 Failed: 0)
  Parse errors: Bad plan.  You planned 8 tests but ran 7.
tests/basic/tier/bug-1214222-directories_missing_after_attach_tier.t
 (Wstat: 0 Tests: 19 Failed: 2)
  Failed tests:  17-18
tests/basic/tier/record-metadata-heat.t
(Wstat: 0 Tests: 18 Failed: 1)
  Failed test:  16
tests/basic/tier/tier.t
(Wstat: 0 Tests: 46 Failed: 1)
  Failed test:  44
tests/bugs/access-control/bug-958691.t
 (Wstat: 0 Tests: 24 Failed: 2)
  Failed tests:  12, 19
tests/bugs/glusterd/bug-1260185-donot-allow-detach-commit-unnecessarily.t
(Wstat: 0 Tests: 10 Failed: 1)
  Failed test:  10
tests/bugs/glusterfs-server/bug-904300.t
 (Wstat: 0 Tests: 26 Failed: 4)
  Failed tests:  11-12, 23-24
tests/bugs/quota/afr-quota-xattr-mdata-heal.t
(Wstat: 0 Tests: 104 Failed: 26)
  Failed tests:  39-40, 43-46, 48-49, 52-55, 63, 71-72, 75-78
80-81, 84-87, 93
tests/bugs/quota/bug-1049323.t
 (Wstat: 0 Tests: 14 Failed: 1)
  Failed test:  12
tests/bugs/quota/bug-1240991.t
 (Wstat: 0 Tests: 19 Failed: 2)
  Failed tests:  15-16
tests/bugs/replicate/bug-921231.t
(Wstat: 0 Tests: 11 Failed: 1)
  Failed test:  11
tests/bugs/rpc/bug-921072.t
(Wstat: 0 Tests: 67 Failed: 16)
  Failed tests:  10-11, 21-22, 26-27, 30-31, 42-43, 48, 50
62-65
tests/features/ipc.t
 (Wstat: 0 Tests: 6 Failed: 1)
  Failed test:  6
tests/features/ssl-ciphers.t
 (Wstat: 0 Tests: 78 Failed: 9)
  Failed tests:  20, 24-25, 29-30, 37, 43, 51, 63
tests/geo-rep/georep-basic-dr-rsync.t
(Wstat: 0 Tests: 58 Failed: 36)
  Failed tests:  14-20, 22-25, 28-30, 32-40, 43-44, 46-52
55-58
tests/geo-rep/georep-basic-dr-tarssh.t
 (Wstat: 0 Tests: 59 Failed: 37)
  Failed tests:  14-21, 23-26, 29-31, 33-41, 44-45, 47-53
56-59
Files=459, Tests=12592, 10705 wallclock secs ( 5.75 usr  1.11 sys + 435.83
cusr 387.48 csys = 830.17 CPU)
Result: FAIL



We have a patch for one such test here http://review.gluster.org/#/c/13228/.
I request developers to pick tests from their modules and debug the issues.
It may very well be the case that the test passes in a Fedora machine
configured in a particular way, if that is the case it would be good to
document it.

Thanks,
Raghavendra Talur
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gerrit review, submit type and Jenkins testing

[Raghavendra Talur]

On Jan 12, 2016 2:50 AM, "Niels de Vos" <nde...@redhat.com> wrote:
>
> On Fri, Jan 08, 2016 at 12:46:09PM +0530, Raghavendra Talur wrote:
> > On Fri, Jan 8, 2016 at 12:28 PM, Kaushal M <kshlms...@gmail.com> wrote:
> >
> > > On Fri, Jan 8, 2016 at 12:10 PM, Kaushal M <kshlms...@gmail.com>
wrote:
> > > > On Fri, Jan 8, 2016 at 12:03 PM, Raghavendra Talur <
rta...@redhat.com>
> > > wrote:
> > > >> Top posting, this is a very old thread.
> > > >>
> > > >> Keeping in view the recent NetBSD problems and the number of bugs
> > > creeping
> > > >> in, I suggest we do these things right now:
> > > >>
> > > >> a. Change the gerrit merge type to fast forward only.
> > > >> As explained below in the thread, with our current setup even if
both
> > > PatchA
> > > >> and PatchB pass regression separately when both are merged it is
> > > possible
> > > >> that a functional bug creeps in.
> > > >> This is the only solution to prevent that from happening.
> > > >> I will work with Kaushal to get this done.
> > > >>
> > > >> b. In Jenkins, remove gerrit trigger and make it a manual operation
> > > >
> > > > Making it manual might be too much work for maintainers. I suggest
(as
> > > > I've suggested before) we make regressions trigger when a change has
> > > > been reviewed +2 by a maintainer.
> > > >
> > >
> >
> > Makes sense. I have disabled it completely for now and lets keep it that
> > way till
> > developers realize it(a day should be enough). We will change this
trigger
> > to on Code Review +2 by tomorrow.
>
> Ah! And I have been wondering why patches don't get verified
> anymore :-/
>
> An email to the maintainers list as a heads up would have been welcome.
>
> How would we handle patches that get sent by maintainers? Most
> developers that do code reviews will only +1 those changes. Those will
> never get automatically regression tested then. I dont think a
> maintainer should +2 their own patch immediately either, that suggests
> no further reviews are needed.
>
> Niels

After realising this we configured Jenkins to be triggered either on code
review +2 or a verified +1. Even if it is the maintainer who sent the
patch, he/she can certainly give a +1 verified.

There seems to be some problem with both these type of events though. I
tried various combinations yesterday,  yet the events don't reach Jenkins.
I am afraid we will have to go back to patch set triggers until we update
our plugins.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel