Re: [Gluster-devel] Sharding - what next?

[Lindsay Mathieson]

On 16/12/15 22:59, Krutika Dhananjay wrote:
I guess I did not make myself clear. Apologies. I meant to say that 
printing a single list of counts aggregated
from all bricks can be tricky and is susceptible to the possibility of 
same entry getting counted multiple times
if the inode needs a heal on multiple bricks. Eliminating such 
duplicates would be rather difficult.


Or, we could have a sub-command of heal-info dump all the file 
paths/gfids that need heal from all bricks and
you could pipe the output to 'sort | uniq | wc -l' to eliminate 
duplicates. Would that be OK? :)



Sorry, my fault - I did understand that. Aggregate counts per brick 
would be fine, I have no desire to complicate things for the devs :)




--
Lindsay Mathieson

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GlusterD 2.0 ReST API - Call for review

[Atin Mukherjee]

We plan to freeze the review gate by end of this month. Appreciate if
you can take a look at it and log your comments if any.

Thanks,
Atin

On 11/17/2015 12:07 PM, Atin Mukherjee wrote:
> A gentle reminder for your review comments!!
> 
> ~Atin
> 
> On 11/09/2015 10:17 AM, Atin Mukherjee wrote:
>> As communicated earlier [1], here is the first cut of the ReST API
>> document ready for the review [2]. The current set of document covers
>> the basic volume and peer commands like volume
>> create/start/stop/delete/status & peer probe/detach/status.
>>
>> Your comments are more than welcome and we expect your contribution in
>> reviewing these APIs in early stage just that we can finalize on the
>> APIs part. We'd need inputs from each feature owners/users to make the
>> APIs adhere to the existing semantics in terms of your feature and also
>> if any improvement can be made on.
>>
>> For review comments, please use [3]
>>
>> [1] http://www.gluster.org/pipermail/gluster-devel/2015-October/046865.html
>> [2] https://github.com/gluster/glusterd2/wiki/ReST-API
>> [3] https://github.com/gluster/glusterd2/issues/27
>>
>> Thanks,
>> Atin
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] REMINDER: Weekly gluster community meeting to start in ~10 minutes

[Niels de Vos]

On Wed, Dec 16, 2015 at 12:53:12PM +0100, Niels de Vos wrote:
> Hi All,
> 
> The weekly Gluster community meeting will start in ~90 minutes.

Note, the subject was correct. The meeting starts in a few minutes.

Sorry for the confusion,
Niels


> 
> Meeting details:
> - location: #gluster-meeting on Freenode IRC
> - date: every Wednesday
> - time: 12:00 UTC, 14:00 CEST, 17:30 IST
> (in your terminal, run: date -d "12:00 UTC")
> - agenda: https://public.pad.fsfe.org/p/gluster-community-meetings
> 
> Today's agenda includes,
> - Roll Call
> - Status of last week's action items
> - Gluster 3.7
> - Gluster 3.8
> - Gluster 3.6
> - Gluster 3.5
> - Gluster 4.0
> - Open Floor
>   - bring your own topic! Add your own topic to the agenda.
> 
> Thanks,
> Niels




signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Sharding - what next?

[Krutika Dhananjay]

- Original Message -

> From: "Lindsay Mathieson" 
> To: "Krutika Dhananjay" 
> Cc: "Gluster Devel" , "gluster-users"
> 
> Sent: Wednesday, December 16, 2015 6:56:03 AM
> Subject: Re: Sharding - what next?

> Hi, late reply again ...

> On 10/12/2015 5:33 PM, Krutika Dhananjay wrote:

> > There is a 'heal-info summary' command that is under review, written by
> > Mohammed Ashiq @ http://review.gluster.org/#/c/12154/3 which prints the
> > number of files that are yet to be healed.
> 
> > It could perhaps be enhanced to print files in split-brain and also files
> > which are possibly being healed. Note that these counts are printed per
> > brick.
> 
> > It does not print a single list of counts with aggregated values. Would
> > that
> > be something you would consider useful?
> 

> Very much so, that would be perfect.

> I can get close to this just with the following

> gluster volume heal datastore1 info | grep 'Brick\|Number'

> And if one is feeling fancy or just wants to keep an eye on progress

> watch "gluster volume heal datastore1 info | grep 'Brick\|Number'"

> though of course this runs afoul of the heal info delay.
I guess I did not make myself clear. Apologies. I meant to say that printing a 
single list of counts aggregated 
from all bricks can be tricky and is susceptible to the possibility of same 
entry getting counted multiple times 
if the inode needs a heal on multiple bricks. Eliminating such duplicates would 
be rather difficult. 

Or, we could have a sub-command of heal-info dump all the file paths/gfids that 
need heal from all bricks and 
you could pipe the output to 'sort | uniq | wc -l' to eliminate duplicates. 
Would that be OK? :) 

-Krutika 

> > > Also, it would be great if the heal info command could return faster,
> > > sometimes it takes over a minute.
> > 
> 
> > Yeah, I think part of the problem could be eager-lock feature which is
> > causing the GlusterFS client process to not relinquish the network lock on
> > the file soon enough, causing the heal info utility to be blocked for
> > longer
> > duration.
> 
> > There is an enhancement Anuradha Talur is working on where heal-info would
> > do
> > away with taking locks altogether. Once that is in place, heal-info should
> > return faster.
> 

> Excellent, I look fwd to that. Even if removing the locks results in the
> occasional inaccurate cout, I don't think that would mattter - From my POV
> its an indicator, not a absolute.

> Thanks,
> --
> Lindsay Mathieson
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Recognizing contributors and displaying other useful bits?

[Niels de Vos]

Hi,

Many GUI tools provide an "About" box that displays some information
about the project. Some applications (Wireshark for example) go an extra
step by including a list of all people that contributed patches. That is
quite a nice way for contributors to see the appreciation of their work.

I think it would be really awesome if we could do something similar. We
have our reserved+hidden ".glusterfs" directory where all access is
denied. We could populate this directory with static compiled-in
contents. When accessed from a client, the licenses, versions, authors
and other bits could be displayed. A normal readdir should still not
list the directory at all (no change from current behaviour).

Any thoughts?
Niels


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster testing matrix

[M S Vishwanath Bhat]

More subcategories to add as suggested:

1. RDMA transport ptotocol
2. Openstatck-Manila integration
3. IPv6 support

New test categories which was missed in the last mail

1. Static Code analysis (Coverity/SonarQube)
2. Doc checks (Most likely be manual - All new feature or updates should
have corresponding documentation)


Exit criterion for

a) each patch -
   1. Smoke tests upon submission
   2. Functional/regression after review and before merging

b) Nightly builds
   1. Regression + Functional
   2. Static Code Analysis
   3. performance regression
   4. Basic distro packaging

c) Minor releases (.x releases)
   1. Regression + Functional
   2. performance regression
   3. Full distro packaging sanity tests
   4. Longevity*
   5. Integration**
   6. Doc checks
   7. Upgrade testing

d) Major releases
   1. Regression + Functional
   2. performance regression
   3. Full distro packaging sanity tests
   4. Longevity*
   5. Integration**
   6. Doc checks
   7. Upgrade testing


* We are not sure about longevity in minor releases considering the one
month release schedule, and longevity tests has to run for at least 1-2
weeks.
** Currently integration tests are owned and maintained by the other
projects (like swift/openstack-manila). Should we setup our own integration
tests? Not sure how to qualify integration tests before releasing.

If we have missed anything or If you have any suggestions, please let us
know.


Best Regards,
msvbhat & rtalur



On 25 November 2015 at 17:39, Raghavendra Talur  wrote:

> Hi All,
>
> Here is a table representing the current state of Gluster testing.
>
> * Things in green are categories for which we have some kind of testing in
> place.
> * Things in red are the ones which don't have any tests.
> * Things in yellow are the ones which have no known tests or are not
> managed by Gluster community.
>
>
>
> Test Category/Sub-Category
>
>
>
>
>
> smoke source build + Posix complaince + Dbench
>
>
>
>
> functional tests/basic
>
>
>
>
> regression tests/bugs
>
>
>
>
> performance regression N/A
>
>
>
>
> integration Backends/FS Protocols Consumers/Cloud Environments Tools libgfapi
> bindings OS environment xfs smb qemu gdeploy java firewalld ext4 nfs
> openstack/cinder heketi python ufw btrfs swift openshift/docker/containers
> ruby selinux zfs
> aws
> go apparmor
>
> azure
>
>
>
>
> hadoop
>
>
> update major version upgrades minor version upgrades
>
>
>
> longevity a. memory leaks
> b. log accumulation
>
>
>
>
> distro packaging a. pkg build + smoke
> b. sysvinit/systemd
>
>
>
>
>
>
> I will send separate mails for each of the categories above highlighting
> plans for them.
>
> Use this thread to indicate addition/deletion/changes to this matrix.
> We will put this at gluster.org website once it is final.
>
> Thanks,
> Raghavendra Talur & MSV Bhat
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] REMINDER: Weekly gluster community meeting to start in ~10 minutes

[Niels de Vos]

Hi All,

The weekly Gluster community meeting will start in ~90 minutes.

Meeting details:
- location: #gluster-meeting on Freenode IRC
- date: every Wednesday
- time: 12:00 UTC, 14:00 CEST, 17:30 IST
(in your terminal, run: date -d "12:00 UTC")
- agenda: https://public.pad.fsfe.org/p/gluster-community-meetings

Today's agenda includes,
- Roll Call
- Status of last week's action items
- Gluster 3.7
- Gluster 3.8
- Gluster 3.6
- Gluster 3.5
- Gluster 4.0
- Open Floor
  - bring your own topic! Add your own topic to the agenda.

Thanks,
Niels


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Minutes of todays weekly gluster community meeting

[Niels de Vos]

Thanks to everyone who attended todays meeting. The minutes and full
chat-log can be found here:

Minutes: 
http://meetbot.fedoraproject.org/gluster-meeting/2015-12-16/weekly_gluster_community_meeting.2015-12-16-12.01.html
Minutes (text): 
http://meetbot.fedoraproject.org/gluster-meeting/2015-12-16/weekly_gluster_community_meeting.2015-12-16-12.01.txt
Log: 
http://meetbot.fedoraproject.org/gluster-meeting/2015-12-16/weekly_gluster_community_meeting.2015-12-16-12.01.log.html


I hope to see you and anyone else interested in the Gluster Community
next week, same time (Wednesday 12:00 UTC) and same place
(#gluster-meeting on Freenode).

Cheers,
Niels


  #gluster-meeting: Weekly Gluster Community Meeting

   Meeting started by ndevos at 12:01:05 UTC (full logs).

  Meeting summary

1.   a. Agenda: https://public.pad.fsfe.org/p/gluster-community-meetings 
(ndevos, 12:01:20)
2. Roll Call (ndevos, 12:01:25)
3. Action Items from last week (ndevos, 12:03:37)
4. ndevos to send out a reminder to the maintainers about more actively 
enforcing backports of bugfixes (ndevos, 12:03:41)
5. raghu to call for volunteers and help from maintainers for doing 
backports listed by rwareing to 3.6.8 (ndevos, 12:04:19)
6. rastar and msvbhat to publish a test exit criterion for major/minor 
releases on gluster.org (ndevos, 12:06:14)
 a. 
http://www.gluster.org/pipermail/gluster-devel/2015-December/047423.html 
(ndevos, 12:09:13)
 b. feedback on the "Gluster testing matrix" requested, results will be 
included in glusterweb (ndevos, 12:09:54)
 c. ACTION: msvbhat and rastar to follow up on the results of the 
"Gluster testing matrix" email (ndevos, 12:10:27)
7. kshlm & csim to set up faux/pseudo user email for gerrit, bugzilla, 
github (ndevos, 12:10:44)
8. hagarth to decide on 3.7.7 release manager (ndevos, 12:11:38)
9. amye to get on top of disucssion on long-term releases (ndevos, 12:13:57)
   10. hagarth to post Gluster Monthly News this week (ndevos, 12:18:54)
 a. weekly newsletters, we'll be moving to monthly starting in January 
to be able to highlight what's going on in the project, upcoming events and 
meetups, ways to get involved (ndevos, 12:21:35)
   11. hagarth to create 3.6.8 for bugzilla version (ndevos, 12:29:14)
   12. raghu to ask for volunteers for release manager for 3.6.8 (ndevos, 
12:30:28)
 a. 
http://thread.gmane.org/gmane.comp.file-systems.gluster.maintainers/145 
(ndevos, 12:32:09)
   13. kkeithley to send a mail about using sanity checker tools in the 
codebase (ndevos, 12:32:26)
   14. rastar to continue the discussion on rebase+fast forward as an option to 
gerrit submit type (ndevos, 12:33:44)
 a. rebase+fastforward patch submission in Gerrit depends on the 
regression testing that is needed to be done (ndevos, 12:36:32)
 b. maintainers need to read the "Gluster testing matrix" email and 
reply to the rebase+fastforward notes in there (ndevos, 12:42:00)
 c. ACTION: rtalur/rastar will send a seperate email about the Gerrit 
patch merge strategies to the maintainers list (ndevos, 12:44:50)
   15. GlusterFS 3.7 (ndevos, 12:44:59)
 a. HELP: need a volunteer release manager for 3.7.7 (ndevos, 12:46:45)
   16. GlusterFS 3.6 (ndevos, 12:46:58)
   17. GlusterFS 3.5 (ndevos, 12:51:32)
   18. GlusterFS 3.8 (ndevos, 12:53:07)
 a. https://public.pad.fsfe.org/p/gluster-3.8-features (ndevos, 
12:53:13)
   19. Gluster 4.0 (ndevos, 12:54:05)
   20. Open Floor (ndevos, 12:56:14)
 a. Weekly reminder to announce Gluster attendance of events: 
https://public.pad.fsfe.org/p/gluster-events (ndevos, 12:56:24)
 b. REMINDER to put (even minor) interesting topics on 
https://public.pad.fsfe.org/p/gluster-weekly-news (ndevos, 12:56:37)
 c. Etherpad for backport requests 
https://public.pad.fsfe.org/p/gluster-backport-requests (ndevos, 12:56:51)
 d. Gluster meetup in Bengaluru 
http://www.meetup.com/glusterfs-India/events/227287952/ (ndevos, 12:59:07)

   Meeting ended at 13:00:23 UTC (full logs).

  Action items

1. msvbhat and rastar to follow up on the results of the "Gluster testing 
matrix" email
2. rtalur/rastar will send a seperate email about the Gerrit patch merge 
strategies to the maintainers list

  Action items, by person

1. msvbhat
 1. msvbhat and rastar to follow up on the results of the "Gluster 
testing matrix" email
2. rtalur
 1. rtalur/rastar will send a seperate email about the Gerrit patch 
merge strategies to the maintainers list

  People present (lines said)

1. ndevos (121)
2. amye (16)
3. rtalur (15)
4. msvbhat (13)
5. obnox (10)
6. jiffin (4)
7. Manikandan (4)
8. zodbot (4)
9. raghu (4)
   10. kkeithley (4)
   11. glusterbot (3)
   12. skoduri (2)
   13. atinm (2)
   14. pranithk (2)
   15. sankarshan (2)
   16. anoopcs (1)
   17. hagarth (1)
   18. Saravana_ (1)
   19. sac (1)
   20. 

Re: [Gluster-devel] Recognizing contributors and displaying other useful bits?

[Ravishankar N]

On 12/16/2015 07:36 PM, Niels de Vos wrote:

Hi,

Many GUI tools provide an "About" box that displays some information
about the project. Some applications (Wireshark for example) go an extra
step by including a list of all people that contributed patches. That is
quite a nice way for contributors to see the appreciation of their work.

I think it would be really awesome if we could do something similar. We
have our reserved+hidden ".glusterfs" directory where all access is
denied. We could populate this directory with static compiled-in
contents. When accessed from a client, the licenses, versions, authors
and other bits could be displayed. A normal readdir should still not
list the directory at all (no change from current behaviour).

Any thoughts?


I think It would be better to revive the who-wrote-glusterfs reports 
that Vijay used to send on the mailing list instead of statically adding 
contributor names to code.
We could have a `gluster --credits` or `gluster--help`  cli that points 
to gluster.org and maybe add a button thingy on the website which when 
clicked runs the who-wrote-glusterfs.sh  on   and 
displays the result. Things like version numbers and licenses are/should 
be covered in gluster--version.


But yes, if  we do want to expose this information via the mount point, 
then maybe /mount/.meta is a good place.


My 2 cents.
Ravi




Niels


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



--
> : Any porters out there should feel happier knowing that DEC is 
shipping > : me an AlphaPC that I intend to try getting linux running 
on: this will > : definitely help flush out some of the most flagrant 
unportable stuff. > : The Alpha is much more different from the i386 
than the 68k stuff is, so > : it's likely to get most of the stuff 
fixed. > > It's posts like this that almost convince us non-believers 
that there > really is a god. (A follow-up by 
alov...@kerberos.demon.co.uk, Anthony Lovell, to Linus's remarks about 
porting)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Recognizing contributors and displaying other useful bits?

[Venky Shankar]

Ravishankar N wrote:

On 12/16/2015 07:36 PM, Niels de Vos wrote:

Hi,

Many GUI tools provide an "About" box that displays some information
about the project. Some applications (Wireshark for example) go an extra
step by including a list of all people that contributed patches. That is
quite a nice way for contributors to see the appreciation of their work.

I think it would be really awesome if we could do something similar. We
have our reserved+hidden ".glusterfs" directory where all access is
denied. We could populate this directory with static compiled-in
contents. When accessed from a client, the licenses, versions, authors
and other bits could be displayed. A normal readdir should still not
list the directory at all (no change from current behaviour).

Any thoughts?


I think It would be better to revive the who-wrote-glusterfs reports
that Vijay used to send on the mailing list instead of statically adding
contributor names to code.
We could have a `gluster --credits` or `gluster--help`  cli that points
to gluster.org and maybe add a button thingy on the website which when
clicked runs the who-wrote-glusterfs.sh  on   and
displays the result. Things like version numbers and licenses are/should
be covered in gluster--version.


or a tshirt with contributors list on the back :)



But yes, if  we do want to expose this information via the mount point,
then maybe /mount/.meta is a good place.


Ummm.. makes sense.



My 2 cents.
Ravi




Niels


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel



--
 > : Any porters out there should feel happier knowing that DEC is
shipping > : me an AlphaPC that I intend to try getting linux running
on: this will > : definitely help flush out some of the most flagrant
unportable stuff. > : The Alpha is much more different from the i386
than the 68k stuff is, so > : it's likely to get most of the stuff
fixed. > > It's posts like this that almost convince us non-believers
that there > really is a god. (A follow-up by
alov...@kerberos.demon.co.uk, Anthony Lovell, to Linus's remarks about
porting)

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel