Re: [Gluster-devel] [Gluster-users] Announcing Gluster release 7.4

[Vijay Bellur]

On Mon, Mar 23, 2020 at 2:53 PM Artem Russakovskii 
wrote:

> Typo here:"cashes"
> #1785323: glusterfsd cashes after a few seconds
>


Thank you for catching that. I have opened a pull request [1] to address a
couple of typos in the release notes.

Cheers,
Vijay

[1] https://github.com/gluster/glusterdocs/pull/553
___

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] In-place volume type conversion

[Vijay Bellur]

On Thu, Dec 5, 2019 at 7:37 AM Dmitry Antipov  wrote:

> Is it technically possible/feasible to implement an in-place volume
> conversion,
> at least for volumes with the same number of bricks (say, from 'replica 3'
> to
> 'disperse 3')? If so, any thoughts on initial steps?
>
>
The backend layouts for replicate and disperse are different. Hence it is
not recommended to try out an in-place volume conversion for these types.

Regards,
Vijay
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/441850968


NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/441850968

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Proposal: move glusterfs development to github workflow, completely

[Vijay Bellur]

On Fri, Aug 23, 2019 at 6:12 AM Amar Tumballi  wrote:

> Hi developers,
>
> With this email, I want to understand what is the general feeling around
> this topic.
>
> We from gluster org (in github.com/gluster) have many projects which
> follow complete github workflow, where as there are few, specially the main
> one 'glusterfs', which uses 'Gerrit'.
>
> While this has worked all these years, currently, there is a huge set of
> brain-share on github workflow as many other top projects, and similar
> projects use only github as the place to develop, track and run tests etc.
> As it is possible to have all of the tools required for this project in
> github itself (code, PR, issues, CI/CD, docs), lets look at how we are
> structured today:
>
> Gerrit - glusterfs code + Review system
> Bugzilla - For bugs
> Github - For feature requests
> Trello - (not very much used) for tracking project development.
> CI/CD - CentOS-ci / Jenkins, etc but maintained from different repo.
> Docs - glusterdocs - different repo.
> Metrics - Nothing (other than github itself tracking contributors).
>
> While it may cause a minor glitch for many long time developers who are
> used to the flow, moving to github would bring all these in single place,
> makes getting new users easy, and uniform development practices for all
> gluster org repositories.
>
> As it is just the proposal, I would like to hear people's thought on this,
> and conclude on this another month, so by glusterfs-8 development time, we
> are clear about this.
>
>
+1 to the idea.

While we are at this, any more thoughts about consolidating
IRC/Slack/gitter etc.?

Thanks,
Vijay
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Re-Compile glusterd1 and add it to the stack

[Vijay Bellur]

Hi David,

If the option is related to a particular translator, you would need to add
that option in the options table of the translator and add code in
glusterd-volgen.c to generate that option in the volfiles.

Would it be possible to share the code diff that you are trying out?

Regards,
Vijay

On Wed, Jul 10, 2019 at 3:11 AM David Spisla  wrote:

> Hello Gluster Devels,
>
> I add a custom volume option to glusterd-volume-set.c . I could build my
> own RPMs but I don't want this, I only want to add new compiled glusterd to
> the stack. I tried it out to copy glusterd.so to
> /usr/lib64/glusterfs/x.x/xlator/mgmt . After this glusterd is running
> normally and I can create volumes but in the vol files my new option is not
> there and if I want to start the volume it failed.
>
> It seems to be that I need to add some other files to the stack. Any idea?
>
> Regards
> David Spisla
> ___
>
> Community Meeting Calendar:
>
> APAC Schedule -
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/836554017
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/486278655
>
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Meeting Details on footer of the gluster-devel and gluster-user mailing list

[Vijay Bellur]

On Wed, May 8, 2019 at 12:08 AM Niels de Vos  wrote:

> On Tue, May 07, 2019 at 11:37:27AM -0700, Vijay Bellur wrote:
> > On Tue, May 7, 2019 at 11:15 AM FNU Raghavendra Manjunath <
> rab...@redhat.com>
> > wrote:
> >
> > >
> > > + 1 to this.
> > >
> >
> > I have updated the footer of gluster-devel. If that looks ok, we can
> extend
> > it to gluster-users too.
> >
> > In case of a month with 5 Tuesdays, we can skip the 5th Tuesday and
> always
> > stick to the first 4 Tuesdays of every month. That will help in
> describing
> > the community meeting schedule better. If we want to keep the schedule
> > running on alternate Tuesdays, please let me know and the mailing list
> > footers can be updated accordingly :-).
> >
> >
> > > There is also one more thing. For some reason, the community meeting is
> > > not visible in my calendar (especially NA region). I am not sure if
> anyone
> > > else also facing this issue.
> > >
> >
> > I did face this issue. Realized that we had a meeting today and showed up
> > at the meeting a while later but did not see many participants. Perhaps,
> > the calendar invite has to be made a recurring one.
>
> Maybe a new invite can be sent with the minutes after a meeting has
> finished. This makes it easier for people that recently subscribed to
> the list to add it to their calendar?
>
>
>
That is a good point. I have observed in google groups based mailing lists
that a calendar invite for a recurring event is sent automatically to
people after they subscribe to the list. I don't think mailman has a
similar feature yet.

Thanks,
Vijay
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Meeting Details on footer of the gluster-devel and gluster-user mailing list

[Vijay Bellur]

On Tue, May 7, 2019 at 11:15 AM FNU Raghavendra Manjunath 
wrote:

>
> + 1 to this.
>

I have updated the footer of gluster-devel. If that looks ok, we can extend
it to gluster-users too.

In case of a month with 5 Tuesdays, we can skip the 5th Tuesday and always
stick to the first 4 Tuesdays of every month. That will help in describing
the community meeting schedule better. If we want to keep the schedule
running on alternate Tuesdays, please let me know and the mailing list
footers can be updated accordingly :-).


> There is also one more thing. For some reason, the community meeting is
> not visible in my calendar (especially NA region). I am not sure if anyone
> else also facing this issue.
>

I did face this issue. Realized that we had a meeting today and showed up
at the meeting a while later but did not see many participants. Perhaps,
the calendar invite has to be made a recurring one.

Thanks,
Vijay


>
> Regards,
> Raghavendra
>
> On Tue, May 7, 2019 at 5:19 AM Ashish Pandey  wrote:
>
>> Hi,
>>
>> While we send a mail on gluster-devel or gluster-user mailing list,
>> following content gets auto generated and placed at the end of mail.
>>
>> Gluster-users mailing list
>> gluster-us...@gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>> In the similar way, is it possible to attach meeting schedule and link at 
>> the end of every such mails?
>> Like this -
>>
>> Meeting schedule -
>>
>>
>>- APAC friendly hours
>>   - Tuesday 14th May 2019, 11:30AM IST
>>   - Bridge: https://bluejeans.com/836554017
>>   - NA/EMEA
>>   - Tuesday 7th May 2019, 01:00 PM EDT
>>   - Bridge: https://bluejeans.com/486278655
>>
>> Or just a link to meeting minutes details??
>>  
>> https://github.com/gluster/community/tree/master/meetings
>>
>> This will help developers and users of the community to know when and where 
>> meeting happens and how to attend those meetings.
>>
>> ---
>> Ashish
>>
>>
>>
>>
>>
>>
>> ___
>> Gluster-users mailing list
>> gluster-us...@gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
___

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/836554017

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/486278655

Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Query regarding dictionary logic

[Vijay Bellur]

Hi Mohit,

Thank you for the update. More inline.

On Wed, May 1, 2019 at 11:45 PM Mohit Agrawal  wrote:

> Hi Vijay,
>
> I have tried to execute smallfile tool on volume(12x3), i have not found
> any significant performance improvement
> for smallfile operations, I have configured 4 clients and 8 thread to run
> operations.
>

For measuring performance, did you measure both time taken and cpu
consumed? Normally O(n) computations are cpu expensive and we might see
better results with a hash table when a large number of objects ( a few
thousands) are present in a single dictionary. If you haven't gathered cpu
statistics, please also gather that for comparison.


> I have generated statedump and found below data for dictionaries specific
> to gluster processes
>
> brick
> max-pairs-per-dict=50
> total-pairs-used=192212171
> total-dicts-used=24794349
> average-pairs-per-dict=7
>
>
> glusterd
> max-pairs-per-dict=301
> total-pairs-used=156677
> total-dicts-used=30719
> average-pairs-per-dict=5
>
>
> fuse process
> [dict]
> max-pairs-per-dict=50
> total-pairs-used=88669561
> total-dicts-used=12360543
> average-pairs-per-dict=7
>
> It seems dictionary has max-pairs in case of glusterd and while no. of
> volumes are high the number can be increased.
> I think there is no performance regression in case of brick and fuse. I
> have used hash_size 20 for the dictionary.
> Let me know if you can provide some other test to validate the same.
>

A few more items to try out:

1. Vary the number of buckets and test.
2. Create about 1 volumes and measure performance for a volume info
 operation on some random volume?
3. Check the related patch from Facebook and see if we can incorporate any
ideas from their patch.

Thanks,
Vijay



> Thanks,
> Mohit Agrawal
>
> On Tue, Apr 30, 2019 at 2:29 PM Mohit Agrawal  wrote:
>
>> Thanks, Amar for sharing the patch, I will test and share the result.
>>
>> On Tue, Apr 30, 2019 at 2:23 PM Amar Tumballi Suryanarayan <
>> atumb...@redhat.com> wrote:
>>
>>> Shreyas/Kevin tried to address it some time back using
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1428049 (
>>> https://review.gluster.org/16830)
>>>
>>> I vaguely remember the reason to keep the hash value 1 was done during
>>> the time when we had dictionary itself sent as on wire protocol, and in
>>> most other places, number of entries in dictionary was on an avg, 3. So, we
>>> felt, saving on a bit of memory for optimization was better at that time.
>>>
>>> -Amar
>>>
>>> On Tue, Apr 30, 2019 at 12:02 PM Mohit Agrawal 
>>> wrote:
>>>
>>>> sure Vijay, I will try and update.
>>>>
>>>> Regards,
>>>> Mohit Agrawal
>>>>
>>>> On Tue, Apr 30, 2019 at 11:44 AM Vijay Bellur 
>>>> wrote:
>>>>
>>>>> Hi Mohit,
>>>>>
>>>>> On Mon, Apr 29, 2019 at 7:15 AM Mohit Agrawal 
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>>   I was just looking at the code of dict, I have one query current
>>>>>> dictionary logic.
>>>>>>   I am not able to understand why we use hash_size is 1 for a
>>>>>> dictionary.IMO with the
>>>>>>   hash_size of 1 dictionary always work like a list, not a hash, for
>>>>>> every lookup
>>>>>>   in dictionary complexity is O(n).
>>>>>>
>>>>>>   Before optimizing the code I just want to know what was the exact
>>>>>> reason to define
>>>>>>   hash_size is 1?
>>>>>>
>>>>>
>>>>> This is a good question. I looked up the source in gluster's historic
>>>>> repo [1] and hash_size is 1 even there. So, this could have been the case
>>>>> since the first version of the dictionary code.
>>>>>
>>>>> Would you be able to run some tests with a larger hash_size and share
>>>>> your observations?
>>>>>
>>>>> Thanks,
>>>>> Vijay
>>>>>
>>>>> [1]
>>>>> https://github.com/gluster/historic/blob/master/libglusterfs/src/dict.c
>>>>>
>>>>>
>>>>>
>>>>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Query regarding dictionary logic

[Vijay Bellur]

Hi Mohit,

On Mon, Apr 29, 2019 at 7:15 AM Mohit Agrawal  wrote:

> Hi All,
>
>   I was just looking at the code of dict, I have one query current
> dictionary logic.
>   I am not able to understand why we use hash_size is 1 for a
> dictionary.IMO with the
>   hash_size of 1 dictionary always work like a list, not a hash, for every
> lookup
>   in dictionary complexity is O(n).
>
>   Before optimizing the code I just want to know what was the exact reason
> to define
>   hash_size is 1?
>

This is a good question. I looked up the source in gluster's historic repo
[1] and hash_size is 1 even there. So, this could have been the case since
the first version of the dictionary code.

Would you be able to run some tests with a larger hash_size and share your
observations?

Thanks,
Vijay

[1] https://github.com/gluster/historic/blob/master/libglusterfs/src/dict.c



>
>   Please share your view on the same.
>
> Thanks,
> Mohit Agrawal
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Quick update on glusterd's volume scalability improvements

[Vijay Bellur]

On Fri, Mar 29, 2019 at 6:42 AM Atin Mukherjee  wrote:

> All,
>
> As many of you already know that the design logic with which GlusterD
> (here on to be referred as GD1) was implemented has some fundamental
> scalability bottlenecks at design level, especially around it's way of
> handshaking configuration meta data and replicating them across all the
> peers. While the initial design was adopted with a factor in mind that GD1
> will have to deal with just few tens of nodes/peers and volumes, the
> magnitude of the scaling bottleneck this design can bring in was never
> realized and estimated.
>
> Ever since Gluster has been adopted in container storage land as one of
> the storage backends, the business needs have changed. From tens of
> volumes, the requirements have translated to hundreds and now to thousands.
> We introduced brick multiplexing which had given some relief to have a
> better control on the memory footprint when having many number of
> bricks/volumes hosted in the node, but this wasn't enough. In one of our (I
> represent Red Hat) customer's deployment  we had seen on a 3 nodes cluster,
> whenever the number of volumes go beyond ~1500 and for some reason if one
> of the storage pods get rebooted, the overall time it takes to complete the
> overall handshaking (not only in a factor of n X n peer handshaking but
> also the number of volume iterations, building up the dictionary and
> sending it over the write) consumes a huge time as part of the handshaking
> process, the hard timeout of an rpc request which is 10 minutes gets
> expired and we see cluster going into a state where none of the cli
> commands go through and get stuck.
>
> With such problem being around and more demand of volume scalability, we
> started looking into these areas in GD1 to focus on improving (a) volume
> scalability (b) node scalability. While (b) is a separate topic for some
> other day we're going to focus on more on (a) today.
>
> While looking into this volume scalability problem with a deep dive, we
> realized that most of the bottleneck which was causing the overall delay in
> the friend handshaking and exchanging handshake packets between peers in
> the cluster was iterating over the in-memory data structures of the
> volumes, putting them into the dictionary sequentially. With 2k like
> volumes the function glusterd_add_volumes_to_export_dict () was quite
> costly and most time consuming. From pstack output when glusterd instance
> was restarted in one of the pods, we could always see that control was
> iterating in this function. Based on our testing on a 16 vCPU, 32 GB RAM 3
> nodes cluster, this function itself took almost *7.5 minutes . *The
> bottleneck is primarily because of sequential iteration of volumes,
> sequentially updating the dictionary with lot of (un)necessary keys.
>
> So what we tried out was making this loop to work on a worker thread model
> so that multiple threads can process a range of volume list and not all of
> them so that we can get more parallelism within glusterd. But with that we
> still didn't see any improvement and the primary reason for that was our
> dictionary APIs need locking. So the next idea was to actually make threads
> work on multiple dictionaries and then once all the volumes are iterated
> the subsequent dictionaries to be merged into a single one. Along with
> these changes there are few other improvements done on skipping comparison
> of snapshots if there's no snap available, excluding tiering keys if the
> volume type is not tier. With this enhancement [1] we see the overall time
> it took to complete building up the dictionary from the in-memory structure
> is *2 minutes 18 seconds* which is close*  ~3x* improvement. We firmly
> believe that with this improvement, we should be able to scale up to 2000
> volumes on a 3 node cluster and that'd help our users to get benefited with
> supporting more PVCs/volumes.
>
> Patch [1] is still in testing and might undergo few minor changes. But we
> welcome you for review and comment on it. We plan to get this work
> completed, tested and release in glusterfs-7.
>
> Last but not the least, I'd like to give a shout to Mohit Agrawal (In cc)
> for all the work done on this for last few days. Thank you Mohit!
>
>

This sounds good! Thank you for the update on this work.

Did you ever consider using etcd with GD1 (like as it is used with GD2)?
Having etcd as a backing store for configuration could remove expensive
handshaking as well as persistence of configuration on every node. I am
interested in understanding if you are aware of any drawbacks with that
approach. If there haven't been any thoughts in that direction, it might be
a fun experiment to try.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Following up on the "Github teams/repo cleanup"

[Vijay Bellur]

On Thu, Mar 28, 2019 at 12:59 PM Sankarshan Mukhopadhyay <
sankarshan.mukhopadh...@gmail.com> wrote:

>
>
> On Fri 29 Mar, 2019, 01:04 Vijay Bellur,  wrote:
>
>>
>>
>> On Thu, Mar 28, 2019 at 11:39 AM Sankarshan Mukhopadhyay <
>> sankarshan.mukhopadh...@gmail.com> wrote:
>>
>>> On Thu, Mar 28, 2019 at 11:34 PM John Strunk  wrote:
>>> >
>>> > Thanks for bringing this to the list.
>>> >
>>> > I think this is a good set of guidelines, and we should publicly post
>>> and enforce them once agreement is reached.
>>> > The integrity of the gluster github org is important for the future of
>>> the project.
>>> >
>>>
>>> I agree. And so, I am looking forward to additional
>>> individuals/maintainers agreeing to this so that we can codify it
>>> under the Gluster.org Github org too.
>>>
>>
>>
>> Looks good to me.
>>
>> While at this, I would also like us to think about evolving some
>> guidelines for creating a new repository in the gluster github
>> organization. Right now, a bug report does get a new repository created and
>> I feel that the process could be a bit more involved to ensure that we
>> host projects with the right content in github.
>>
>
> The bug ensures that there is a recorded trail for the request. What might
> be the additional detail required which can emphasize on greater
> involvement? At this point, I don't see many fly-by-night projects. Just
> inactive ones and those too for myriad reasons.
>

I certainly did not intend fly-by-night projects.

Having a tracking mechanism is indeed desirable. Of the 98 projects we host
in github, there are projects without commits, without licenses and some
whose relevance to Gluster is not easily understood. We could evaluate
factors like these before hosting a repo under the gluster umbrella.

Thanks,
Vijay

>
>
>> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Following up on the "Github teams/repo cleanup"

[Vijay Bellur]

On Thu, Mar 28, 2019 at 11:39 AM Sankarshan Mukhopadhyay <
sankarshan.mukhopadh...@gmail.com> wrote:

> On Thu, Mar 28, 2019 at 11:34 PM John Strunk  wrote:
> >
> > Thanks for bringing this to the list.
> >
> > I think this is a good set of guidelines, and we should publicly post
> and enforce them once agreement is reached.
> > The integrity of the gluster github org is important for the future of
> the project.
> >
>
> I agree. And so, I am looking forward to additional
> individuals/maintainers agreeing to this so that we can codify it
> under the Gluster.org Github org too.
>


Looks good to me.

While at this, I would also like us to think about evolving some guidelines
for creating a new repository in the gluster github organization. Right
now, a bug report does get a new repository created and I feel that the
process could be a bit more involved to ensure that we host projects with
the right content in github.

Thanks,
Vijay

>
> >
> > On Wed, Mar 27, 2019 at 10:21 PM Sankarshan Mukhopadhyay <
> sankarshan.mukhopadh...@gmail.com> wrote:
> >>
> >> The one at <
> https://lists.gluster.org/pipermail/gluster-infra/2018-June/004589.html>
> >> I am not sure if the proposal from Michael was agreed to separately
> >> and it was done. Also, do we want to do this periodically?
> >>
> >> Feedback is appreciated.
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] GF_CALLOC to GF_MALLOC conversion - is it safe?

[Vijay Bellur]

On Thu, Mar 21, 2019 at 8:44 AM Yaniv Kaul  wrote:

>
>
> On Thu, Mar 21, 2019 at 5:23 PM Nithya Balachandran 
> wrote:
>
>>
>>
>> On Thu, 21 Mar 2019 at 16:16, Atin Mukherjee  wrote:
>>
>>> All,
>>>
>>> In the last few releases of glusterfs, with stability as a primary theme
>>> of the releases, there has been lots of changes done on the code
>>> optimization with an expectation that such changes will have gluster to
>>> provide better performance. While many of these changes do help, but off
>>> late we have started seeing some diverse effects of them, one especially
>>> being the calloc to malloc conversions. While I do understand that malloc
>>> syscall will eliminate the extra memset bottleneck which calloc bears, but
>>> with recent kernels having in-built strong compiler optimizations I am not
>>> sure whether that makes any significant difference, but as I mentioned
>>> earlier certainly if this isn't done carefully it can potentially introduce
>>> lot of bugs and I'm writing this email to share one of such experiences.
>>>
>>> Sanju & I were having troubles for last two days to figure out why
>>> https://review.gluster.org/#/c/glusterfs/+/22388/ wasn't working in
>>> Sanju's system but it had no problems running the same fix in my gluster
>>> containers. After spending a significant amount of time, what we now
>>> figured out is that a malloc call [1] (which was a calloc earlier) is the
>>> culprit here. As you all can see, in this function we allocate txn_id and
>>> copy the event->txn_id into it through gf_uuid_copy () . But when we were
>>> debugging this step wise through gdb, txn_id wasn't exactly copied with the
>>> exact event->txn_id and it had some junk values which made the
>>> glusterd_clear_txn_opinfo to be invoked with a wrong txn_id later on
>>> resulting the leaks to remain the same which was the original intention of
>>> the fix.
>>>
>>> This was quite painful to debug and we had to spend some time to figure
>>> this out. Considering we have converted many such calls in past, I'd urge
>>> that we review all such conversions and see if there're any side effects to
>>> it. Otherwise we might end up running into many potential memory related
>>> bugs later on. OTOH, going forward I'd request every patch
>>> owners/maintainers to pay some special attention to these conversions and
>>> see they are really beneficial and error free. IMO, general guideline
>>> should be - for bigger buffers, malloc would make better sense but has to
>>> be done carefully, for smaller size, we stick to calloc.
>>>
>>
>>> What do others think about it?
>>>
>>
>> I believe that replacing calloc with malloc everywhere without adequate
>> testing and review is not safe and am against doing so for the following
>> reasons:
>>
>
> No patch should get in without adequate testing and thorough review.
>


There are lots of interesting points to glean in this thread. However, this
particular one caught my attention. How about we introduce a policy that no
patch gets merged unless it is thoroughly tested? The onus would be on the
developer to provide a .t test case to show completeness in the testing of
that patch. If the developer does not or cannot for any reason, we could
have the maintainer run tests and add a note in gerrit explaining the tests
run. This would provide more assurance about the patches being tested
before getting merged. Obviously, patches that fix typos or that cannot
affect any functionality need not be subject to this policy.

As far as review thoroughness is concerned, it might be better to mandate
acks from respective maintainers before merging a patch that affects
several components. More eyeballs that specialize in particular
component(s) will hopefully catch some of these issues during the review
phase.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Network Block device (NBD) on top of glusterfs

[Vijay Bellur]

Hi Xiubo,

On Fri, Mar 22, 2019 at 5:48 PM Xiubo Li  wrote:

> On 2019/3/21 11:29, Xiubo Li wrote:
>
> All,
>
> I am one of the contributor for gluster-block
> [1] project, and also I
> contribute to linux kernel and open-iscsi 
> project.[2]
>
> NBD was around for some time, but in recent time, linux kernel’s Network
> Block Device (NBD) is enhanced and made to work with more devices and also
> the option to integrate with netlink is added. So, I tried to provide a
> glusterfs client based NBD driver recently. Please refer github issue #633
> [3], and good news is I
> have a working code, with most basic things @ nbd-runner project
> [4].
>
>
This is nice. Thank you for your work!


> As mentioned the nbd-runner(NBD proto) will work in the same layer with
> tcmu-runner(iSCSI proto), this is not trying to replace the
> gluster-block/ceph-iscsi-gateway great projects.
>
> It just provides the common library to do the low level stuff, like the
> sysfs/netlink operations and the IOs from the nbd kernel socket, and the
> great tcmu-runner project is doing the sysfs/uio operations and IOs from
> the kernel SCSI/iSCSI.
>
> The nbd-cli tool will work like the iscsi-initiator-utils, and the
> nbd-runner daemon will work like the tcmu-runner daemon, that's all.
>

Do you have thoughts on how nbd-runner currently differs or would differ
from tcmu-runner? It might be useful to document the differences in github
(or elsewhere) so that users can make an informed choice between nbd-runner
& tcmu-runner.

In tcmu-runner for different backend storages, they have separate handlers,
> glfs.c handler for Gluster, rbd.c handler for Ceph, etc. And what the
> handlers here are doing the actual IOs with the backend storage services
> once the IO paths setup are done by ceph-iscsi-gateway/gluster-block
>
> Then we can support all the kind of backend storages, like the
> Gluster/Ceph/Azure... as one separate handler in nbd-runner, which no need
> to care about the NBD low level's stuff updates and changes.
>

Given that the charter for this project is to support multiple backend
storage projects, would not it be better to host the project in the github
repository associated with nbd [5]? Doing it that way could provide a more
neutral (as perceived by users) venue for hosting nbd-runner and help you
in getting more adoption for your work.

Thanks,
Vijay

[5] https://github.com/NetworkBlockDevice/nbd




> Thanks.
>
>
> While this email is about announcing the project, and asking for more
> collaboration, I would also like to discuss more about the placement of the
> project itself. Currently nbd-runner project is expected to be shared by
> our friends at Ceph project too, to provide NBD driver for Ceph. I have
> personally worked with some of them closely while contributing to
> open-iSCSI project, and we would like to take this project to great success.
>
> Now few questions:
>
>1. Can I continue to use http://github.com/gluster/nbd-runner as home
>for this project, even if its shared by other filesystem projects?
>
>
>- I personally am fine with this.
>
>
>1. Should there be a separate organization for this repo?
>
>
>- While it may make sense in future, for now, I am not planning to
>start any new thing?
>
> It would be great if we have some consensus on this soon as nbd-runner is
> a new repository. If there are no concerns, I will continue to contribute
> to the existing repository.
>
> Regards,
> Xiubo Li (@lxbsz)
>
> [1] - https://github.com/gluster/gluster-block
> [2] - https://github.com/open-iscsi
> [3] - https://github.com/gluster/glusterfs/issues/633
> [4] - https://github.com/gluster/nbd-runner
>
> ___
> Gluster-users mailing 
> listGluster-users@gluster.orghttps://lists.gluster.org/mailman/listinfo/gluster-users
>
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Proposal to mark few features as Deprecated / SunSet from Version 5.0

[Vijay Bellur]

I tried this configuration on my local setup and the test passed fine.

Adding the fuse and write-behind maintainers in Gluster to check if they
are aware of any oddities with using mmap & fuse.

Thanks,
Vijay

On Tue, Mar 19, 2019 at 2:21 PM Jim Kinney  wrote:

> Volume Name: home
> Type: Replicate
> Volume ID: 5367adb1-99fc-44c3-98c4-71f7a41e628a
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1 x 2 = 2
> Transport-type: tcp,rdma
> Bricks:
> Brick1: bmidata1:/data/glusterfs/home/brick/brick
> Brick2: bmidata2:/data/glusterfs/home/brick/brick
> Options Reconfigured:
> performance.client-io-threads: off
> storage.build-pgfid: on
> cluster.self-heal-daemon: enable
> performance.readdir-ahead: off
> nfs.disable: off
>
>
> There are 11 other volumes and all are similar.
>
>
> On Tue, 2019-03-19 at 13:59 -0700, Vijay Bellur wrote:
>
> Thank you for the reproducer! Can you please let us know the output of
> `gluster volume info`?
>
> Regards,
> Vijay
>
> On Tue, Mar 19, 2019 at 12:53 PM Jim Kinney  wrote:
>
> This python will fail when writing to a file in a glusterfs fuse mounted
> directory.
>
> import mmap
>
> # write a simple example file
> with open("hello.txt", "wb") as f:
> f.write("Hello Python!\n")
>
> with open("hello.txt", "r+b") as f:
> # memory-map the file, size 0 means whole file
> mm = mmap.mmap(f.fileno(), 0)
> # read content via standard file methods
> print mm.readline()  # prints "Hello Python!"
> # read content via slice notation
> print mm[:5]  # prints "Hello"
> # update content using slice notation;
> # note that new content must have same size
> mm[6:] = " world!\n"
> # ... and read again using standard file methods
> mm.seek(0)
> print mm.readline()  # prints "Hello  world!"
> # close the map
> mm.close()
>
>
>
>
>
>
>
> On Tue, 2019-03-19 at 12:06 -0400, Jim Kinney wrote:
>
> Native mount issue with multiple clients (centos7 glusterfs 3.12).
>
> Seems to hit python 2.7 and 3+. User tries to open file(s) for write on
> long process and system eventually times out.
>
> Switching to NFS stops the error.
>
> No bug notice yet. Too many pans on the fire :-(
>
> On Tue, 2019-03-19 at 18:42 +0530, Amar Tumballi Suryanarayan wrote:
>
> Hi Jim,
>
> On Tue, Mar 19, 2019 at 6:21 PM Jim Kinney  wrote:
>
>
> Issues with glusterfs fuse mounts cause issues with python file open for
> write. We have to use nfs to avoid this.
>
> Really want to see better back-end tools to facilitate cleaning up of
> glusterfs failures. If system is going to use hard linked ID, need a
> mapping of id to file to fix things. That option is now on for all exports.
> It should be the default If a host is down and users delete files by the
> thousands, gluster _never_ catches up. Finding path names for ids across
> even a 40TB mount, much less the 200+TB one, is a slow process. A network
> outage of 2 minutes and one system didn't get the call to recursively
> delete several dozen directories each with several thousand files.
>
>
>
> Are you talking about some issues in geo-replication module or some other
> application using native mount? Happy to take the discussion forward about
> these issues.
>
> Are there any bugs open on this?
>
> Thanks,
> Amar
>
>
>
>
> nfs
> On March 19, 2019 8:09:01 AM EDT, Hans Henrik Happe  wrote:
>
> Hi,
>
> Looking into something else I fell over this proposal. Being a shop that
> are going into "Leaving GlusterFS" mode, I thought I would give my two
> cents.
>
> While being partially an HPC shop with a few Lustre filesystems,  we chose
> GlusterFS for an archiving solution (2-3 PB), because we could find files
> in the underlying ZFS filesystems if GlusterFS went sour.
>
> We have used the access to the underlying files plenty, because of the
> continuous instability of GlusterFS'. Meanwhile, Lustre have been almost
> effortless to run and mainly for that reason we are planning to move away
> from GlusterFS.
>
> Reading this proposal kind of underlined that "Leaving GluserFS" is the
> right thing to do. While I never understood why GlusterFS has been in
> feature crazy mode instead of stabilizing mode, taking away crucial
> features I don't get. With RoCE, RDMA is getting mainstream. Quotas are
> very useful, even though the current implementation are not perfect.
> Tiering also makes so much sense, but, for large files, not on a per-file
> level.
>
> To be honest we only use quotas. We got scared of 

Re: [Gluster-devel] [Gluster-users] Proposal to mark few features as Deprecated / SunSet from Version 5.0

[Vijay Bellur]

Thank you for the reproducer! Can you please let us know the output of
`gluster volume info`?

Regards,
Vijay

On Tue, Mar 19, 2019 at 12:53 PM Jim Kinney  wrote:

> This python will fail when writing to a file in a glusterfs fuse mounted
> directory.
>
> import mmap
>
> # write a simple example file
> with open("hello.txt", "wb") as f:
> f.write("Hello Python!\n")
>
> with open("hello.txt", "r+b") as f:
> # memory-map the file, size 0 means whole file
> mm = mmap.mmap(f.fileno(), 0)
> # read content via standard file methods
> print mm.readline()  # prints "Hello Python!"
> # read content via slice notation
> print mm[:5]  # prints "Hello"
> # update content using slice notation;
> # note that new content must have same size
> mm[6:] = " world!\n"
> # ... and read again using standard file methods
> mm.seek(0)
> print mm.readline()  # prints "Hello  world!"
> # close the map
> mm.close()
>
>
>
>
>
>
>
> On Tue, 2019-03-19 at 12:06 -0400, Jim Kinney wrote:
>
> Native mount issue with multiple clients (centos7 glusterfs 3.12).
>
> Seems to hit python 2.7 and 3+. User tries to open file(s) for write on
> long process and system eventually times out.
>
> Switching to NFS stops the error.
>
> No bug notice yet. Too many pans on the fire :-(
>
> On Tue, 2019-03-19 at 18:42 +0530, Amar Tumballi Suryanarayan wrote:
>
> Hi Jim,
>
> On Tue, Mar 19, 2019 at 6:21 PM Jim Kinney  wrote:
>
>
> Issues with glusterfs fuse mounts cause issues with python file open for
> write. We have to use nfs to avoid this.
>
> Really want to see better back-end tools to facilitate cleaning up of
> glusterfs failures. If system is going to use hard linked ID, need a
> mapping of id to file to fix things. That option is now on for all exports.
> It should be the default If a host is down and users delete files by the
> thousands, gluster _never_ catches up. Finding path names for ids across
> even a 40TB mount, much less the 200+TB one, is a slow process. A network
> outage of 2 minutes and one system didn't get the call to recursively
> delete several dozen directories each with several thousand files.
>
>
>
> Are you talking about some issues in geo-replication module or some other
> application using native mount? Happy to take the discussion forward about
> these issues.
>
> Are there any bugs open on this?
>
> Thanks,
> Amar
>
>
>
>
> nfs
> On March 19, 2019 8:09:01 AM EDT, Hans Henrik Happe  wrote:
>
> Hi,
>
> Looking into something else I fell over this proposal. Being a shop that
> are going into "Leaving GlusterFS" mode, I thought I would give my two
> cents.
>
> While being partially an HPC shop with a few Lustre filesystems,  we chose
> GlusterFS for an archiving solution (2-3 PB), because we could find files
> in the underlying ZFS filesystems if GlusterFS went sour.
>
> We have used the access to the underlying files plenty, because of the
> continuous instability of GlusterFS'. Meanwhile, Lustre have been almost
> effortless to run and mainly for that reason we are planning to move away
> from GlusterFS.
>
> Reading this proposal kind of underlined that "Leaving GluserFS" is the
> right thing to do. While I never understood why GlusterFS has been in
> feature crazy mode instead of stabilizing mode, taking away crucial
> features I don't get. With RoCE, RDMA is getting mainstream. Quotas are
> very useful, even though the current implementation are not perfect.
> Tiering also makes so much sense, but, for large files, not on a per-file
> level.
>
> To be honest we only use quotas. We got scared of trying out new
> performance features that potentially would open up a new back of issues.
>
> Sorry for being such a buzzkill. I really wanted it to be different.
>
> Cheers,
> Hans Henrik
> On 19/07/2018 08.56, Amar Tumballi wrote:
>
>
> * Hi all, Over last 12 years of Gluster, we have developed many features,
> and continue to support most of it till now. But along the way, we have
> figured out better methods of doing things. Also we are not actively
> maintaining some of these features. We are now thinking of cleaning up some
> of these ‘unsupported’ features, and mark them as ‘SunSet’ (i.e., would be
> totally taken out of codebase in following releases) in next upcoming
> release, v5.0. The release notes will provide options for smoothly
> migrating to the supported configurations. If you are using any of these
> features, do let us know, so that we can help you with ‘migration’.. Also,
> we are happy to guide new developers to work on those components which are
> not actively being maintained by current set of developers. List of
> features hitting sunset: ‘cluster/stripe’ translator: This translator was
> developed very early in the evolution of GlusterFS, and addressed one of
> the very common question of Distributed FS, which is “What happens if one
> of my file is bigger than the available brick. Say, I have 2 TB hard drive,
> exported in glusterfs, my file is 

Re: [Gluster-devel] GlusterFS development Environment-Debug

[Vijay Bellur]

Hello Rajib,

On Mon, Mar 18, 2019 at 1:29 PM Rajib Hossen  wrote:

> Hello All,
> I am new to development of GlusterFS and have some experience in
> competitive programming in C. I want to implement systematic code for
> erasure coding in glusterfs. I would like to setup debugging environment. I
> can use gdb or any IDE. So far, I tried with CLion and Visual Studio Code
> but no luck. I also tried gdb. Maybe I am missing something or doing
> it wrong. Could you please let me know how can I setup my development
> environment and play with the code? Thank you very much.
>
>
Welcome to Gluster development!

Can you provide a few more details on the problems you are encountering
with gdb? For working with gdb, please ensure that your glusterfs install
has been built with symbols.

Also, Are you in a position to share your thoughts on implementing
systematic erasure coding in GlusterFS?

Thanks,
Vijay

[1]
http://pl.atyp.us/hekafs.org/index.php/2011/11/translator-101-lesson-4-debugging-a-translator/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Different glusterfs clients's data not consistent.

[Vijay Bellur]

On Mon, Mar 18, 2019 at 1:21 PM 快乐 <994506...@qq.com> wrote:

> Three node： node1， node2, node3
>
> Steps:
>
> 1. gluster volume create volume_test node1:/brick1
> 2. gluster volume set volume_test cluster.server-quorum-ratio 51
> 3. gluster volume set volume_test cluster.server-quorum-type  server
> 4. On node1, mount -t glusterfs node1:/volume_test /mnt.
> 5. On node2, mount -t glusterfs node2:/volume_test /mnt.
> 6. On node1, killall glusterd
> 7. On node2, gluster volume add-brick volume_test node2:/brick2
> 8. On node2. mkdir /mnt/test
> 8. touch /mnt/test/file1 on two nodes.
>
> On node1, found /brick1/file1. But on node2, also found /brick2/file1.
>


Can you please check the output of stat on file1 in both the bricks?
There's a good likelihood that one of them is a link file [1].


>
> I don't want to set cluster.server-quorum-ratio to 100.
> Cound you help me to solve this porblem?
>

If it is a link file, a volume rebalance operation might provide the
behavior you are looking for.

Regards,
Vijay

[1]
http://pl.atyp.us/hekafs.org/index.php/2012/03/glusterfs-algorithms-distribution/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] GlusterFS - 6.0RC - Test days (27th, 28th Feb)

[Vijay Bellur]

On Thu, Mar 7, 2019 at 10:20 AM Atin Mukherjee  wrote:

> I am not sure how BZ 1683815
>  can be a blocker at
> RC. We have a fix ready, but to me it doesn't look like a blocker. Vijay -
> any objections?
>

None from me as it is an existing minor problem across multiple releases
now.

I guess it got added as a blocker for RC as I used the URL in the testing
day announcement email. We can remove it from the blocker list for RC.

Thanks,
Vijay


> Also the bugzilla dependency of all bugs attached to the release-6 is sort
> of messed up. I see most of the times a mainline bug along with its clones
> are attached to the tracker which is unnecessary. This has happened because
> of default clone but I request every bugzilla assignees to spend few
> additional seconds to establish the right dependency.
>
> I have tried to correct few of them and will do the rest by next Monday.
> That’d help us to filter out the unnecessary ones and get to know how many
> actual blockers we have.
>
> On Tue, Mar 5, 2019 at 11:51 PM Shyam Ranganathan 
> wrote:
>
>> On 3/4/19 12:33 PM, Shyam Ranganathan wrote:
>> > On 3/4/19 10:08 AM, Atin Mukherjee wrote:
>> >>
>> >>
>> >> On Mon, 4 Mar 2019 at 20:33, Amar Tumballi Suryanarayan
>> >> mailto:atumb...@redhat.com>> wrote:
>> >>
>> >> Thanks to those who participated.
>> >>
>> >> Update at present:
>> >>
>> >> We found 3 blocker bugs in upgrade scenarios, and hence have marked
>> >> release
>> >> as pending upon them. We will keep these lists updated about
>> progress.
>> >>
>> >>
>> >> I’d like to clarify that upgrade testing is blocked. So just fixing
>> >> these test blocker(s) isn’t enough to call release-6 green. We need to
>> >> continue and finish the rest of the upgrade tests once the respective
>> >> bugs are fixed.
>> >
>> > Based on fixes expected by tomorrow for the upgrade fixes, we will build
>> > an RC1 candidate on Wednesday (6-Mar) (tagging early Wed. Eastern TZ).
>> > This RC can be used for further testing.
>>
>> There have been no backports for the upgrade failures, request folks
>> working on the same to post a list of bugs that need to be fixed, to
>> enable tracking the same. (also, ensure they are marked against the
>> release-6 tracker [1])
>>
>> Also, we need to start writing out the upgrade guide for release-6, any
>> volunteers for the same?
>>
>> Thanks,
>> Shyam
>>
>> [1] Release-6 tracker bug:
>> https://bugzilla.redhat.com/show_bug.cgi?id=glusterfs-6.0
>>
> --
> - Atin (atinm)
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] I/O performance

[Vijay Bellur]

On Tue, Feb 5, 2019 at 10:57 PM Xavi Hernandez 
wrote:

> On Wed, Feb 6, 2019 at 7:00 AM Poornima Gurusiddaiah 
> wrote:
>
>>
>>
>> On Tue, Feb 5, 2019, 10:53 PM Xavi Hernandez > wrote:
>>
>>> On Fri, Feb 1, 2019 at 1:51 PM Xavi Hernandez 
>>> wrote:
>>>
 On Fri, Feb 1, 2019 at 1:25 PM Poornima Gurusiddaiah <
 pguru...@redhat.com> wrote:

> Can the threads be categorised to do certain kinds of fops?
>

 Could be, but creating multiple thread groups for different tasks is
 generally bad because many times you end up with lots of idle threads which
 waste resources and could increase contention. I think we should only
 differentiate threads if it's absolutely necessary.


> Read/write affinitise to certain set of threads, the other metadata
> fops to other set of threads. So we limit the read/write threads and not
> the metadata threads? Also if aio is enabled in the backend the threads
> will not be blocked on disk IO right?
>

 If we don't block the thread but we don't prevent more requests to go
 to the disk, then we'll probably have the same problem. Anyway, I'll try to
 run some tests with AIO to see if anything changes.

>>>
>>> I've run some simple tests with AIO enabled and results are not good. A
>>> simple dd takes >25% more time. Multiple parallel dd take 35% more time to
>>> complete.
>>>
>>
>>
>> Thank you. That is strange! Had few questions, what tests are you running
>> for measuring the io-threads performance(not particularly aoi)? is it dd
>> from multiple clients?
>>
>
> Yes, it's a bit strange. What I see is that many threads from the thread
> pool are active but using very little CPU. I also see an AIO thread for
> each brick, but its CPU usage is not big either. Wait time is always 0 (I
> think this is a side effect of AIO activity). However system load grows
> very high. I've seen around 50, while on the normal test without AIO it's
> stays around 20-25.
>
> Right now I'm running the tests on a single machine (no real network
> communication) using an NVMe disk as storage. I use a single mount point.
> The tests I'm running are these:
>
>- Single dd, 128 GiB, blocks of 1MiB
>- 16 parallel dd, 8 GiB per dd, blocks of 1MiB
>- fio in sequential write mode, direct I/O, blocks of 128k, 16
>threads, 8GiB per file
>- fio in sequential read mode, direct I/O, blocks of 128k, 16 threads,
>8GiB per file
>- fio in random write mode, direct I/O, blocks of 128k, 16 threads,
>8GiB per file
>- fio in random read mode, direct I/O, blocks of 128k, 16 threads,
>8GiB per file
>- smallfile create, 16 threads, 256 files per thread, 32 MiB per file
>(with one brick down, for the following test)
>- self-heal of an entire brick (from the previous smallfile test)
>- pgbench init phase with scale 100
>
> I run all these tests for a replica 3 volume and a disperse 4+2 volume.
>


Are these performance results available somewhere? I am quite curious to
understand the performance gains on NVMe!

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] I/O performance

[Vijay Bellur]

On Thu, Jan 31, 2019 at 11:12 PM Xavi Hernandez 
wrote:

> On Fri, Feb 1, 2019 at 7:54 AM Vijay Bellur  wrote:
>
>>
>>
>> On Thu, Jan 31, 2019 at 10:01 AM Xavi Hernandez 
>> wrote:
>>
>>> Hi,
>>>
>>> I've been doing some tests with the global thread pool [1], and I've
>>> observed one important thing:
>>>
>>> Since this new thread pool has very low contention (apparently), it
>>> exposes other problems when the number of threads grows. What I've seen is
>>> that some workloads use all available threads on bricks to do I/O, causing
>>> avgload to grow rapidly and saturating the machine (or it seems so), which
>>> really makes everything slower. Reducing the maximum number of threads
>>> improves performance actually. Other workloads, though, do little I/O
>>> (probably most is locking or smallfile operations). In this case limiting
>>> the number of threads to a small value causes a performance reduction. To
>>> increase performance we need more threads.
>>>
>>> So this is making me thing that maybe we should implement some sort of
>>> I/O queue with a maximum I/O depth for each brick (or disk if bricks share
>>> same disk). This way we can limit the amount of requests physically
>>> accessing the underlying FS concurrently, without actually limiting the
>>> number of threads that can be doing other things on each brick. I think
>>> this could improve performance.
>>>
>>
>> Perhaps we could throttle both aspects - number of I/O requests per disk
>> and the number of threads too?  That way we will have the ability to behave
>> well when there is bursty I/O to the same disk and when there are multiple
>> concurrent requests to different disks. Do you have a reason to not limit
>> the number of threads?
>>
>
> No, in fact the global thread pool does have a limit for the number of
> threads. I'm not saying to replace the thread limit for I/O depth control,
> I think we need both. I think we need to clearly identify which threads are
> doing I/O and limit them, even if there are more threads available. The
> reason is easy: suppose we have a fixed number of threads. If we have heavy
> load sent in parallel, it's quite possible that all threads get blocked
> doing some I/O. This has two consequences:
>
>1. There are no more threads to execute other things, like sending
>answers to the client, or start processing new incoming requests. So CPU is
>underutilized.
>2. Massive parallel access to a FS actually decreases performance
>
> This means that we can do less work and this work takes more time, which
> is bad.
>
> If we limit the number of threads that can actually be doing FS I/O, it's
> easy to keep FS responsive and we'll still have more threads to do other
> work.
>


Got it, thx.


>
>
>>
>>> Maybe this approach could also be useful in client side, but I think
>>> it's not so critical there.
>>>
>>
>> Agree, rate limiting on the server side would be more appropriate.
>>
>
> Only thing to consider here is that if we limit rate on servers but
> clients can generate more requests without limit, we may require lots of
> memory to track all ongoing requests. Anyway, I think this is not the most
> important thing now, so if we solve the server-side problem, then we can
> check if this is really needed or not (it could happen that client
> applications limit themselves automatically because they will be waiting
> for answers from server before sending more requests, unless the number of
> application running concurrently is really huge).
>

We could enable throttling in the rpc layer to handle a client performing
aggressive I/O.  RPC throttling should be able to handle the scenario
described above.

-Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] I/O performance

[Vijay Bellur]

On Thu, Jan 31, 2019 at 10:01 AM Xavi Hernandez 
wrote:

> Hi,
>
> I've been doing some tests with the global thread pool [1], and I've
> observed one important thing:
>
> Since this new thread pool has very low contention (apparently), it
> exposes other problems when the number of threads grows. What I've seen is
> that some workloads use all available threads on bricks to do I/O, causing
> avgload to grow rapidly and saturating the machine (or it seems so), which
> really makes everything slower. Reducing the maximum number of threads
> improves performance actually. Other workloads, though, do little I/O
> (probably most is locking or smallfile operations). In this case limiting
> the number of threads to a small value causes a performance reduction. To
> increase performance we need more threads.
>
> So this is making me thing that maybe we should implement some sort of I/O
> queue with a maximum I/O depth for each brick (or disk if bricks share same
> disk). This way we can limit the amount of requests physically accessing
> the underlying FS concurrently, without actually limiting the number of
> threads that can be doing other things on each brick. I think this could
> improve performance.
>

Perhaps we could throttle both aspects - number of I/O requests per disk
and the number of threads too?  That way we will have the ability to behave
well when there is bursty I/O to the same disk and when there are multiple
concurrent requests to different disks. Do you have a reason to not limit
the number of threads?


> Maybe this approach could also be useful in client side, but I think it's
> not so critical there.
>

Agree, rate limiting on the server side would be more appropriate.


Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Performance improvements

[Vijay Bellur]

Thank you for the detailed update, Xavi! This looks very interesting.

On Thu, Jan 24, 2019 at 7:50 AM Xavi Hernandez 
wrote:

> Hi all,
>
> I've just updated a patch [1] that implements a new thread pool based on a
> wait-free queue provided by userspace-rcu library. The patch also includes
> an auto scaling mechanism that only keeps running the needed amount of
> threads for the current workload.
>
> This new approach has some advantages:
>
>- It's provided globally inside libglusterfs instead of inside an
>xlator
>
> This makes it possible that fuse thread and epoll threads transfer the
> received request to another thread sooner, wating less CPU and reacting
> sooner to other incoming requests.
>
>
>- Adding jobs to the queue used by the thread pool only requires an
>atomic operation
>
> This makes the producer side of the queue really fast, almost with no
> delay.
>
>
>- Contention is reduced
>
> The producer side has negligible contention thanks to the wait-free
> enqueue operation based on an atomic access. The consumer side requires a
> mutex, but the duration is very small and the scaling mechanism makes sure
> that there are no more threads than needed contending for the mutex.
>
>
> This change disables io-threads, since it replaces part of its
> functionality. However there are two things that could be needed from
> io-threads:
>
>- Prioritization of fops
>
> Currently, io-threads assigns priorities to each fop, so that some fops
> are handled before than others.
>
>
>- Fair distribution of execution slots between clients
>
> Currently, io-threads processes requests from each client in round-robin.
>
>
> These features are not implemented right now. If they are needed, probably
> the best thing to do would be to keep them inside io-threads, but change
> its implementation so that it uses the global threads from the thread pool
> instead of its own threads.
>


These features are indeed useful to have and hence modifying the
implementation of io-threads to provide this behavior would be welcome.



>
>
> These tests have shown that the limiting factor has been the disk in most
> cases, so it's hard to tell if the change has really improved things. There
> is only one clear exception: self-heal on a dispersed volume completes
> 12.7% faster. The utilization of CPU has also dropped drastically:
>
> Old implementation: 12.30 user, 41.78 sys, 43.16 idle,  0.73 wait
>
> New implementation: 4.91 user,  5.52 sys, 81.60 idle,  5.91 wait
>
>
> Now I'm running some more tests on NVMe to try to see the effects of the
> change when disk is not limiting performance. I'll update once I've more
> data.
>
>
Will look forward to these numbers.


Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] Server outage yesterday

[Vijay Bellur]

On Wed, Dec 12, 2018 at 2:20 PM Michael Scherer  wrote:

> Hi,
>
> I just found out that we suffered a outage yesterday night from 22h UTC
> to 23h40 (I was out on PTO after a flight, and supervision couldn't
> send text messages due to network issue).
>
> The root cause is a networking issue, and from what I gathered from irc
> logs yesterday, some kind of networking loop. I am waiting for a
> detailed report from IT to post more information, but situation is
> stable.
>
>
Thank you for the update, Misc!

We got some feedback in social media about d.g.o not being available around
the same time. Your email probably explains that as well.

Best Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] On making ctime generator enabled by default in stack

[Vijay Bellur]

On Sun, Nov 11, 2018 at 8:25 PM Raghavendra Gowdappa 
wrote:

>
>
> On Sun, Nov 11, 2018 at 11:41 PM Vijay Bellur  wrote:
>
>>
>>
>> On Mon, Nov 5, 2018 at 8:31 PM Raghavendra Gowdappa 
>> wrote:
>>
>>>
>>>
>>> On Tue, Nov 6, 2018 at 9:58 AM Vijay Bellur  wrote:
>>>
>>>>
>>>>
>>>> On Mon, Nov 5, 2018 at 7:56 PM Raghavendra Gowdappa <
>>>> rgowd...@redhat.com> wrote:
>>>>
>>>>> All,
>>>>>
>>>>> There is a patch [1] from Kotresh, which makes ctime generator as
>>>>> default in stack. Currently ctime generator is being recommended only for
>>>>> usecases where ctime is important (like for Elasticsearch). However, a
>>>>> reliable (c)(m)time can fix many consistency issues within glusterfs stack
>>>>> too. These are issues with caching layers having stale (meta)data
>>>>> [2][3][4]. Basically just like applications, components within glusterfs
>>>>> stack too need a time to find out which among racing ops (like write, 
>>>>> stat,
>>>>> etc) has latest (meta)data.
>>>>>
>>>>> Also note that a consistent (c)(m)time is not an optional feature, but
>>>>> instead forms the core of the infrastructure. So, I am proposing to merge
>>>>> this patch. If you've any objections, please voice out before Nov 13, 2018
>>>>> (a week from today).
>>>>>
>>>>> As to the existing known issues/limitations with ctime generator, my
>>>>> conversations with Kotresh, revealed following:
>>>>> * Potential performance degradation (we don't yet have data to
>>>>> conclusively prove it, preliminary basic tests from Kotresh didn't 
>>>>> indicate
>>>>> a significant perf drop).
>>>>>
>>>>
>>>> Do we have this data captured somewhere? If not, would it be possible
>>>> to share that data here?
>>>>
>>>
>>> I misquoted Kotresh. He had measured impact of gfid2path and said both
>>> features might've similar impact as major perf cost is related to storing
>>> xattrs on backend fs. I am in the process of getting a fresh set of
>>> numbers. Will post those numbers when available.
>>>
>>>
>>
>> I observe that the patch under discussion has been merged now [1]. A
>> quick search did not yield me any performance data. Do we have the
>> performance numbers posted somewhere?
>>
>
> No. Perf benchmarking is a task pending on me.
>

When can we expect this task to be complete?

In any case, I don't think it is ideal for us to merge a patch without
completing our due diligence on it. How do we want to handle this scenario
since the patch is already merged?

We could:

1. Revert the patch now
2. Review the performance data and revert the patch if performance
characterization indicates a significant dip. It would be preferable to
complete this activity before we branch off for the next release.
3. Think of some other option?

Thanks,
Vijay


>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] On making ctime generator enabled by default in stack

[Vijay Bellur]

On Mon, Nov 5, 2018 at 8:31 PM Raghavendra Gowdappa 
wrote:

>
>
> On Tue, Nov 6, 2018 at 9:58 AM Vijay Bellur  wrote:
>
>>
>>
>> On Mon, Nov 5, 2018 at 7:56 PM Raghavendra Gowdappa 
>> wrote:
>>
>>> All,
>>>
>>> There is a patch [1] from Kotresh, which makes ctime generator as
>>> default in stack. Currently ctime generator is being recommended only for
>>> usecases where ctime is important (like for Elasticsearch). However, a
>>> reliable (c)(m)time can fix many consistency issues within glusterfs stack
>>> too. These are issues with caching layers having stale (meta)data
>>> [2][3][4]. Basically just like applications, components within glusterfs
>>> stack too need a time to find out which among racing ops (like write, stat,
>>> etc) has latest (meta)data.
>>>
>>> Also note that a consistent (c)(m)time is not an optional feature, but
>>> instead forms the core of the infrastructure. So, I am proposing to merge
>>> this patch. If you've any objections, please voice out before Nov 13, 2018
>>> (a week from today).
>>>
>>> As to the existing known issues/limitations with ctime generator, my
>>> conversations with Kotresh, revealed following:
>>> * Potential performance degradation (we don't yet have data to
>>> conclusively prove it, preliminary basic tests from Kotresh didn't indicate
>>> a significant perf drop).
>>>
>>
>> Do we have this data captured somewhere? If not, would it be possible to
>> share that data here?
>>
>
> I misquoted Kotresh. He had measured impact of gfid2path and said both
> features might've similar impact as major perf cost is related to storing
> xattrs on backend fs. I am in the process of getting a fresh set of
> numbers. Will post those numbers when available.
>
>

I observe that the patch under discussion has been merged now [1]. A quick
search did not yield me any performance data. Do we have the performance
numbers posted somewhere?

Thanks,
Vijay

[1] https://review.gluster.org/#/c/glusterfs/+/21060/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Fwd: New Defects reported by Coverity Scan for gluster/glusterfs

[Vijay Bellur]

On Tue, Nov 6, 2018 at 7:53 AM Atin Mukherjee 
wrote:

> new defects introduced in posix xlator.
>
>
The new defect is in the features/locks xlator. I have sent a patch to
address that.

Thanks,
Vijay

-- Forwarded message -
> From: 
> Date: Tue, Nov 6, 2018 at 8:44 PM
> Subject: New Defects reported by Coverity Scan for gluster/glusterfs
>
> Hi,
>
> Please find the latest report on new defect(s) introduced to
> gluster/glusterfs found with Coverity Scan.
>
> 2 new defect(s) introduced to gluster/glusterfs found with Coverity Scan.
> 1 defect(s), reported by Coverity Scan earlier, were marked fixed in the
> recent build analyzed by Coverity Scan.
>
> New defect(s) Reported-by: Coverity Scan
> Showing 2 of 2 defect(s)
>
>
> ** CID 1396581:  Program hangs  (LOCK)
> /xlators/features/locks/src/posix.c: 2952 in pl_metalk()
>
>
>
> 
> *** CID 1396581:  Program hangs  (LOCK)
> /xlators/features/locks/src/posix.c: 2952 in pl_metalk()
> 2946 gf_msg(this->name, GF_LOG_WARNING, EINVAL, 0,
> 2947"More than one meta-lock can not be granted on"
> 2948"the inode");
> 2949 ret = -1;
> 2950 }
> 2951 }
> >>> CID 1396581:  Program hangs  (LOCK)
> >>> "pthread_mutex_lock" locks "pl_inode->mutex" while it is locked.
> 2952 pthread_mutex_lock(&pl_inode->mutex);
> 2953
> 2954 if (ret == -1) {
> 2955 goto out;
> 2956 }
> 2957
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] On making ctime generator enabled by default in stack

[Vijay Bellur]

On Mon, Nov 5, 2018 at 7:56 PM Raghavendra Gowdappa 
wrote:

> All,
>
> There is a patch [1] from Kotresh, which makes ctime generator as default
> in stack. Currently ctime generator is being recommended only for usecases
> where ctime is important (like for Elasticsearch). However, a reliable
> (c)(m)time can fix many consistency issues within glusterfs stack too.
> These are issues with caching layers having stale (meta)data [2][3][4].
> Basically just like applications, components within glusterfs stack too
> need a time to find out which among racing ops (like write, stat, etc) has
> latest (meta)data.
>
> Also note that a consistent (c)(m)time is not an optional feature, but
> instead forms the core of the infrastructure. So, I am proposing to merge
> this patch. If you've any objections, please voice out before Nov 13, 2018
> (a week from today).
>
> As to the existing known issues/limitations with ctime generator, my
> conversations with Kotresh, revealed following:
> * Potential performance degradation (we don't yet have data to
> conclusively prove it, preliminary basic tests from Kotresh didn't indicate
> a significant perf drop).
>

Do we have this data captured somewhere? If not, would it be possible to
share that data here?

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Consolidating Feature Requests in github

[Vijay Bellur]

Hi All,

I am triaging the open RFEs in bugzilla [1]. Since our new(er) workflow
involves managing RFEs as github issues, I am considering migrating
relevant open RFEs from bugzilla to github. Once migrated, a RFE in
bugzilla would be closed with an appropriate comment. I can also update the
external tracker to point to the respective github issue. Once the
migration is done, all our feature requests can be further triaged and
tracked in github.

Any objections to doing this?

Thanks,
Vijay

[1] https://goo.gl/7fsgTs
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] gluster-ansible: status of the project

[Vijay Bellur]

Thank you for this report!

On Fri, Sep 28, 2018 at 4:34 AM Sachidananda URS  wrote:

> Hi,
>
> gluster-ansible project is aimed at automating the deployment and
> maintenance of GlusterFS cluster.
>
> The project can be found at:
>
> * https://github.com/gluster/gluster-ansible
> * https://github.com/gluster/gluster-ansible-infra
> * https://github.com/gluster/gluster-ansible-features
> * https://github.com/gluster/gluster-ansible-maintenance
> * https://github.com/gluster/gluster-ansible-cluster
> * https://github.com/gluster/gluster-ansible-repositories
>

Are there any plans to have these roles in Ansible Galaxy? Wonder if these
roles can also be consolidated into a single repo with the new multi role
support [1] in Galaxy?

Thanks,
Vijay

[1] https://galaxy.ansible.com/docs/contributing/creating_multi.html
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs v4.1.1 issue encountered while executing test case ./tests/features/trash.t

[Vijay Bellur]

Hi Abhay,

trash is not enabled by default and is considered an experimental feature
in Gluster. You can ignore the failure in trash.t while we fix the problem
for your architecture.

I will update through the bug if any other information is necessary from
you.

Thanks,
Vijay

On Thu, Sep 27, 2018 at 11:51 PM Abhay Singh  wrote:

> Hi,
>
> I have reported an issue
> on bugzilla
> regarding the test case failure of  ./tests/features/trash.t for the
> Glusterfs v4.1.1
> Do we have any updates on this?
>
> Please let us know if any additional info is needed from our side.
> The test failure is blocking our work on the package, so could you please
> take it on priority and give us any updates as soon as possible.
>
> Regards,
> Abhay Singh
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] add-brick fails with error about /dev/fuse

[Vijay Bellur]

On Fri, Sep 21, 2018 at 12:44 AM Chaloulos, Klearchos (Nokia - GR/Athens) <
klearchos.chalou...@nokia.com> wrote:

> Hello,
>
> We are using glusterfs version 3.7.14, and have deployed the glusterfs
> servers in containers. We are trying to use the “gluster volume add-brick”
> command to extend a volume, but it fails:
>
> gluster volume add-brick oam replica 3 172.01.01.01:/mnt/bricks/oam force
> volume add-brick: failed: Commit failed on 172.01.01.01. Please check log
> file for details.
>
> The logs show that it fails to mount /dev/fuse:
>
> [mount.c:341:gf_fuse_mount] 0-glusterfs-fuse: cannot open \/dev\/fuse (No
> such file or directory)
> [glusterd-utils.c:11294:glusterd_handle_replicate_brick_ops] 0-management:
> mount command failed.
> [MSGID: 106074] [glusterd-brick-ops.c:2372:glusterd_op_add_brick]
> 0-glusterd: Unable to add bricks
> [MSGID: 106123] [glusterd-mgmt.c:294:gd_mgmt_v3_commit_fn] 0-management:
> Add-brick commit failed.
>
> Questions:
>
>1. Why does it need /dev/fuse in order to add a brick?
>
>
/dev/fuse or the fuse kernel module is needed on a server that contains a
brick as several features in gluster rely on mounting a native(fuse) client
locally on the server.


>1. Is there a way to bypass the need for /dev/fuse in order to add the
>extra brick?
>
>
I don't think there is an easy way at the moment. Making the fuse kernel
module available to the glusterfs installation in the container would be
the preferred route as of now.


>1. Is this related to this issue:
>*https://bugzilla.redhat.com/show_bug.cgi?id=1420027*
> ?
>
>
Yes, this is related to the issue reported in the bug. There is a fair
amount of work needed to avoid dependency of the glusterfs server stack on
a fuse mount.

Best Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Cloudsync with AFR

[Vijay Bellur]

On Wed, Sep 12, 2018 at 6:49 PM Anuradha Talur  wrote:

> Hi,
>
> We recently started testing cloudsync xlator on a replica volume.
> And we have noticed a few issues. We would like some advice on how to
> proceed with them.
>
> 1) As we know, when stubbing a file cloudsync uses mtime of files to
> decide whether a file should be truncated or not.
>
> If the mtime provided as part of the setfattr operation is lesser than the
> current mtime of the file on brick, stubbing isn't completed.
>
> This works fine in a plain distribute volume. But in case of a replica
> volume, the mtime could be different for the files on each of the replica
> brick.
>
>
> During our testing we came across the following scenario for a replica 3
> volume with 3 bricks:
>
> We performed `setfattr -n "trusted.glusterfs.csou.complete" -v m1
> file1` from our gluster mount to stub the files.
> It so happened that on brick1 this operation succeeded and truncated
> file1 as it should have. But on brick2 and brick3, mtime found on file1
> was greater than m1, leading to failure there.
>
> From AFR's perspective this operation failed as a whole because quorum
> could not be met. But on the brick where this setxattr succeeded,
> truncate was already performed. So now we have one of the replica bricks
> out of sync and AFR has no awareness of this. This file needs to be rolled
> back to its state before the
>
> setfattr.
>
> Ideally, it appears that we should add intelligence in AFR to handle this. How
> do you suggest we do that?
>
> The case is also applicable to EC volumes of course.
>
> 2) Given that cloudsync depends on mtime to make the decision of
> truncating, how do we ensure that we don't end up in this situation again?
>

Thank you for your feedback.

At the outset it looks like these problems can be addressed by enabling
consistent attributes feature in posix [1]. Can you please enable that
option and re-test these cases?

Regards,
Vijay

[1] https://review.gluster.org/#/c/19267/8/doc/features/ctime.md
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs v4.1.1 issue encountered while executing test case ./tests/features/trash.t

[Vijay Bellur]

On Mon, Aug 20, 2018 at 5:24 AM Abhay Singh  wrote:

> Hi Vijay,
>
> As per your previous reply, i tried running the test cases using the
> endianess check through the command  lscpu | grep "Big endian". Thankfully,
> the namespace.t test case passed successfully.


This is good to hear. Would you mind submitting a patch through gerrit for
this?


> However, the trash.t test case is still failing with the following error:-
>
> =
> TEST 37 (line 190): 0 rebalance_completed
> ok 37, LINENUM:190
> RESULT 37: 0
> ls: cannot access '/d/backends/patchy3/rebal*': No such file or directory
> basename: missing operand
> Try 'basename --help' for more information.
> =
> TEST 38 (line 196): Y wildcard_exists /d/backends/patchy3/1
> /d/backends/patchy3/a
> ok 38, LINENUM:196
> RESULT 38: 0
> =
> TEST 39 (line 197): Y wildcard_exists
> /d/backends/patchy1/.trashcan/internal_op/*
> not ok 39 Got "N" instead of "Y", LINENUM:197
> RESULT 39: 1
> =
>
> =
> TEST 63 (line 247): Y wildcard_exists
> /d/backends/patchy1/abc/internal_op/rebal*
> not ok 63 Got "N" instead of "Y", LINENUM:247
> RESULT 63: 1
> rm: cannot remove '/mnt/glusterfs/0/abc/internal_op': Operation not
> permitted
> =
>
> Failed 2/66 subtests
>
> Test Summary Report
> ---
> ./tests/features/trash.t (Wstat: 0 Tests: 66 Failed: 2)
>   Failed tests:  39, 63
> Files=1, Tests=66
> Result: FAIL
>
> Although, running these test cases on an xfs backend is still pending.
> Please let me know how do I proceed further or if anything more needs to
> be done.
>
>

As per our IRC chat, it looks like running on an xfs backend does not make
any difference. I will attempt to recreate this problem on a Big Endian
machine.

Jiffin - is there anything else that you would like to suggest for the
failing tests?

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs v4.1.1 issue encountered while executing test case ./tests/basic/namespace.t

[Vijay Bellur]

Hi Abhay,

Would it be possible for you to change the namsepace.t and trash.t (as
discussed in IRC) tests to work with both endian architectures?

Both failures seem to be linked to the issue that SuperFastHash() provides
different values on little & big endian architectures. You could do
something like `lspcu | grep "Little Endian" to determine the endianess and
this value can be used to have the right behavior in tests.

Please feel free to let me know if you need more assistance in fixing both
namespace.t and trash.t tests.

Regards,
Vijay


On Mon, Aug 13, 2018 at 12:26 PM Abhay Singh  wrote:

> Hi,
> I am working on Glusterfs v4.1.1 for Ubuntu 16.04 on big endian
> architecture. After successful build and running the test cases, I
> encountered a test case failure. The test case is:-
> ·./tests/basic/namespace.t
>
> In the test case *./tests/basic/namespace.t*, a NAMESPACE_HASH is
> generated after  calling  a SuperFastHash() function on the corresponding
> folder names. This hash differs on big endian  and little endian
> architectures. Therefore, I have changed the code accordingly. Although
> there is another subtest in this test which fails with the following error:-
> TEST 30 (line 119): Y check_samples CREATE 1268089390 /namespace3/file
> patchy0
> getfattr: /d/backends/patchy0/namespace3/file: No such file or directory
>
> As seen above, the error is occurring because the folder
> /d/backends/patchy0/namespace3/ doesn’t contain “file”. However, I resolved
> this subtest by changing the folder to /d/backends/patchy6/namespace3/
> where “file” is actually present.
> But same is not the case for little endian architectures where the test
> case passes without any changes.
> The type of filesystem /d/backends is “ext4” and there is enough space
> allocated to the directory.
> Therefore, could you please provide me with some more insight as to why is
> this happening?
>
>
> *Thanks and Regards,*
> *Abhay Singh*
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Announcing Gluster for Container Storage (GCS)

[Vijay Bellur]

Hi all,

We would like to let you  know that some of us have started focusing on an
initiative called ‘Gluster for Container Storage’ (in short GCS). As of
now, one can already use Gluster as storage for containers by making use of
different projects available in github repositories associated with gluster
 & Heketi .
The goal of the GCS initiative is to provide an easier integration of these
projects so that they can be consumed together as designed. We are
primarily focused on integration with Kubernetes (k8s) through this
initiative.

Key projects for GCS include:
Glusterd2 (GD2)

Repo: https://github.com/gluster/glusterd2

The challenge we have with current management layer of Gluster (glusterd)
is that it is not designed for a service oriented architecture. Heketi
overcame this limitation and made Gluster consumable in k8s by providing
all the necessary hooks needed for supporting Persistent Volume Claims.

Glusterd2 provides a service oriented architecture for volume & cluster
management. Gd2 also intends to provide many of the Heketi functionalities
needed by Kubernetes natively. Hence we are working on merging Heketi with
gd2 and you can follow more of this action in the issues associated with
the gd2 github repository.
gluster-block

Repo: https://github.com/gluster/gluster-block

This project intends to expose files in a gluster volume as block devices.
Gluster-block enables supporting ReadWriteOnce (RWO) PVCs and the
corresponding workloads in Kubernetes using gluster as the underlying
storage technology.

Gluster-block is intended to be consumed by stateful RWO applications like
databases and k8s infrastructure services like logging, metrics etc.
gluster-block is more preferred than file based Persistent Volumes in K8s
for stateful/transactional workloads as it provides better performance &
consistency guarantees.
anthill / operator

Repo: https://github.com/gluster/anthill

This project aims to add an operator for Gluster in Kubernetes., Since it
is relatively new, there are areas where you can contribute to make the
operator experience better (please refer to the list of issues). This
project intends to make the whole Gluster experience in k8s much smoother
by automatic management of operator tasks like installation, rolling
upgrades etc.
gluster-csi-driver

Repo: http://github.com/gluster/gluster-csi-driver

This project will provide CSI (Container Storage Interface) compliant
drivers for GlusterFS & gluster-block in k8s.
gluster-kubernetes

Repo: https://github.com/gluster/gluster-kubernetes

This project is intended to provide all the required installation and
management steps for getting gluster up and running in k8s.
GlusterFS

Repo: https://github.com/gluster/glusterfs

GlusterFS is the main and the core repository of Gluster. To support
storage in container world, we don’t need all the features of Gluster.
Hence, we would be focusing on a stack which would be absolutely required
in k8s. This would allow us to plan and execute tests well, and also
provide users with a setup which works without too many options to tweak.

Notice that glusterfs default volumes would continue to work as of now, but
the translator stack which is used in GCS will be much leaner and geared to
work optimally in k8s.
Monitoring
Repo: https://github.com/gluster/gluster-prometheus

As k8s ecosystem provides its own native monitoring mechanisms, we intend
to have this project be the placeholder for required monitoring plugins.
The scope of this project is currently WIP and we welcome your inputs to
shape the project.

More details on this can be found at:
https://lists.gluster.org/pipermail/gluster-users/2018-July/034435.html

Gluster-Containers

*Repo: https://github.com/gluster/gluster-containers
This repository provides
container specs / Dockerfiles that can be used with a container runtime
like cri-o & docker.Note that this is not an exhaustive or final list of
projects involved with GCS. We will continue to update the project list
depending on the new requirements and priorities that we discover in this
journey.*

*We welcome you to join this journey by looking up the repositories and
contributing to them. As always, we are happy to hear your thoughts about
this initiative and please stay tuned as we provide periodic updates about
GCS here!Regards,*

*Vijay*

*(on behalf of Gluster maintainers @ Red Hat)*
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster Documentation Hackathon - 7/19 through 7/23

[Vijay Bellur]

Once you are done with your bit for Gluster documentation, please update
your contributions here [2] for better co-ordination.

Thanks,
Vijay

[2] http://bit.ly/gluster-doc-hack-report

On Wed, Jul 18, 2018 at 9:57 AM Vijay Bellur  wrote:

> Hey All,
>
> We are organizing a hackathon to improve our upstream documentation. More
> details about the hackathon can be found at [1].
>
> Please feel free to let us know if you have any questions.
>
> Thanks,
> Amar & Vijay
>
> [1]
> https://docs.google.com/document/d/11LLGA-bwuamPOrKunxojzAEpHEGQxv8VJ68L3aKdPns/edit?usp=sharing
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Gluster Documentation Hackathon - 7/19 through 7/23

[Vijay Bellur]

Hey All,

We are organizing a hackathon to improve our upstream documentation. More
details about the hackathon can be found at [1].

Please feel free to let us know if you have any questions.

Thanks,
Amar & Vijay

[1]
https://docs.google.com/document/d/11LLGA-bwuamPOrKunxojzAEpHEGQxv8VJ68L3aKdPns/edit?usp=sharing
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Re-thinking gluster regression logging

[Vijay Bellur]

On Mon, Jul 2, 2018 at 4:59 AM Nigel Babu  wrote:

> Hello folks,
>
> Deepshikha is working on getting the distributed-regression testing into
> production. This is a good time to discuss how we log our regression. We
> tend to go with the approach of "get as many logs as possible" and then we
> try to make sense of it when it something fails.
>
> In a setup where we distribute the tests to 10 machines, that means
> fetching runs from 10 machines and trying to make sense of it. Granted, the
> number of files will most likely remain the same since a successful test is
> only run once, but a failed test is re-attempted two more times on
> different machines. So we will now have duplicates.
>
> I have a couple of suggestions and I'd like to see what people think.
> 1. We stop doing tar of tars to do the logs and just tar the
> /var/log/glusterfs folder at the end of the run. That will probably achieve
> better compression.
> 2. We could stream the logs to a service like ELK that we host. This means
> that no more tarballs. It also lets us test any logging improvements we
> plan to make for Gluster in one place.
> 2. I've been looking at Citellus[1] to write parsers that help us identify
> critical problems. This could be a way for us to build a repo of parsers
> that can identify common gluster issues.
>
> Perhaps our solution would be a mix of all 2 and 3. Ideally, I'd like us
> to avoid archiving tarballs to debug regression issues in the future.
>
>
>
A combination of 2 and 3 sounds good to me.

If we could dogfood gluster somewhere in this setup (storage backend for
Elastic?), it would be even more awesome! :)

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] compilation failure: rpcsvc.c:1003:9: error: implicit declaration of function 'xdr_sizeof' [-Werror=implicit-function-declaration]

[Vijay Bellur]

On Fri, Jun 29, 2018 at 11:59 AM Yaniv Kaul  wrote:

> Hello,
>
> First time trying to compile from source.
>
> The compilation part of 'make install' fails with:
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 2 has type 'u_int32_t' [-Wformat=]
>  gf_log (GF_RPCSVC, GF_LOG_ERROR, "auth failed on request.
> "
>  ^
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 3 has type 'rpcvers_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 4 has type 'rpcprog_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 5 has type 'rpcvers_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 6 has type 'rpcproc_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 7 has type 'u_int32_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 8 has type 'rpcvers_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 9 has type 'rpcprog_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 10 has type 'rpcvers_t' [-Wformat=]
> rpcsvc.c:528:17: warning: format '%lu' expects argument of type 'long
> unsigned int', but argument 11 has type 'rpcproc_t' [-Wformat=]
> rpcsvc.c: In function 'rpcsvc_callback_build_record':
> rpcsvc.c:1003:9: error: implicit declaration of function 'xdr_sizeof'
> [-Werror=implicit-function-declaration]
>  xdr_size = xdr_sizeof ((xdrproc_t)xdr_callmsg, &request);
>  ^
> cc1: some warnings being treated as errors
> make[4]: *** [rpcsvc.lo] Error 1
>
>
Is this on RHEL/CentOS 7? IIRC, libtirpc on RHEL7 does not have xdr_sizeof.
Using --without-libtirpc with configure might help overcome this problem as
it falls back on glibc for xdr functions.

Thanks,
Vijay


>
>
> I'm on commit b3c2116d99a5c049e4ee0f88f35440258b49496e , but I don't think
> it's related.
>
> The following did get me RPMs:
> ./configure --enable-fusemount --disable-tiering
> make dist
> cd extras/LinuxRPM/
> make glusterrpms
>
> Thanks,
> Y.
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Storing list of dentries of children in parent inode

[Vijay Bellur]

On Wed, Jun 27, 2018 at 10:15 PM Raghavendra Gowdappa 
wrote:

> All,
>
> There is a requirement in write-behind where during readdirp we may have
> to invalidate iatts/stats of some of the children of the directory [1]. For
> lack of better alternatives I added a dentry list to parent inode which
> contains all children that've been linked (through lookup or readdirp on
> directory). I myself am not too comfortable with this solution as it might
> eat up significant memory for large directories.
>
> Thoughts?
>


Reading [2] makes me wonder if write-behind is the appropriate place for
this change. Shouldn't md-cache be made aware of inode generations or
something similar?

Thanks,
Vijay

[2] https://bugzilla.redhat.com/show_bug.cgi?id=1512691#c18



>
> [1] https://review.gluster.org/20413
>
> regards,
> Raghavendra
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Disabling use of anonymous fds in open-behind

[Vijay Bellur]

On Mon, Jun 11, 2018 at 7:44 PM, Raghavendra Gowdappa 
wrote:

> All,
>
> This is an option in open-behind, which lets fuse native mounts to use
> anonymous fds. The reasoning being since anonymous fds are stateless,
> overhead of open is avoided and hence better performance. However, bugs
> filed [1][2] seemed to indicate contrary results.
>
> Also, using anonymous fds affects other xlators which rely on per fd state
> [3].
>
> So, this brings to the point do anonymous-fds actually improve performance
> on native fuse mounts? If not, we can disable them. May be they are useful
> for light weight metadata operations like fstat, but the workload should
> only be limited to them. Note that anonymous fds are used by open-behind by
> only two fops - readv and fstat. But, [1] has shown that they actually
> regress performance for sequential reads.
>


Perhaps a more intelligent open-behind based on size could help? IIRC,
open-behind was originally developed to improve latency for small file
operations. For large files, it is unnecessary and can affect read-ahead
behavior as observed in the referenced bugs. Could we alter the behavior to
disable open-behind for those files which are bigger than a configurable
size threshold?

Thanks,
Vijay


> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1419807
> [2] https://bugzilla.redhat.com/1489513, "read-ahead underperrforms
> expectations"
>   open-behind without patch (MiB/s) with patch (MiB/s)
>   on  132.87133.51
>   off 139.70139.77
>
> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1084508
>
> PS: Anonymous fds are stateless fds, where a client like native fuse mount
> doesn't do an explicit open. Instead, bricks do the open on-demand during
> fops which need an fd (like readv, fstat etc).
>
> regards,
> Raghavendra
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Update: Gerrit review system has one more command now

[Vijay Bellur]

On Mon, May 21, 2018 at 2:29 AM, Amar Tumballi  wrote:

> Hi all,
>
> As a push towards more flexibility to our developers, and options to run
> more tests without too much effort, we are moving towards more and more
> options to trigger tests from Gerrit during reviews.
>
> One such example was 'regression-on-demand-multiplex' tests, where any
> one can ask for a brick-mux regression for a particular patch.
>
> In the same way, in certain cases where developers are making changes, and
> more than 1 tests would be impacted, there was no easy way to run all the
> regression, other than sending one patchset with changes to 'run-tests.sh'
> to not fail on failures. This was tedious, and also is not known to many
> new developers. Hence a new command is added to gerrit, where one can
> trigger all the runs (if something is failing), by entering *'run full
> regression'* in a single line at the top of your review comments.
>
> With this, a separate job will be triggered which will run the full
> regression suite with the patch. So, no more requirement to make
> 'run-tests.sh' changes.
>
> More on this at http://bugzilla.redhat.com/1564119
>
>
>

Thank you, Amar! I think it will be quite useful for us.

I am not sure if there's a document that details all possible options &
tricks with gerrit. If there's none, we could add one to our
repository/developer-guide so that new developers find it easy to use these
options.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Proposal to change the version numbers of Gluster project

[Vijay Bellur]

On Wed, Mar 14, 2018 at 9:48 PM, Atin Mukherjee  wrote:

>
>
> On Thu, Mar 15, 2018 at 9:45 AM, Vijay Bellur  wrote:
>
>>
>>
>> On Wed, Mar 14, 2018 at 5:40 PM, Shyam Ranganathan 
>> wrote:
>>
>>> On 03/14/2018 07:04 PM, Joe Julian wrote:
>>> >
>>> >
>>> > On 03/14/2018 02:25 PM, Vijay Bellur wrote:
>>> >>
>>> >>
>>> >> On Tue, Mar 13, 2018 at 4:25 AM, Kaleb S. KEITHLEY
>>> >> mailto:kkeit...@redhat.com>> wrote:
>>> >>
>>> >> On 03/12/2018 02:32 PM, Shyam Ranganathan wrote:
>>> >> > On 03/12/2018 10:34 AM, Atin Mukherjee wrote:
>>> >> >>   *
>>> >> >>
>>> >> >> After 4.1, we want to move to either continuous
>>> >> numbering (like
>>> >> >> Fedora), or time based (like ubuntu etc) release
>>> >> numbers. Which
>>> >> >> is the model we pick is not yet finalized. Happy to
>>> >> hear opinions.
>>> >> >>
>>> >> >>
>>> >> >> Not sure how the time based release numbers would make more
>>> >> sense than
>>> >> >> the one which Fedora follows. But before I comment further on
>>> >> this I
>>> >> >> need to first get a clarity on how the op-versions will be
>>> >> managed. I'm
>>> >> >> assuming once we're at GlusterFS 4.1, post that the releases
>>> >> will be
>>> >> >> numbered as GlusterFS5, GlusterFS6 ... So from that
>>> >> perspective, are we
>>> >> >> going to stick to our current numbering scheme of op-version
>>> >> where for
>>> >> >> GlusterFS5 the op-version will be 5?
>>> >> >
>>> >> > Say, yes.
>>> >> >
>>> >> > The question is why tie the op-version to the release number?
>>> That
>>> >> > mental model needs to break IMO.
>>> >> >
>>> >> > With current options like,
>>> >> > https://docs.gluster.org/en/latest/Upgrade-Guide/op_version/
>>> >> <https://docs.gluster.org/en/latest/Upgrade-Guide/op_version/>
>>> it is
>>> >> > easier to determine the op-version of the cluster and what it
>>> >> should be,
>>> >> > and hence this need not be tied to the gluster release version.
>>> >> >
>>> >> > Thoughts?
>>> >>
>>> >> I'm okay with that, but——
>>> >>
>>> >> Just to play the Devil's Advocate, having an op-version that bears
>>> >> some
>>> >> resemblance to the _version_ number may make it easy/easier to
>>> >> determine
>>> >> what the op-version ought to be.
>>> >>
>>> >> We aren't going to run out of numbers, so there's no reason to be
>>> >> "efficient" here. Let's try to make it easy. (Easy to not make a
>>> >> mistake.)
>>> >>
>>> >> My 2¢
>>> >>
>>> >>
>>> >> +1 to the overall release cadence change proposal and what Kaleb
>>> >> mentions here.
>>> >>
>>> >> Tying op-versions to release numbers seems like an easier approach
>>> >> than others & one to which we are accustomed to. What are the benefits
>>> >> of breaking this model?
>>> >>
>>> > There is a bit of confusion among the user base when a release happens
>>> > but the op-version doesn't have a commensurate bump. People ask why
>>> they
>>> > can't set the op-version to match the gluster release version they have
>>> > installed. If it was completely disconnected from the release version,
>>> > that might be a great enough mental disconnect that the expectation
>>> > could go away which would actually cause less confusion.
>>>
>>> Above is the reason I state it as well (the breaking of the mental model
>>> around this), why tie it together when it is not totally related. I also
>>> agree that, the notion is present that i

Re: [Gluster-devel] [Gluster-Maintainers] Proposal to change the version numbers of Gluster project

[Vijay Bellur]

On Wed, Mar 14, 2018 at 5:40 PM, Shyam Ranganathan 
wrote:

> On 03/14/2018 07:04 PM, Joe Julian wrote:
> >
> >
> > On 03/14/2018 02:25 PM, Vijay Bellur wrote:
> >>
> >>
> >> On Tue, Mar 13, 2018 at 4:25 AM, Kaleb S. KEITHLEY
> >> mailto:kkeit...@redhat.com>> wrote:
> >>
> >> On 03/12/2018 02:32 PM, Shyam Ranganathan wrote:
> >> > On 03/12/2018 10:34 AM, Atin Mukherjee wrote:
> >> >>   *
> >> >>
> >> >> After 4.1, we want to move to either continuous
> >> numbering (like
> >> >> Fedora), or time based (like ubuntu etc) release
> >> numbers. Which
> >> >> is the model we pick is not yet finalized. Happy to
> >> hear opinions.
> >> >>
> >> >>
> >> >> Not sure how the time based release numbers would make more
> >> sense than
> >> >> the one which Fedora follows. But before I comment further on
> >> this I
> >> >> need to first get a clarity on how the op-versions will be
> >> managed. I'm
> >> >> assuming once we're at GlusterFS 4.1, post that the releases
> >> will be
> >> >> numbered as GlusterFS5, GlusterFS6 ... So from that
> >> perspective, are we
> >> >> going to stick to our current numbering scheme of op-version
> >> where for
> >> >> GlusterFS5 the op-version will be 5?
> >> >
> >> > Say, yes.
> >> >
> >> > The question is why tie the op-version to the release number? That
> >> > mental model needs to break IMO.
> >> >
> >> > With current options like,
> >> > https://docs.gluster.org/en/latest/Upgrade-Guide/op_version/
> >> <https://docs.gluster.org/en/latest/Upgrade-Guide/op_version/> it
> is
> >> > easier to determine the op-version of the cluster and what it
> >> should be,
> >> > and hence this need not be tied to the gluster release version.
> >> >
> >> > Thoughts?
> >>
> >> I'm okay with that, but——
> >>
> >> Just to play the Devil's Advocate, having an op-version that bears
> >> some
> >> resemblance to the _version_ number may make it easy/easier to
> >> determine
> >> what the op-version ought to be.
> >>
> >> We aren't going to run out of numbers, so there's no reason to be
> >> "efficient" here. Let's try to make it easy. (Easy to not make a
> >> mistake.)
> >>
> >> My 2¢
> >>
> >>
> >> +1 to the overall release cadence change proposal and what Kaleb
> >> mentions here.
> >>
> >> Tying op-versions to release numbers seems like an easier approach
> >> than others & one to which we are accustomed to. What are the benefits
> >> of breaking this model?
> >>
> > There is a bit of confusion among the user base when a release happens
> > but the op-version doesn't have a commensurate bump. People ask why they
> > can't set the op-version to match the gluster release version they have
> > installed. If it was completely disconnected from the release version,
> > that might be a great enough mental disconnect that the expectation
> > could go away which would actually cause less confusion.
>
> Above is the reason I state it as well (the breaking of the mental model
> around this), why tie it together when it is not totally related. I also
> agree that, the notion is present that it is tied together and hence
> related, but it may serve us better to break it.
>
>

I see your perspective. Another related reason for not introducing an
op-version bump in a new release would be that there are no incompatible
features introduced (in the new release). Hence it makes sense to preserve
the older op-version.

To make everyone's lives simpler, would it be useful to introduce a command
that provides the max op-version to release number mapping? The output of
the command could look like:

op-version X: 3.7.0 to 3.7.11
op-version Y: 3.7.12 to x.y.z

and so on.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Proposal to change the version numbers of Gluster project

[Vijay Bellur]

On Tue, Mar 13, 2018 at 4:25 AM, Kaleb S. KEITHLEY 
wrote:

> On 03/12/2018 02:32 PM, Shyam Ranganathan wrote:
> > On 03/12/2018 10:34 AM, Atin Mukherjee wrote:
> >>   *
> >>
> >> After 4.1, we want to move to either continuous numbering (like
> >> Fedora), or time based (like ubuntu etc) release numbers. Which
> >> is the model we pick is not yet finalized. Happy to hear
> opinions.
> >>
> >>
> >> Not sure how the time based release numbers would make more sense than
> >> the one which Fedora follows. But before I comment further on this I
> >> need to first get a clarity on how the op-versions will be managed. I'm
> >> assuming once we're at GlusterFS 4.1, post that the releases will be
> >> numbered as GlusterFS5, GlusterFS6 ... So from that perspective, are we
> >> going to stick to our current numbering scheme of op-version where for
> >> GlusterFS5 the op-version will be 5?
> >
> > Say, yes.
> >
> > The question is why tie the op-version to the release number? That
> > mental model needs to break IMO.
> >
> > With current options like,
> > https://docs.gluster.org/en/latest/Upgrade-Guide/op_version/ it is
> > easier to determine the op-version of the cluster and what it should be,
> > and hence this need not be tied to the gluster release version.
> >
> > Thoughts?
>
> I'm okay with that, but——
>
> Just to play the Devil's Advocate, having an op-version that bears some
> resemblance to the _version_ number may make it easy/easier to determine
> what the op-version ought to be.
>
> We aren't going to run out of numbers, so there's no reason to be
> "efficient" here. Let's try to make it easy. (Easy to not make a mistake.)
>
> My 2¢
>
>
+1 to the overall release cadence change proposal and what Kaleb mentions
here.

Tying op-versions to release numbers seems like an easier approach than
others & one to which we are accustomed to. What are the benefits of
breaking this model?

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] gNFS service management from glusterd

[Vijay Bellur]

On Fri, Feb 23, 2018 at 1:04 PM, Niels de Vos  wrote:

> On Wed, Feb 21, 2018 at 08:25:21PM +0530, Atin Mukherjee wrote:
> > On Wed, Feb 21, 2018 at 4:24 PM, Xavi Hernandez 
> wrote:
> >
> > > Hi all,
> > >
> > > currently glusterd sends a SIGKILL to stop gNFS, while all other
> services
> > > are stopped with a SIGTERM signal first (this can be seen in
> > > glusterd_svc_stop() function of mgmt/glusterd xlator).
> > >
> >
> > > The question is why it cannot be stopped with SIGTERM as all other
> > > services. Using SIGKILL blindly while write I/O is happening can cause
> > > multiple inconsistencies at the same time. For a replicated volume
> this is
> > > not a problem because it will take one of the replicas as the "good"
> one
> > > and continue, but for a disperse volume, if the number of
> inconsistencies
> > > is bigger than the redundancy value, a serious problem could appear.
> > >
> > > The probability of this is very small (I've tried to reproduce this
> > > problem on my laptop but I've been unable), but it exists.
> > >
> > > Is there any known issue that prevents gNFS to be stopped with a
> SIGTERM ?
> > > or can it be changed safely ?
> > >
> >
> > I firmly believe that we need to send SIGTERM as that's the right way to
> > gracefully shutdown a running process but what I'd request from NFS folks
> > to confirm if there's any background on why it was done with SIGKILL.
>
> No background about this is known to me. I had a quick look through the
> git logs, but could not find an explanation.
>
> I agree that SIGTERM would be more appropriate.
>
>

I think there were two reasons for replacing SIGTERM with SIGKILL in gNFS:

1.  To avoid races in the graceful shutdown path that would affect the
restart of gNFS process.

2.  Graceful shutdown of gNFS might have caused clients to return errors to
applications.

Improvements done for gracefully shutting down GlusterFS might have already
addressed 1. I am not entirely certain if 2. was an issue or if it still is
one. If we attempt replacing SIGKILL with SIGTERM, it would be worth
testing out these scenarios carefully.

I also see references to other SIGKILLs in glusterd and other components:

xlators/mgmt/glusterd/src/glusterd-bitd-svc.c:1
xlators/mgmt/glusterd/src/glusterd-geo-rep.c:3
xlators/mgmt/glusterd/src/glusterd-nfs-svc.c:1
xlators/mgmt/glusterd/src/glusterd-proc-mgmt.c:1
xlators/mgmt/glusterd/src/glusterd-quota.c:1
xlators/mgmt/glusterd/src/glusterd-scrub-svc.c:1
xlators/mgmt/glusterd/src/glusterd-svc-helper.c:1
xlators/mgmt/glusterd/src/glusterd-utils.c:2
xlators/nfs/server/src/nlm4.c:1

It might be worth analyzing why we need SIGKILLs and document the reason if
they are indeed necessary.

HTH,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Reducing regression time of glusterd test cases

[Vijay Bellur]

On Sat, Feb 10, 2018 at 8:56 AM, Atin Mukherjee  wrote:

> This is completed and the changes are in mainline. We have managed to
> bring down the regression time for glusterd tests by more than half. A big
> shout to Sanju and Gaurav for taking this to completion.
>


That sounds great! Thank you all for making this happen.

Regards,
Vijay


>
> On Thu, 4 Jan 2018 at 09:57, Sanju Rakonde  wrote:
>
>> Hi all,
>>
>> You can use the below link, if you are accessing the document via your
>> gmail accout.
>>
>> https://docs.google.com/document/d/1u8o4-wocrsuPDI8BwuBU6yi_x4xA_
>> pf2qSrFY6WEQpo/edit?usp=sharing
>>
>> Thanks,
>> Sanju
>>
>> On Fri, Dec 29, 2017 at 6:58 AM, Sanju Rakonde 
>> wrote:
>>
>>> Okay Nigel, I'll share through gmail.
>>>
>>> Thanks,
>>> Sanju
>>>
>>> On 28-Dec-2017 8:56 PM, "Nigel Babu"  wrote:
>>>
 Hey Sanju

 You may want to share this document from your personal Google account.
 It's not visible by anyone outside of Red Hat.

>>>
 On Thu, Dec 28, 2017 at 5:36 PM, Sanju Rakonde 
 wrote:

> Hi All,
>
> To reduce the overall time taken by every regression job for all the
> glusterd tests, we are thinking of optimizing glusterd tests so that some
> of the duplicate tests can be avoided if we can group them based on types
> of volumes, operations to be performed, test to be performed on single 
> node
> or multi node simulated environment.
>
> In the attached document, we have similar test cases which falls under
> the same group followed by suggested test case for that group of test
> cases. Please review the document and give suggestions for any edits.
>
> Please find the doc here
> 
> .
>
> Thanks,
> Sanju
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>



 --
 nigelb

>>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
> --
> - Atin (atinm)
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs and Structured data

[Vijay Bellur]

On Wed, Feb 7, 2018 at 10:35 PM, Raghavendra G 
wrote:

>
>
> On Tue, Feb 6, 2018 at 8:15 PM, Vijay Bellur  wrote:
>
>>
>>
>> On Sun, Feb 4, 2018 at 3:39 AM, Raghavendra Gowdappa > > wrote:
>>
>>> All,
>>>
>>> One of our users pointed out to the documentation that glusterfs is not
>>> good for storing "Structured data" [1], while discussing an issue [2].
>>
>>
>>
>> As far as I remember, the content around structured data in the Install
>> Guide is from a FAQ that was being circulated in Gluster, Inc. indicating
>> the startup's market positioning. Most of that was based on not wanting to
>> get into performance based comparisons of storage systems that are
>> frequently seen in the structured data space.
>>
>>
>>> Does any of you have more context on the feasibility of storing
>>> "structured data" on Glusterfs? Is one of the reasons for such a suggestion
>>> "staleness of metadata" as encountered in bugs like [3]?
>>>
>>
>>
>> There are challenges that distributed storage systems face when exposed
>> to applications that were written for a local filesystem interface. We have
>> encountered problems with applications like tar [4] that are not in the
>> realm of "Structured data". If we look at the common theme across all these
>> problems, it is related to metadata & read after write consistency issues
>> with the default translator stack that gets exposed on the client side.
>> While the default stack is optimal for other scenarios, it does seem that a
>> category of applications needing strict metadata consistency is not well
>> served by that. We have observed that disabling a few performance
>> translators and tuning cache timeouts for VFS/FUSE have helped to overcome
>> some of them. The WIP effort on timestamp consistency across the translator
>> stack, patches that have been merged as a result of the bugs that you
>> mention & other fixes for outstanding issues should certainly help in
>> catering to these workloads better with the file interface.
>>
>> There are deployments that I have come across where glusterfs is used for
>> storing structured data. gluster-block  & qemu-libgfapi overcome the
>> metadata consistency problem by exposing a file as a block device & by
>> disabling most of the performance translators in the default stack.
>> Workloads that have been deemed problematic with the file interface for the
>> reasons alluded above, function well with the block interface.
>>
>
> I agree that gluster-block due to its usage of a subset of glusterfs fops
> (mostly reads/writes I guess), runs into less number of consistency issues.
> However, as you've mentioned we seem to disable perf xlator stack in our
> tests/use-cases till now. Note that perf xlator stack is one of worst
> offenders as far as the metadata consistency is concerned (relatively less
> scenarios of data inconsistency). So, I wonder,
> * what would be the scenario if we enable perf xlator stack for
> gluster-block?
>


tcmu-runner opens block devices with O_DIRECT. So enabling perf xlators for
gluster-block would not make a difference as translators like io-cache &
read-ahead do not enable caching for open() with O_DIRECT. In addition,
since bulk of the operations happen to be reads & writes on large files
with gluster-block, md-cache & quick-read are not appropriate for the stack
that tcmu-runner operates on.


* Is performance on gluster-block satisfactory so that we don't need these
> xlators?
>   - Or is it that these xlators are not useful for the workload usually
> run on gluster-block (For random read/write workload, read/write caching
> xlators offer less or no advantage)?
>   - Or theoretically the workload is ought to benefit from perf xlators,
> but we don't see them in our results (there are open bugs to this effect)?
>


Owing to the reasons mentioned above, most performance xlators do not seem
very useful for gluster-block workloads.


 Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs and Structured data

[Vijay Bellur]

On Sun, Feb 4, 2018 at 3:39 AM, Raghavendra Gowdappa 
wrote:

> All,
>
> One of our users pointed out to the documentation that glusterfs is not
> good for storing "Structured data" [1], while discussing an issue [2].



As far as I remember, the content around structured data in the Install
Guide is from a FAQ that was being circulated in Gluster, Inc. indicating
the startup's market positioning. Most of that was based on not wanting to
get into performance based comparisons of storage systems that are
frequently seen in the structured data space.


> Does any of you have more context on the feasibility of storing
> "structured data" on Glusterfs? Is one of the reasons for such a suggestion
> "staleness of metadata" as encountered in bugs like [3]?
>


There are challenges that distributed storage systems face when exposed to
applications that were written for a local filesystem interface. We have
encountered problems with applications like tar [4] that are not in the
realm of "Structured data". If we look at the common theme across all these
problems, it is related to metadata & read after write consistency issues
with the default translator stack that gets exposed on the client side.
While the default stack is optimal for other scenarios, it does seem that a
category of applications needing strict metadata consistency is not well
served by that. We have observed that disabling a few performance
translators and tuning cache timeouts for VFS/FUSE have helped to overcome
some of them. The WIP effort on timestamp consistency across the translator
stack, patches that have been merged as a result of the bugs that you
mention & other fixes for outstanding issues should certainly help in
catering to these workloads better with the file interface.

There are deployments that I have come across where glusterfs is used for
storing structured data. gluster-block  & qemu-libgfapi overcome the
metadata consistency problem by exposing a file as a block device & by
disabling most of the performance translators in the default stack.
Workloads that have been deemed problematic with the file interface for the
reasons alluded above, function well with the block interface. I feel that
we have come a long way from the time the install guide was written and an
update for removing the "staleness of content" might be in order there :-).

Regards,
Vijay

[4] https://bugzilla.redhat.com/show_bug.cgi?id=1058526


>
> [1] http://docs.gluster.org/en/latest/Install-Guide/Overview/
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1512691
> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1390050
>
> regards,
> Raghavendra
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Release 4.0: Making it happen!

[Vijay Bellur]

On Tue, Jan 16, 2018 at 7:27 AM, Shyam Ranganathan 
wrote:

> On 01/10/2018 01:14 PM, Shyam Ranganathan wrote:
> > Hi,
> >
> > 4.0 branching date is slated on the 16th of Jan 2018 and release is
> > slated for the end of Feb (28th), 2018.
>
> This is today! So read on...
>
> Short update: I am going to wait a couple more days before branching, to
> settle release content and exceptions. Branching is hence on Jan, 18th
> (Thursday).
>
> >
> > We are at the phase when we need to ensure our release scope is correct
> > and *must* release features are landing. Towards this we need the
> > following information for all contributors.
> >
> > 1) Features that are making it to the release by branching date
> >
> > - There are currently 35 open github issues marked as 4.0 milestone [1]
> > - Need contributors to look at this list and let us know which will meet
> > the branching date
>
> Other than the protocol changes (from Amar), I did not receive any
> requests for features that are making it to the release. I have compiled
> a list of features based on patches in gerrit that are open, to check
> what features are viable to make it to 4.0. This can be found here [3].
>
> NOTE: All features, other than the ones in [3] are being moved out of
> the 4.0 milestone.
>
> > - Need contributors to let us know which may slip and hence needs a
> > backport exception to 4.0 branch (post branching).
> > - Need milestone corrections on features that are not making it to the
> > 4.0 release
>
> I need the following contributors to respond and state if the feature in
> [3] should still be tracked against 4.0 and how much time is possibly
> needed to make it happen.
>
> - Poornima, Amar, Jiffin, Du, Susant, Sanoj, Vijay
>
>
I request an extension till end of next week to complete my task.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Integration of GPU with glusterfs

[Vijay Bellur]

On Mon, Jan 15, 2018 at 12:06 AM, Ashish Pandey  wrote:

>
> It is disappointing to see the limitation being put by Nvidia on low cost
> GPU usage on data centers.
> https://www.theregister.co.uk/2018/01/03/nvidia_server_gpus/
>
> We thought of providing an option in glusterfs by which we can control if
> we want to use GPU or not.
> So, the concern of gluster eating out GPU's which could be used by others
> can be addressed.
>


It would be definitely be interesting to try this out. This limitation may
change in the future and the higher end GPUs are still usable in data
centers.

Do you have more details on the proposed implementation?

Thanks,
Vijay



>
> ---
> Ashish
>
>
>
> --
> *From: *"Jim Kinney" 
> *To: *gluster-us...@gluster.org, "Lindsay Mathieson" <
> lindsay.mathie...@gmail.com>, "Darrell Budic" ,
> "Gluster Users" 
> *Cc: *"Gluster Devel" 
> *Sent: *Friday, January 12, 2018 6:00:25 PM
> *Subject: *Re: [Gluster-devel] [Gluster-users] Integration of GPU
> withglusterfs
>
>
>
> On January 11, 2018 10:58:28 PM EST, Lindsay Mathieson <
> lindsay.mathie...@gmail.com> wrote:
> >On 12/01/2018 3:14 AM, Darrell Budic wrote:
> >> It would also add physical resource requirements to future client
> >> deploys, requiring more than 1U for the server (most likely), and I’m
> >
> >> not likely to want to do this if I’m trying to optimize for client
> >> density, especially with the cost of GPUs today.
> >
> >Nvidia has banned their GPU's being used in Data Centers now to, I
> >imagine they are planning to add a licensing fee.
>
> Nvidia banned only the lower cost, home user versions of their GPU line
> from datacenters.
> >
> >--
> >Lindsay Mathieson
> >
> >___
> >Gluster-users mailing list
> >gluster-us...@gluster.org
> >http://lists.gluster.org/mailman/listinfo/gluster-users
>
> --
> Sent from my Android device with K-9 Mail. All tyopes are thumb related
> and reflect authenticity.
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Determine if a file is open in a cluster

[Vijay Bellur]

Hi Ram,

Do you want to check this from within a translator? If so, you can look
for GLUSTERFS_OPEN_FD_COUNT in xlators like dht, afr, ec etc. where they
check for open file descriptors in various FOPs.

Regards,
Vijay

On Thu, Jan 11, 2018 at 10:40 AM, Ram Ankireddypalle 
wrote:

> Hi,
>
>Is it possible to find out within a cluster if a file is currently
> open by any of the clients/self-heal daemon or any other daemon’s within a
> cluster. Please point to the sample code in any of the Xlator which does
> such a check.
>
>
>
> Thanks and Regards,
>
> Ram
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for
> the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 2018 - Plans and Expectations on Gluster Community

[Vijay Bellur]

On Tue, Jan 2, 2018 at 4:09 AM, Kaleb S. KEITHLEY 
wrote:

>
>
>>
>> On Tue, Jan 2, 2018 at 2:36 PM, Hetz Ben Hamo > h...@hetz.biz>> wrote:
>>
>> Hi Amar,
>>
>> If can say something about the development of GlusterFS - is that
>> there are 2 missing things:
>>
>> 1. Breakage between releases. I'm "stuck" using GlusterFS 3.8
>> because someone [removed] support to enable NFS-Ganesha. from the
>> gluster
>> command has been vanished without anything mentioned in the error
>> message what other commands replaces it.
>>
>
> Not true. The nfs-ganesha support is still there in GlusterFS-3.10.
>
> Judging from other people's
>> answers - the answer is storhaug does it from now, but there are
>> practically *0 Documentation* how to use it.
>>
>
> The people who were writing storhaug never finished it. Keep using 3.10
> until storhaug gets finished.
>


Since 3.10 will be EOL in approximately 2 months from now, what would be
our answer for NFS HA if storahug is not finished by then?

 -   Use ctdb
 -   Restore nfs.ganesha CLI support
 -   Something else?

Have we already documented upgrade instructions for those users utilizing
nfs.ganesha CLI in 3.8? If not already done so, it would be useful to have
them listed somewhere.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Release 4.0: Schedule and scope clarity (responses needed)

[Vijay Bellur]

Top posting here.

I would like to propose that all new features for 4.x have tests in glusto
and that one successful run of those tests as a necessary condition for the
release to happen. This effort would help us catch regressions that happen
in multi node setups and also prevent accrual of further technical debt
with respect to testing.

Thoughts?

Thanks,
Vijay

On Mon, Nov 20, 2017 at 1:04 PM, Shyam Ranganathan 
wrote:

> Hi,
>
> As this is a longish mail, there are a few asks below, that I request
> folks to focus and answer.
>
> 4.0 is a STM release (Short Term Maintenance), further, 4.1 is also slated
> as a STM release (although the web pages read differently and will be
> corrected shortly). Finally 4.2 would be the first LTM (Long Term ...) in
> the 4.x release line for Gluster.
>
> * Schedule *
> The above also considers that 4.0 will release 2 months from 3.13, which
> puts 4.0 branching (also read as feature freeze deadline) around
> mid-December (4 weeks from now).
>
> 4.0/1/2 release calendar hence looks as follows,
>
> - Release 4.0: (STM)
>   - Feature freeze/branching: mid-December
>   - Release date: Jan, 31st 2018
> - Release 4.1: (STM)
>   - Feature freeze/branching: mid-March
>   - Release date: Apr, 30th 2018
> - Release 4.2: (LTM, release 3.10 EOL'd)
>   - Feature freeze/branching: mid-June
>   - Release date: Jul, 31st 2018
>
> * Scope *
>
> The main focus in 4.0 is landing GlusterD2, and all efforts towards this
> take priority.
>
> Further big features in 4.0 are around GFProxy, protocol layer changes,
> monitoring and usability changes, FUSE catchup, +1 scaling.
>
> Also, some code cleanup/debt areas are in focus.
>
> Now, glusterfs/github [1] reads ~50 issues as being targets in 4.0, and
> among this about 2-4 are marked closed (or done).
>
> Ask1: Request each of you to go through the issue list and coordinate with
> a maintainer, to either mark an issues milestone correctly (i.e retain it
> in 4.0 or move it out) and also leave a comment on the issue about its
> readiness.
>
> Ask 2: If there are issues that you are working on and are not marked
> against the 4.0 milestone, please do the needful for the same.
>
> Ask 3: Please mail the devel list, on features that are making it to 4.0,
> so that the project board can be rightly populated with the issue.
>
> Ask 4: If the 4.0 branching date was extended by another 4 weeks, would
> that enable you to finish additional features that are already marked for
> 4.0? This helps us move the needle on branching to help land the right set
> of features.
>
> Thanks,
> Shyam
> ___
> maintainers mailing list
> maintain...@gluster.org
> http://lists.gluster.org/mailman/listinfo/maintainers
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Coverity fixes

[Vijay Bellur]

On Fri, Nov 3, 2017 at 9:25 AM, Atin Mukherjee  wrote:

>
> On Fri, 3 Nov 2017 at 18:31, Kaleb S. KEITHLEY 
> wrote:
>
>> On 11/02/2017 10:19 AM, Atin Mukherjee wrote:
>> > While I appreciate the folks to contribute lot of coverity fixes over
>> > last few days, I have an observation for some of the patches the
>> > coverity issue id(s) are *not* mentioned which gets maintainers in a
>> > difficult situation to understand the exact complaint coming out of the
>> > coverity. From my past experience in fixing coverity defects, sometimes
>> > the fixes might look simple but they are not.
>> >
>> > May I request all the developers to include the defect id in the commit
>> > message for all the coverity fixes?
>> >
>>
>> How does that work? AFAIK the defect IDs are constantly changing as some
>> get fixed and new ones get added.
>
>
> We’d need atleast (a) the defect id with pointer to the coverity link
> which most of the devs are now following I guess but with a caveat that
> link goes stale in 7 days and the review needs to be done by that time or
> (b) the commit message should exactly have the coverity description which
> is more neat.
>
> ( I was not knowing the fact the defect id are not constant and later on
> got to know this from Nigel today)
>
>>
>>

+1 to providing a clean description of the issue rather than using a
temporary defect ID.

-Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Client Server incompatibility with Gluster 4.0

[Vijay Bellur]

On Thu, Oct 12, 2017 at 10:22 AM, Shyam Ranganathan 
wrote:

> On 10/12/2017 04:25 AM, Ric Wheeler wrote:
>
>> I worry about having to update all of the clients when we have new code
>> on servers.
>>
>> Typically, for example with NFS, the client negotiates the protocol
>> version it understands and we default to the highest version the clients
>> and servers both support.
>>
>> I know that is a pain, but we should keep in mind what the standard is
>> our users are accustomed to with other protocols
>>
>
> I concur, upgrade to 4.0, if the older clients are not compatible, would
> mean a down client(s) upgrade.
>


Thinking more about it, we could leverage the op-version infrastructure to
maintain compatibility at a cluster/volume level.  This could help by not
requiring the clients to be upgraded simultaneously with the servers.
However, as with current behavior, once a 4.0 feature that is disruptive to
3.x clients is enabled, such clients would not be able to access those
volumes with advanced capabilities.



>
> Further, rolling upgrades of the server becomes a moot point, as some
> would be a higher version in the interim and hence prevent existing clients
> to talk to the same, as our upgrade process is servers first and then
> clients.
>

I do not think we will have rolling upgrades on the server side as we would
need glusterd1 and glusterd2 to interoperate for that. From what I
understand, we will not have that interoperability. A downtime would be
needed for upgrading servers although the expectation is that upgrading the
servers would happen quickly.

Kaushal, Atin - can you please share details on how the upgrade process
would look like from a glusterd perspective?



>
> Which then essentially boils down to, stop services, upgrade everything,
> and restart the volume and the clients.
>
> I understand the additional code and reasoning that Amar has pointed out,
> but taking down time for the upgrade will introduce a lot of pain for the
> users, and hence create resistance to upgrading to 4.0 probable and in some
> cases (consider the service using the gluster volume critical) not possible.
>
> As we are about providing increased availability considering replication
> and the additional storage that it involves, down time for upgrades should
> not be a choice that we should consider, when possible.
>


Agreed.  We can and should reduce the  user pain for this upgrade. Let us
also explore ways to keep our code clean and prevent that from becoming a
spaghetti mess :-).


Regards,
Vijay


>
>> Regards,
>>
>> Ric
>>
>>
>> On Oct 12, 2017 6:30 AM, "Vijay Bellur" > vbel...@redhat.com>> wrote:
>>
>>
>>
>> On Wed, Oct 11, 2017 at 5:06 AM, Amar Tumballi > <mailto:atumb...@redhat.com>> wrote:
>>
>> Was (Re: [Gluster-devel] Proposed Protocol changes for 4.0: Need
>> feedback.)
>>
>> All,
>>
>> While we are at making all the below tasks' color coding to
>> GREEN, it would make sense to discuss 1 main thing.
>>
>> With 4.0, we will anyways say 3.y series server nodes are not
>> going to be compatible with 4.x servers, is it the same case
>> with clients?
>>
>> If yes, I am considering some changes to the current way RPC
>> conversion is handled in protocol layer, and make it simpler a
>> bit.
>>
>> If no, then I have to add lot of 'if..else' in existing code or
>> extra code wherever applicable, now, to make sure we handle the
>> compatibility better.
>>
>> My personal opinion is, talk about incompatibility now, and plan
>> to have smooth sail even when 5.0 lands. We are anyways coming
>> out with GD2 (which makes servers incompatible), and gfproxy
>> (which makes clients missing this feature in older releases),
>> and also possible cherrypicks from upstream fuse project to
>> utilize more features from there, so for the user, there are lot
>> of reason to upgrade the clients.
>>
>>
>>
>> Since we are bumping the major release number, I think it would be
>> acceptable to have 3.x clients being not compatible with 4.x servers
>> and vice-versa. We should ensure that accesses from incompatible
>> clients are handled gracefully by both servers and clients.
>>
>> Regards,
>> Vijay
>>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org <mailto:Gluster-devel@gluster.org>
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>> <http://lists.gluster.org/mailman/listinfo/gluster-devel>
>>
>>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Client Server incompatibility with Gluster 4.0

[Vijay Bellur]

On Wed, Oct 11, 2017 at 5:06 AM, Amar Tumballi  wrote:

> Was (Re: [Gluster-devel] Proposed Protocol changes for 4.0: Need feedback.)
>
> All,
>
> While we are at making all the below tasks' color coding to GREEN, it
> would make sense to discuss 1 main thing.
>
> With 4.0, we will anyways say 3.y series server nodes are not going to be
> compatible with 4.x servers, is it the same case with clients?
>
> If yes, I am considering some changes to the current way RPC conversion is
> handled in protocol layer, and make it simpler a bit.
>
> If no, then I have to add lot of 'if..else' in existing code or extra code
> wherever applicable, now, to make sure we handle the compatibility better.
>
> My personal opinion is, talk about incompatibility now, and plan to have
> smooth sail even when 5.0 lands. We are anyways coming out with GD2 (which
> makes servers incompatible), and gfproxy (which makes clients missing this
> feature in older releases), and also possible cherrypicks from upstream
> fuse project to utilize more features from there, so for the user, there
> are lot of reason to upgrade the clients.
>


Since we are bumping the major release number, I think it would be
acceptable to have 3.x clients being not compatible with 4.x servers and
vice-versa. We should ensure that accesses from incompatible clients are
handled gracefully by both servers and clients.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] gfproxy

[Vijay Bellur]

On Wed, Aug 23, 2017 at 12:41 PM, Poornima Gurusiddaiah  wrote:

> Hi,
>
> This mail is regarding the gfproxy feature, please go through the same and
> let us know your thoughts.
>
> About the gfproxy feature:
> ---
> As per the current architecture of Gluster, the client is more intelligent
> and has all the clustering logic. This approach has its own pros and cons.
> In several use cases, it is desirable to have all this clustering logic on
> the server side and have, as thin client as possible. Eg: Samba, Qemu,
> Block device export etc. This makes the upgrades easier, and is more
> scalable as the resources consumed by thin clients are much less than
> normal client.
>
> Approach:
> Client volfile is split into two volfiles:
> 1. Thin client volfile: master(gfapi/Fuse) followed by Protocol/client
> 2. gfproxyd volfile: protocol/server, performance xlators, cluster
> xlators, protocol/servers.
> With this model, the thin client connects to gfproxyd and glusterd(like
> always). gfproxyd connects to all the bricks. The major problem with this
> is performance, when the client and gfproxyd are not co-located.
>
>
> What is already done by Facebook:
> -
> 1. Volgen code for generating thin client volfile and the gfproxyd daemon
> volfile.
> 2. AHA translator on the thin client, so that on a restart/network
> disruptions between thin client and gfproxyd, we retry fops and the client
> doesn't become inaccessible.
>
>
> What remains to be done:
> -
> 1. Glusterd managing the gfproxyd
> Currently the gfproxy daemon listens on 4 port, if we want to run
> multiple gfproxyd (one per volume)
>

One per volume seems reasonable. However as we start scaling the number of
volumes, the number of gfproxy processes might become overwhelming and
necessitate us to multiplex (as has been the case with bricks). We can also
consider the possibility of a subset of volumes being exported through a
node and have different subset of volumes be exported through other nodes.
Further thought is necessary to evolve the set of policies needed for
managing gfproxyd daemons on trusted storage pool nodes and possibly
outside trusted storage pool too.



> 2. Redo the volgen and daemon management in glusterd2
> -  Ability to be able to run daemons on subset of cluster nodes
> -  ssl
> - Validate with other features like snap, tier,
> 3. Graph switch for the gfproxyd
>


I wonder if we can implement a delay interval before failing over to a
different server in AHA. If we can do that, then we may not have to worry
about graph switch and instead resort to restart of gfproxyd daemons upon
configuration changes that affect graph topology. Delay before failing over
will also help in situations where there is a transient network
interruption.



> 4. Failover from one gfproxyd to another
>

What are the problems we need to consider here?


> 5. Less resource consumption on thin client - Memory and threads
> 6. Performance analysis
>
> Issue: https://github.com/gluster/glusterfs/issues/242
> 
>


Might be a good idea to capture this discussion on the issue and continue
there!

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Gluster documentation search

[Vijay Bellur]

Great work, Nigel! This solves a problem that has bothered us for a while.
Thank you!

-Vijay

On Mon, Aug 28, 2017 at 7:14 AM, Nigel Babu  wrote:

> Hello folks,
>
> I spend some time today mucking about trying to figure out how to make our
> documentation search a better experience. The short answer is, search kind
> of works now.
>
> Long answer: mkdocs creates a client side file which is used for search.
> RTD overrides this by referring people to Elasticsearch. However, that
> doesn't clear out stale entries and we're plagued with a whole lot of stale
> entries. I've made some changes that other consumers of RTD have done to
> override our search to use the JS file rather than Elasticsearch.
>
> --
> nigelb
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Brick count limit in a volume

[Vijay Bellur]

Can you also please provide more detail on why those many bricks are needed
in a single volume?

Thanks,
Vijay

On Wed, Aug 23, 2017 at 12:43 AM, Atin Mukherjee 
wrote:

> An upstream bug would be ideal as github issue is mainly used for
> enhancements. In the mean time, could you point to the exact failure shown
> at the command line and the log entry from cli.log?
>
> On Wed, Aug 23, 2017 at 12:10 AM, Serkan Çoban 
> wrote:
>
>> Hi, I think this is the line limiting brick count:
>> https://github.com/gluster/glusterfs/blob/c136024613c697fec8
>> 7aaff3a070862b92c57977/cli/src/cli-cmd-parser.c#L84
>>
>> Can gluster-devs increase this limit? Should I open a github issue?
>>
>> On Mon, Aug 21, 2017 at 7:01 PM, Serkan Çoban 
>> wrote:
>> > Hi,
>> > Gluster version is 3.10.5. I am trying to create a 5500 brick volume,
>> > but getting an error stating that  bricks is the limit. Is this a
>> > known limit? Can I change this with an option?
>> >
>> > Thanks,
>> > Serkan
>> ___
>> Gluster-users mailing list
>> gluster-us...@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Fwd: tendrl-release v1.5.0 (Milestone1) is available

[Vijay Bellur]

Thank you Rohan for letting us know about this release!

I looked for screenshots of the dashboard and found a few at [1]. Are there
more screenshots that you can share with us?

Regards,
 Vijay

[1] https://github.com/Tendrl/documentation/wiki/Metrics

On Fri, Aug 4, 2017 at 10:22 AM, Rohan Kanade  wrote:

> The Tendrl project is building up an easily configured monitoring
> dashboard for Gluster and below is an early build (with instructions) about
> how to set it up. Looking forward to feedback via our mailing lists (
> tendrl-de...@redhat.com) or, our Gitter channel "tendrl-devel".
>
> -- Forwarded message --
> From: Rohan Kanade 
> Date: Fri, Aug 4, 2017 at 7:48 PM
> Subject: tendrl-release v1.5.0 (Milestone1) is available
> To: Mailing list for the contributors to the Tendrl project <
> tendrl-de...@redhat.com>
>
>
> Hello,
>
> The Tendrl team is happy to present tendrl-release v1.5.0 (Milestone1)
>
> Summary: https://github.com/Tendrl/documentation/wiki/Tendrl-release-
> v1.5.0-(summary)
>
> Install docs: https://github.com/Tendrl/documentation/wiki/Tendrl-release-
> v1.5.0-(install-doc)
>
> Cheers!
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Changing the relative order of read-ahead and open-behind

[Vijay Bellur]

On Fri, Jul 21, 2017 at 3:26 AM, Raghavendra Gowdappa 
wrote:

> Hi all,
>
> We've a bug [1], due to which read-ahead is completely disabled when the
> workload is read-only. One of the easy fix was to make read-ahead as an
> ancestor of open-behind in xlator graph (Currently its a descendant). A
> patch has been sent out by Rafi to do the same. As noted in one of the
> comments, one flip side of this solution is that small files (which are
> eligible to be cached by quick read) are cached twice - once each in
> read-ahead and quick-read - wasting up precious memory. However, there are
> no other simpler solutions for this issue. If you've concerns on the
> approach followed by [2] or have other suggestions please voice them out.
> Otherwise, I am planning to merge [2] for lack of better alternatives.
>


Since the maximum size of files cached by quick-read is 64KB, can we have
read-ahead kick in for offsets greater than 64KB?

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] gfid and volume-id extended attributes lost

[Vijay Bellur]

Do you observe any event pattern (self-healing / disk failures / reboots
etc.) after which the extended attributes are missing?

Regards,
Vijay

On Fri, Jul 7, 2017 at 5:28 PM, Ankireddypalle Reddy 
wrote:

> We lost the attributes on all the bricks on servers glusterfs2 and
> glusterfs3 again.
>
>
>
> [root@glusterfs2 Log_Files]# gluster volume info
>
>
>
> Volume Name: StoragePool
>
> Type: Distributed-Disperse
>
> Volume ID: 149e976f-4e21-451c-bf0f-f5691208531f
>
> Status: Started
>
> Number of Bricks: 20 x (2 + 1) = 60
>
> Transport-type: tcp
>
> Bricks:
>
> Brick1: glusterfs1sds:/ws/disk1/ws_brick
>
> Brick2: glusterfs2sds:/ws/disk1/ws_brick
>
> Brick3: glusterfs3sds:/ws/disk1/ws_brick
>
> Brick4: glusterfs1sds:/ws/disk2/ws_brick
>
> Brick5: glusterfs2sds:/ws/disk2/ws_brick
>
> Brick6: glusterfs3sds:/ws/disk2/ws_brick
>
> Brick7: glusterfs1sds:/ws/disk3/ws_brick
>
> Brick8: glusterfs2sds:/ws/disk3/ws_brick
>
> Brick9: glusterfs3sds:/ws/disk3/ws_brick
>
> Brick10: glusterfs1sds:/ws/disk4/ws_brick
>
> Brick11: glusterfs2sds:/ws/disk4/ws_brick
>
> Brick12: glusterfs3sds:/ws/disk4/ws_brick
>
> Brick13: glusterfs1sds:/ws/disk5/ws_brick
>
> Brick14: glusterfs2sds:/ws/disk5/ws_brick
>
> Brick15: glusterfs3sds:/ws/disk5/ws_brick
>
> Brick16: glusterfs1sds:/ws/disk6/ws_brick
>
> Brick17: glusterfs2sds:/ws/disk6/ws_brick
>
> Brick18: glusterfs3sds:/ws/disk6/ws_brick
>
> Brick19: glusterfs1sds:/ws/disk7/ws_brick
>
> Brick20: glusterfs2sds:/ws/disk7/ws_brick
>
> Brick21: glusterfs3sds:/ws/disk7/ws_brick
>
> Brick22: glusterfs1sds:/ws/disk8/ws_brick
>
> Brick23: glusterfs2sds:/ws/disk8/ws_brick
>
> Brick24: glusterfs3sds:/ws/disk8/ws_brick
>
> Brick25: glusterfs4sds.commvault.com:/ws/disk1/ws_brick
>
> Brick26: glusterfs5sds.commvault.com:/ws/disk1/ws_brick
>
> Brick27: glusterfs6sds.commvault.com:/ws/disk1/ws_brick
>
> Brick28: glusterfs4sds.commvault.com:/ws/disk10/ws_brick
>
> Brick29: glusterfs5sds.commvault.com:/ws/disk10/ws_brick
>
> Brick30: glusterfs6sds.commvault.com:/ws/disk10/ws_brick
>
> Brick31: glusterfs4sds.commvault.com:/ws/disk11/ws_brick
>
> Brick32: glusterfs5sds.commvault.com:/ws/disk11/ws_brick
>
> Brick33: glusterfs6sds.commvault.com:/ws/disk11/ws_brick
>
> Brick34: glusterfs4sds.commvault.com:/ws/disk12/ws_brick
>
> Brick35: glusterfs5sds.commvault.com:/ws/disk12/ws_brick
>
> Brick36: glusterfs6sds.commvault.com:/ws/disk12/ws_brick
>
> Brick37: glusterfs4sds.commvault.com:/ws/disk2/ws_brick
>
> Brick38: glusterfs5sds.commvault.com:/ws/disk2/ws_brick
>
> Brick39: glusterfs6sds.commvault.com:/ws/disk2/ws_brick
>
> Brick40: glusterfs4sds.commvault.com:/ws/disk3/ws_brick
>
> Brick41: glusterfs5sds.commvault.com:/ws/disk3/ws_brick
>
> Brick42: glusterfs6sds.commvault.com:/ws/disk3/ws_brick
>
> Brick43: glusterfs4sds.commvault.com:/ws/disk4/ws_brick
>
> Brick44: glusterfs5sds.commvault.com:/ws/disk4/ws_brick
>
> Brick45: glusterfs6sds.commvault.com:/ws/disk4/ws_brick
>
> Brick46: glusterfs4sds.commvault.com:/ws/disk5/ws_brick
>
> Brick47: glusterfs5sds.commvault.com:/ws/disk5/ws_brick
>
> Brick48: glusterfs6sds.commvault.com:/ws/disk5/ws_brick
>
> Brick49: glusterfs4sds.commvault.com:/ws/disk6/ws_brick
>
> Brick50: glusterfs5sds.commvault.com:/ws/disk6/ws_brick
>
> Brick51: glusterfs6sds.commvault.com:/ws/disk6/ws_brick
>
> Brick52: glusterfs4sds.commvault.com:/ws/disk7/ws_brick
>
> Brick53: glusterfs5sds.commvault.com:/ws/disk7/ws_brick
>
> Brick54: glusterfs6sds.commvault.com:/ws/disk7/ws_brick
>
> Brick55: glusterfs4sds.commvault.com:/ws/disk8/ws_brick
>
> Brick56: glusterfs5sds.commvault.com:/ws/disk8/ws_brick
>
> Brick57: glusterfs6sds.commvault.com:/ws/disk8/ws_brick
>
> Brick58: glusterfs4sds.commvault.com:/ws/disk9/ws_brick
>
> Brick59: glusterfs5sds.commvault.com:/ws/disk9/ws_brick
>
> Brick60: glusterfs6sds.commvault.com:/ws/disk9/ws_brick
>
> Options Reconfigured:
>
> performance.readdir-ahead: on
>
> diagnostics.client-log-level: INFO
>
> auth.allow: glusterfs1sds,glusterfs2sds,glusterfs3sds,glusterfs4sds.
> commvault.com,glusterfs5sds.commvault.com,glusterfs6sds.commvault.com
>
>
>
> Thanks and Regards,
>
> Ram
>
> *From:* Pranith Kumar Karampuri [mailto:pkara...@redhat.com]
> *Sent:* Friday, July 07, 2017 12:15 PM
>
> *To:* Ankireddypalle Reddy
> *Cc:* Gluster Devel (gluster-devel@gluster.org); gluster-us...@gluster.org
> *Subject:* Re: [Gluster-devel] gfid and volume-id extended attributes lost
>
>
>
>
>
>
>
> On Fri, Jul 7, 2017 at 9:25 PM, Ankireddypalle Reddy 
> wrote:
>
> 3.7.19
>
>
>
> These are the only callers for removexattr and only _posix_remove_xattr
> has the potential to do removexattr as posix_removexattr already makes sure
> that it is not gfid/volume-id. And surprise surprise _posix_remove_xattr
> happens only from healing code of afr/ec. And this can only happen if the
> source brick doesn't have gfid, which doesn't seem to match with the
> situation you explained.
>
>#   line  filename / context / line
>1   1234  xlators/mgmt/glusterd/src/glusterd-quota.c
>

Re: [Gluster-devel] Gluster statedumps and mallinfo

[Vijay Bellur]

On 07/03/2017 05:55 AM, Raghavendra Gowdappa wrote:

Hi,

Recently I observed one of the mallinfo fields had a negative value.

DUMP-START-TIME: 2017-06-09 10:59:43.747440

[mallinfo]
mallinfo_arena=-1517670400
mallinfo_ordblks=8008214
mallinfo_smblks=0
mallinfo_hblks=1009
mallinfo_hblkhd=863453184
mallinfo_usmblks=0
mallinfo_fsmblks=0
mallinfo_uordblks=1473090528
mallinfo_fordblks=1304206368
mallinfo_keepcost=2232208

As seen above mallinfo_arena is negative.

On probing further I came across posts that said mallinfo is not the ideal 
interface to get metadata about memory allocated by malloc [1]. Instead there 
were two alternatives - malloc_stats and malloc_info - suggested.


Good find!



* what among the above gives accurate and simple explanation about memory 
consumption of glusterfs?
* Should we deprecate mallinfo and just retain malloc_stats and malloc_info 
outputs? IOW, which among these need to be retained in statedump?


Yes, let us deprecate mallinfo() on platforms that support malloc_info().

man 3 malloc_info states:

"The malloc_info() function is designed to address deficiencies in 
malloc_stats(3) and mallinfo(3)."


Hence adding malloc_info() to statedump looks like a better option to me.

Regards,
Vijay



Since I've limited understanding of glibc memory allocator, I am reaching out 
to the wider community for feedback.

[1] http://udrepper.livejournal.com/20948.html

regards,
Raghavendra
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel



___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Support for statx in 4.0

[Vijay Bellur]

Hey All,

Linux 4.11 has added support for a new system call, statx [1]. statx 
provides more information than what stat() does today. Given that there 
could be potential users for this new interface it would be nice to have 
statx supported in 4.0. We could review if some of our translators, nfs 
& smb accesses leverage this interface for some enhanced functionality.


I have not yet checked the current state of support for statx in fuse. 
Needless to say it would be useful to have it in there. If somebody's 
interested in working this out across the entire stack (fuse & gluster), 
do let us know here!


Regards,
Vijay

[1] https://patchwork.kernel.org/patch/8982111/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Qusetions] How profile affect glusterfs performance?

[Vijay Bellur]

On 06/07/2017 02:41 PM, Pranith Kumar Karampuri wrote:





Hi Yuan,



On Wed, Jun 7, 2017 at 1:14 PM, liuy...@cmss.chinamobile.com
 mailto:liuy...@cmss.chinamobile.com>> wrote:

Hi Pranith,

First of all, we owe you guys big thanks for your excellent work.

I'm the lead of storage team in China Mobile, Xiubo is also in my
team :)


Ah! nice :-).



We have deployed more than 100 glusterfs nodes in producion(20 nodes
for the biggest single cluster), so far so good.  Use case ranging
from image/vedio, to small files for mailbox services.
For profile stuff, we'd like to enable it as always to collect perf
stat for monitoring.



Thank you for your feedback and details about deployment. It is always 
fascinating for us to learn about production usage of gluster :-).





Also, I have a question for NDMP support. I bumpped one thread
discussing it but not clear yet.
(http://lists.gluster.org/pipermail/gluster-devel/2016-August/050513.html
)

Do you have any plan to for a glusterfs-ndmp server from scratch? We
are new to the development of glusterfs, I think ndmp support would
be a good starting point for us to join the
development  of the community.  We'd like to see NDMP supported by
glusterfs because some of our customers want it.


I don't know enough about this. Vijay has idea about this, so will
respond later.


We have had some thoughts about implementing a native ndmp server but 
have not had a chance to implement that. If you have an idea or design 
in mind for glusterfs-ndmp server, please share that here. Happy to 
provide any assistance needed for implementing this feature.


Cheers,
Vijay


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Performance experiments with io-stats translator

[Vijay Bellur]

Nice work!

What is the network interconnect bandwidth? How much of the network
bandwidth is in use while the test is being run? Wondering if there is
saturation in the network layer.

-Vijay

On Tue, Jun 6, 2017 at 7:35 AM, Krutika Dhananjay 
wrote:

> Hi,
>
> As part of identifying performance bottlenecks within gluster stack for VM
> image store use-case, I loaded io-stats at multiple points on the client
> and brick stack and ran randrd test using fio from within the hosted vms in
> parallel.
>
> Before I get to the results, a little bit about the configuration ...
>
> 3 node cluster; 1x3 plain replicate volume with group virt settings,
> direct-io.
> 3 FUSE clients, one per node in the cluster (which implies reads are
> served from the replica that is local to the client).
>
> io-stats was loaded at the following places:
> On the client stack: Above client-io-threads and above protocol/client-0
> (the first child of AFR).
> On the brick stack: Below protocol/server, above and below io-threads and
> just above storage/posix.
>
> Based on a 60-second run of randrd test and subsequent analysis of the
> stats dumped by the individual io-stats instances, the following is what I
> found:
>
> *​​Translator Position*   *Avg Latency of READ fop as
> seen by this translator*
>
> 1. parent of client-io-threads1666us
>
> ∆ (1,2) = 50us
>
> 2. parent of protocol/client-01616us
>
> ∆ (2,3) = 1453us
>
> - end of client stack -
> - beginning of brick stack ---
>
> 3. child of protocol/server   163us
>
> ∆ (3,4) = 7us
>
> 4. parent of io-threads156us
>
> ∆ (4,5) = 20us
>
> 5. child-of-io-threads  136us
>
> ∆ (5,6) = 11us
>
> 6. parent of storage/posix   125us
> ...
>  end of brick stack 
>
> So it seems like the biggest bottleneck here is a combination of the
> network + epoll, rpc layer?
> I must admit I am no expert with networks, but I'm assuming if the client
> is reading from the local brick, then
> even latency contribution from the actual network won't be much, in which
> case bulk of the latency is coming from epoll, rpc layer, etc at both
> client and brick end? Please correct me if I'm wrong.
>
> I will, of course, do some more runs and confirm if the pattern is
> consistent.
>
> -Krutika
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Who's using OpenStack Cinder & Gluster? [ Was Re: [Gluster-users] Fwd: Re: GlusterFS removal from Openstack Cinder]

[Vijay Bellur]

Joe,

Agree with you on turning this around into something more positive.

One aspect that would really help us decide on our next steps here is the
actual number of deployments that will be affected by the removal of the
gluster driver in Cinder. If you are running or aware of a deployment of
OpenStack Cinder & Gluster, can you please respond on this thread or to me
& Niels in private providing more details about your deployment? Details
like OpenStack & Gluster versions, number of Gluster nodes & total storage
capactiy would be very useful to us.

Thanks!
Vijay


On Tue, May 30, 2017 at 7:22 PM, Joe Julian  wrote:

> On 05/30/2017 03:52 PM, Ric Wheeler wrote:
>
> On 05/30/2017 06:37 PM, Joe Julian wrote:
>
> On 05/30/2017 03:24 PM, Ric Wheeler wrote:
>
> On 05/27/2017 03:02 AM, Joe Julian wrote:
>
> On 05/26/2017 11:38 PM, Pranith Kumar Karampuri wrote:
>
>
>
> On Wed, May 24, 2017 at 9:10 PM, Joe Julian  <mailto:j...@julianfamily.org> > wrote:
>
> Forwarded for posterity and follow-up.
>
>
>  Forwarded Message 
> Subject: Re: GlusterFS removal from Openstack Cinder
> Date: Fri, 05 May 2017 21:07:27 +
> From: Amye Scavarda  
> <mailto:a...@redhat.com> 
> To:     Eric Harney  
> <mailto:ehar...@redhat.com> , Joe
> Julian  
> <mailto:m...@joejulian.name> , Vijay Bellur
>   <mailto:vbel...@redhat.com>
> 
> CC: Amye Scavarda  
> <mailto:a...@redhat.com> 
>
>
>
> Eric,
> I'm sorry to hear this.
> I'm reaching out internally (within Gluster CI team and CentOS CI
> which
> supports Gluster) to get an idea of the level of effort we'll need to
> provide to resolve this.
> It'll take me a few days to get this, but this is on my radar. In the
> meantime, is there somewhere I should be looking at for requirements
> to
> meet this gateway?
>
> Thanks!
> -- amye
>
> On Fri, May 5, 2017 at 16:09 Joe Julian  <mailto:m...@joejulian.name> > wrote:
>
> On 05/05/2017 12:54 PM, Eric Harney wrote:
> >> On 04/28/2017 12:41 PM, Joe Julian wrote:
> >>> I learned, today, that GlusterFS was deprecated and removed
> from
> >>> Cinder as one of our #gluster (freenode) users was attempting
> to
> >>> upgrade openstack. I could find no rational nor discussion of
> that
> >>> removal. Could you please educate me about that decision?
> >>>
> >
> > Hi Joe,
> >
> > I can fill in on the rationale here.
> >
> > Keeping a driver in the Cinder tree requires running a CI
> platform to
> > test that driver and report results against all patchsets
> submitted to
> > Cinder.  This is a fairly large burden, which we could not meet
> once the
> > Gluster Cinder driver was no longer an active development target
> at
> Red Hat.
> >
> > This was communicated via a warning issued by the driver for
> anyone
> > running the OpenStack Newton code, and via the Cinder release
> notes for
> > the Ocata release.  (I can see in retrospect that this was
> probRecording of the meeting can be found at [3].ably not
> > communicated widely enough.)
> >
> > I apologize for not reaching out to the Gluster community about
> this.
> >
> > If someone from the Gluster world is interested in bringing this
> driver
> > back, I can help coordinate there.  But it will require someone
> stepping
> > in in a big way to maintain it.
> >
> > Thanks,
> > Eric
>
> Ah, Red Hat's statement that the acquisition of InkTank was not an
> abandonment of Gluster seems rather disingenuous now. I'm
> disappointed.
>
>
> I am a Red Hat employee working on gluster and I am happy with the kind of
> investments the company did in GlusterFS. Still am. It is a pretty good
> company and really open. I never had any trouble saying something the
> management did is wrong when I strongly felt and they would give a decent
> reason for their decision.
>
>
> Happy to hear that. Still looks like meddling to an outsider. Not the
> Gluster team's fault though (although more participation of the developers
> in community meetings would probably help with that feeling of being
> disconnected, in my own personal opinion).
>
>
> As a community, each member needs to

Re: [Gluster-devel] [Gluster-users] Fwd: Re: GlusterFS removal from Openstack Cinder

[Vijay Bellur]

On Sat, May 27, 2017 at 3:02 AM, Joe Julian  wrote:

> On 05/26/2017 11:38 PM, Pranith Kumar Karampuri wrote:
>
>
>
> On Wed, May 24, 2017 at 9:10 PM, Joe Julian  wrote:
>
>> Forwarded for posterity and follow-up.
>>
>>  Forwarded Message 
>> Subject: Re: GlusterFS removal from Openstack Cinder
>> Date: Fri, 05 May 2017 21:07:27 +
>> From: Amye Scavarda  
>> To: Eric Harney  , Joe Julian
>>  , Vijay Bellur
>>  
>> CC: Amye Scavarda  
>>
>> Eric,
>> I'm sorry to hear this.
>> I'm reaching out internally (within Gluster CI team and CentOS CI which
>> supports Gluster) to get an idea of the level of effort we'll need to
>> provide to resolve this.
>> It'll take me a few days to get this, but this is on my radar. In the
>> meantime, is there somewhere I should be looking at for requirements to
>> meet this gateway?
>>
>> Thanks!
>> -- amye
>>
>> On Fri, May 5, 2017 at 16:09 Joe Julian  wrote:
>>
>>> On 05/05/2017 12:54 PM, Eric Harney wrote:
>>> >> On 04/28/2017 12:41 PM, Joe Julian wrote:
>>> >>> I learned, today, that GlusterFS was deprecated and removed from
>>> >>> Cinder as one of our #gluster (freenode) users was attempting to
>>> >>> upgrade openstack. I could find no rational nor discussion of that
>>> >>> removal. Could you please educate me about that decision?
>>> >>>
>>> >
>>> > Hi Joe,
>>> >
>>> > I can fill in on the rationale here.
>>> >
>>> > Keeping a driver in the Cinder tree requires running a CI platform to
>>> > test that driver and report results against all patchsets submitted to
>>> > Cinder.  This is a fairly large burden, which we could not meet once
>>> the
>>> > Gluster Cinder driver was no longer an active development target at
>>> Red Hat.
>>> >
>>> > This was communicated via a warning issued by the driver for anyone
>>> > running the OpenStack Newton code, and via the Cinder release notes for
>>> > the Ocata release.  (I can see in retrospect that this was probably not
>>> > communicated widely enough.)
>>> >
>>> > I apologize for not reaching out to the Gluster community about this.
>>> >
>>> > If someone from the Gluster world is interested in bringing this driver
>>> > back, I can help coordinate there.  But it will require someone
>>> stepping
>>> > in in a big way to maintain it.
>>> >
>>> > Thanks,
>>> > Eric
>>>
>>> Ah, Red Hat's statement that the acquisition of InkTank was not an
>>> abandonment of Gluster seems rather disingenuous now. I'm disappointed.
>>>
>>
> I am a Red Hat employee working on gluster and I am happy with the kind of
> investments the company did in GlusterFS. Still am. It is a pretty good
> company and really open. I never had any trouble saying something the
> management did is wrong when I strongly felt and they would give a decent
> reason for their decision.
>
>
> Happy to hear that. Still looks like meddling to an outsider. Not the
> Gluster team's fault though (although more participation of the developers
> in community meetings would probably help with that feeling of being
> disconnected, in my own personal opinion).
>
>
>
>>
>>> Would you please start a thread on the gluster-users and gluster-devel
>>> mailing lists and see if there's anyone willing to take ownership of
>>> this. I'm certainly willing to participate as well but my $dayjob has
>>> gone more kubernetes than openstack so I have only my limited free time
>>> that I can donate.
>>>
>>
> Do we know what would maintaining cinder as active entail? Did Eric get
> back to any of you?
>
>
> Haven't heard anything more, no.
>
>
 Policies for maintaining an active driver in cinder can be found at [1]
and [2]. We will need some work to let the driver be active again (after a
revert of the commit that removed the driver from cinder) and provide CI
support as entailed in [2].

I will co-ordinate further internal discussions within Red Hat on this
topic and provide an update soon on how we can proceed here.

Thanks,
Vijay

[1] https://wiki.openstack.org/wiki/Cinder/how-to-contribute-a-driver

[2] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] lock_revocation.t is hanging in regression tests

[Vijay Bellur]

On Wed, May 10, 2017 at 9:01 AM, Jeff Darcy  wrote:

>
>
>
> On Wed, May 10, 2017, at 06:30 AM, Raghavendra G wrote:
>
> marking it bad won't help as even bad tests are run by build system (and
> they might hang).
>
>
> This is news to me.  Did something change to make it this way?  If so, we
> should change it back.  There's no point in having a way to mark tests as
> bad if the build system ignores it.
>

skip_bad_tests is set to "yes" by default in run-tests.sh. So the
regression job  should skip bad tests during a run.

Nigel can probably educate us if the behavior is different :).

-Vijay





>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Demo slots available in community meeting

[Vijay Bellur]

Hi All,

We have 1-2 slots available in this week's community meeting for demoing
anything interesting that you have been working on. Please drop a note here
if you are interested in demoing this week.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] libgfapi access to snapshot volume

[Vijay Bellur]

On Thu, May 4, 2017 at 9:12 AM, Ankireddypalle Reddy 
wrote:

> Hi,
>
>Can glusterfs snapshot volume be accessed through libgfapi.
>


Yes, activated snapshots can be accessed through libgfapi. User serviceable
snapshots in Gluster makes use of gfapi to access activated snapshots.
xlators/features/snapview-server contains code that accesses snapshots
through libgfapi.

Regards,
Vijay


>
>
> Thanks and Regards,
>
> Ram
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for
> the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] DHT xlator, read and write a file during creating a new file

[Vijay Bellur]

On Tue, May 2, 2017 at 8:00 AM, Tahereh Fattahi 
wrote:

> Hi
>
> I want to use a file as a counter when I create a file in dht xlator.
> I mean, after creating a new file,  I want open a file in the same
> directory with a special name, read that, update the counter and write
> back.
> I think for this purpose I  should open in dht_create_cbk, read in
> dht_open_cbk and write in dht_readv_cbk.
> I think I should use dht_open , dht_readv and dht_writev. Maybe I could
> create inputs for these function expect frame! is it correct to use the
> frame fro dht_create function?
>
> Is this scenario correct or there is better way?
>
>
Have you tried the object count feature [1] ?

Regards,
Vijay

[1]
http://gluster-documentations.readthedocs.io/en/latest/Features/quota-object-count/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Is anyone else having trouble authenticating with review.gluster.org over ssh?

[Vijay Bellur]

h

On Mon, Apr 17, 2017 at 12:48 AM, Nigel Babu  wrote:

> This should be fixed now: https://bugzilla.redhat.com/sh
> ow_bug.cgi?id=1442672
>


Thank you Nigel.


>
>
> Vijay, can you link me to your failed Jenkins job? Jenkins should have
> been able to clone since it uses the git protocol and not SSH.
>


This is a private jenkins instance that I use for running tests and it uses
a ssh clone.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Is anyone else having trouble authenticating with review.gluster.org over ssh?

[Vijay Bellur]

On Sun, Apr 16, 2017 at 11:44 AM, Raghavendra Talur 
wrote:

> On Sun, Apr 16, 2017 at 9:07 PM, Raghavendra Talur 
> wrote:
> > I am not able to login even after specifying the key file
> >
> > $ ssh -T -vvv -i ~/.ssh/gluster raghavendra-ta...@git.gluster.org
> > OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
> > debug1: Reading configuration data /home/rtalur/.ssh/config
> > debug1: Reading configuration data /etc/ssh/ssh_config
> > debug3: /etc/ssh/ssh_config line 56: Including file
> > /etc/ssh/ssh_config.d/05-redhat.conf depth 0
> > debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
> > debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 2: include
> > /etc/crypto-policies/back-ends/openssh.txt matched no files
> > debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 8: Applying options
> for *
> > debug1: auto-mux: Trying existing master
> > debug1: Control socket
> > "/tmp/ssh_mux_git.gluster.org_22_raghavendra-talur" does not exist
> > debug2: resolving "git.gluster.org" port 22
> > debug2: ssh_connect_direct: needpriv 0
> > debug1: Connecting to git.gluster.org [8.43.85.171] port 22.
> > debug1: Connection established.
> > debug1: identity file /home/rtalur/.ssh/gluster type 1
> > debug1: key_load_public: No such file or directory
> > debug1: identity file /home/rtalur/.ssh/gluster-cert type -1
> > debug1: Enabling compatibility mode for protocol 2.0
> > debug1: Local version string SSH-2.0-OpenSSH_7.4
> > ssh_exchange_identification: Connection closed by remote host
>
> Confirmed with Pranith that he is facing same issue.
>


One of my jenkins jobs also bailed out since it was unable to clone from
r.g.o.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Nfs-ganesha-devel] Device or resource busy when runltp cleanup test-files

[Vijay Bellur]

On Wed, Apr 12, 2017 at 10:37 PM, Kinglong Mee 
wrote:

> Yes, this one is silly rename,
>
> >>> rm: cannot remove ‘/mnt/nfs/ltp-JEYAuky2dz/.nfsaa46457a6a72f8ea14f5’:
> Device or resource busy
>
> But, the other one isn't client's silly rename,
>
>  rm: cannot remove ‘/mnt/nfs/ltp-JEYAuky2dz/rmderQsjV’: Directory
> not empty
>
> Also, the second one often appears when testing ganesha nfs by ltp.
>
> I try to get an tcpdump, I found, when ls under the nfs client directory,
>
> the glusterfs client doesn't send readdir to glusterfs server, so that,
> the nfs client gets an empty directory information, but it's empty
> underlying.
>
> ls at some other directory,
> the glusterfs client send readdir to glusterfs server, nfs client gets
> right dirctory
> information as the underlying.
>
> So, I guess maybe there are some problems in MDCHANE or glusterfs client?
>
> ]# cat /var/lib/glusterd/vols/gvtest/trusted-gvtest.tcp-fuse.vol
> volume gvtest-client-0
> type protocol/client
> option send-gids true
> option password d80765c8-95ae-46ed-912a-0c98fdd1e7cd
> option username 7bc6276f-245c-477a-99d0-7ead4fcb2968
> option transport.address-family inet
> option transport-type tcp
> option remote-subvolume /gluster-test/gvtest
> option remote-host 192.168.9.111
> option ping-timeout 42
> end-volume
>
> volume gvtest-client-1
> type protocol/client
> option send-gids true
> option password d80765c8-95ae-46ed-912a-0c98fdd1e7cd
> option username 7bc6276f-245c-477a-99d0-7ead4fcb2968
> option transport.address-family inet
> option transport-type tcp
> option remote-subvolume /gluster-test/gvtest
> option remote-host 192.168.9.111
> option ping-timeout 42
> end-volume
>
> volume gvtest-dht
> type cluster/distribute
> option lock-migration off
> subvolumes gvtest-client-0 gvtest-client-1
> end-volume
>
> volume gvtest
> type debug/io-stats
> option count-fop-hits off
> option latency-measurement off
> option log-level INFO
> subvolumes gvtest-dht
> end-volume
>
>
>

Have you checked the gluster export directories after receiving ENOTEMPTY?
If you observe any files there, can you please check if they are 0-byte
files with access mode 600 and sticky bit set?

When a file is being renamed, if the hash value of its new name happens to
fall in a range that is different than that of the old name, gluster
creates one such file. In the past we have seen instances of these files
not being cleaned up properly in some cases and that causes a ENOTEMPTY
when rmdir lands on the parent directory. Trying to see if we are hitting
the same problem here.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] performance test and a spacial workload

[Vijay Bellur]

On Sun, Apr 16, 2017 at 2:52 AM, Tahereh Fattahi 
wrote:

> Hi
> I want to create a performance test with a special workload:
> 1. create a file in a directory
> 2. setxattr on the directory of the previous file
> I could not merge this two in glister code and could not find a framework
> that generate this workload for me.
> I read the code of smallfile (a framework for performance testing the
> gluster document introduced), maybe there is a way to change the code of
> this software to do a setxattr on directory after create a file.
> Which one is better to spend time? change the gluster code for merge or
> the smallfile code? Can anyone help me?
>


What performance metrics are you interested in measuring?

If you are interested in measuring time, a small bash script like:

time for i in 1..N
do
   touch /mnt/glusterfs/foo/file.$i
setfattr -n  -v  /mnt/glusterfs/foo
done

would be simpler than either approach.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Read-only option for a replicated (replication for fail-over) Gluster volume

[Vijay Bellur]

On Fri, Apr 7, 2017 at 7:57 AM, Mackay, Michael  wrote:

> I’ve updated my patch to work for glusterfs 3.10.0.  I thought that
> targeting the latest stable baseline would be best.
>
>
>
> Could I ask for a starting point to submit the change?  I see a place to
> submit a change on git, but if you could point me to a starting point in
> the whole process I can take it from there, I believe.  I want to make sure
> I’m following your process.
>
>
>

Thanks! Can you please follow the patch submission workflow at [1] for
submitting this?

Regards,
Vijay

[1]
https://gluster.readthedocs.io/en/latest/Developer-guide/Development-Workflow/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Translator issue: Change content of iovec

[Vijay Bellur]

On Mon, Apr 3, 2017 at 10:38 AM, David Spisla 
wrote:

> Dear GlusterDevels,
>
>
>
> I am coding a translator which is changing the content of the iovec if a
> user wants to read the content of the original file. Imagine you have a
> simple text file with the content „hello“.
>
> If the user wants to read that file, my translator changes the content to
> „I am the new content!!!“. But at the moment my translator is not working
> perfectly.
>
>
>
> Original Content à„hello“
>
> New content à „I am the new content!!!“
>
>
>
> If I read that file I get à „I am “
>
> It seems that there is still the old filesize allocated. I am looking for
> a way to allocate a new size so that my new String will fit into that file.
>
> I tried out several things, but without success. Maybe someone of you has
> an idea. At the moment my translator is on the top of the client stack.
>
>
>
> You find the source attached. I did all changes in à stub_readv_cbk
>
>
>


You would need to modify lookup_cbk in stub.c to indicate the appropriate
meta data attributes for the file being read. Specifically, you would need
to set ia_size in struct iatt.

HTH,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] xlator specific mount options

[Vijay Bellur]

On Thu, Mar 30, 2017 at 8:55 AM, Ankireddypalle Reddy 
wrote:

> Hi,
>
>Is there a framework to specify options during volume mount  and
> then the Xlator use those mount options.
>
>
>
>
>

You can use --xlator-option with mount.glusterfs to specify option(s) for a
specific client.

Please check extras/test/bug-920583.t for an example on usage.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] How can add one fop after another fop?

[Vijay Bellur]

On Thu, Mar 30, 2017 at 7:22 AM, Tahereh Fattahi 
wrote:

> Thanks for your answer.
> For example I want to save the number of files in directory as one
> extended attribute.
> After every create operation, this number should be added and set as
> attribute, So after create I need a setxattr.
> But I dont know where this create is completed, I tested in dht translator
> and it was false because the creation is not complete in this translator.
>
>
>
A side effect of inode quota [1] implementation is that it provides a
mechanism to query the number of files/objects in a directory or volume.

Might be worth checking if that meets your needs.

Regards,
Vijay

[1]
http://gluster-documentations.readthedocs.io/en/latest/Features/quota-object-count/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Maintainers 2.0 Proposal

[Vijay Bellur]

On Sat, Mar 25, 2017 at 8:01 AM, Raghavendra Gowdappa 
wrote:

>
>
> - Original Message -
> > From: "Raghavendra Gowdappa" 
> > To: "Pranith Kumar Karampuri" 
> > Cc: "Amar Tumballi" , "GlusterFS Maintainers" <
> maintain...@gluster.org>, "Gluster Devel"
> > , "Michael Scherer" 
> > Sent: Saturday, March 25, 2017 5:22:44 PM
> > Subject: Re: [Gluster-Maintainers] [Gluster-devel] Maintainers 2.0
> Proposal
> >
> >
> >
> > - Original Message -
> > > From: "Pranith Kumar Karampuri" 
> > > To: "Michael Scherer" 
> > > Cc: "Amar Tumballi" , "GlusterFS Maintainers"
> > > , "Gluster Devel"
> > > 
> > > Sent: Friday, March 24, 2017 7:12:32 PM
> > > Subject: Re: [Gluster-Maintainers] [Gluster-devel] Maintainers 2.0
> Proposal
> > >
> > > Do we also plan to publish similar guidelines for deciding on Project
> > > maintainer?
> >
> > +1 for defining roles, responsibilities and qualifications of a Project
> > manager.
>
> s/manager/maintainer/ :)
>


Agreed. There is a need to define the responsibilities of various roles -
architects, project maintainers,  project and community leads. We have used
some of these terms interchangeably in the past. Will add more details on
these roles and provide more clarity.



> >
> > >
> > > On Fri, Mar 24, 2017 at 2:24 AM, Michael Scherer < msche...@redhat.com
> >
> > > wrote:
> > >
> > >
> > > Le samedi 18 mars 2017 à 16:47 +0530, Pranith Kumar Karampuri a écrit :
> > > > On Sat, Mar 18, 2017 at 1:20 AM, Amar Tumballi < atumb...@redhat.com
> >
> > > > wrote:
> > > >
> > > > > I don't want to take the discussions in another direction, but want
> > > > > clarity on few things:
> > > > >
> > > > > 1. Does maintainers means they are only reviewing/ merging patches?
> > > > > 2. Should maintainers be responsible for answering ML / IRC
> questions
> > > > > (well, they should focus more on documentation IMO).
> > > > > 3. Who's responsibility is it to keep the gluster.org webpage? I
> > > > > personally feel the responsibility should be well defined.
> > >
> > > Theses point seems to have been overlooked (as no one answered), yet I
> > > think they do matter if we want to expand the community besides coders.
> > >
> > > And since one of the goal is to "Welcome more contibutors(sic) at a
> > > project impacting level", I think we should be also speaking of
> > > contributions besides code (ie, website, for example, documentation for
> > > another).
> > >
> > > While on it, I would like to see some points about:
> > >
> > > - ensure that someone is responsible for having the design discussion
> in
> > > the open
> > > - ensure that each feature get proper testing when committed, and the
> > > maintainers is responsible for making sure this happen
> > > - ensure that each feature get documented when committed.
> > >
> > > If we think of contribution as a pipeline (kinda like the sales
> funnel),
> > > making sure there is documentation also mean people can use the
> > > software, thus increasing the community, and so helping to recruit
> > > people in a contributor pipeline.
> > >
> > > Proper testing means that it make refactoring easier, thus easing
> > > contributions (ie, people can submit patches and see nothing break,
> even
> > > for new features), thus also making people likely more at ease to
> submit
> > > patches later.
> > >
> > > And making sure the design discussion occurs in the open is also more
> > > welcoming to contributors, since they can see how we discuss, and learn
> > > from it.
>


Agree to all of these. The current guidelines for maintainers / owners
lists most of these points as core responsibilities [1].

Thanks,
Vijay

 [1]
https://gluster.readthedocs.io/en/latest/Contributors-Guide/Guidelines-For-Maintainers/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Read-only option for a replicated (replication for fail-over) Gluster volume

[Vijay Bellur]

On Tue, Mar 21, 2017 at 10:52 AM, Mackay, Michael 
wrote:

> Thanks for the help and advice so far.  It’s difficult at times to
> describe what the use case is, so I’ll try here.
>
>
>
> We need to make sure that no one can write to the physical volume in any
> way.  We want to be able to be sure that it can’t be corrupted. We know
> from working with Gluster that we shouldn’t access the brick directly, and
> that’s part of the point.  We want to make it so it’s impossible to write
> to the volume or the brick under any circumstances.  At the same time, we
> like Gluster’s recovery capability, so if one of two copies of the data
> becomes unavailable (due to failure of the host server or maintenance) the
> other copy will still be up and available.
>
>
>
> Essentially, the filesystem under the brick is a physically read-only disk
> that is set up at integration time and delivered read-only.  We won’t want
> to change it after delivery, and (in this case for security) we want it to
> be immutable so we know we can rely on that data to be the same always, no
> matter what.
>
>
>
> All users will get data from the Gluster mount and use it, but from the
> beginning it would be read-only.
>
>
>
> A new deliver might have new data, or changed data, but that’s the only
> time it will change.
>
>
>
> I want to repeat as well that we’ve identified changes in the code
> baseline that allow this to work, if interested.
>
>
>
>
>

Would it be possible to send this patch on gerrit for master branch? You
can find the workflow for patch submission at [1].

Thanks,
Vijay

[1]
https://gluster.readthedocs.io/en/latest/Developer-guide/Development-Workflow/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Consistent time attributes (ctime, atime and mtime) across replica set and distribution set

[Vijay Bellur]

On Sun, Mar 19, 2017 at 10:14 AM, Amar Tumballi  wrote:

>
>
> On Thu, Mar 16, 2017 at 6:52 AM, Soumya Koduri  wrote:
>
>>
>>
>> On 03/16/2017 02:27 PM, Mohammed Rafi K C wrote:
>>
>>>
>>>
>>> On 03/15/2017 11:31 PM, Soumya Koduri wrote:
>>>
 Hi Rafi,

 I haven't thoroughly gone through design. But have few
 comments/queries which I have posted inline for now .

 On 02/28/2017 01:11 PM, Mohammed Rafi K C wrote:

> Thanks for the reply , Comments are inline
>
>
>
> On 02/28/2017 12:50 PM, Niels de Vos wrote:
>
>> On Tue, Feb 28, 2017 at 11:21:55AM +0530, Mohammed Rafi K C wrote:
>>
>>> Hi All,
>>>
>>>
>>> We discussed the problem $subject in the mail thread [1]. Based on
>>> the
>>> comments and suggestions I will summarize the design (Made as
>>> points for
>>> simplicity.)
>>>
>>>
>>> 1) As part of each fop, top layer will generate a time stamp and
>>> pass it
>>> to the down along with other param.
>>>
>>> 1.1) This will bring a dependency for NTP synced clients along
>>> with
>>> servers
>>>
>> What do you mean with "top layer"? Is this on the Gluster client, or
>> does the time get inserted on the bricks?
>>
> It is the top layer (master xlator) in client graph like fuse, gfapi,
> nfs . My mistake I should have mentioned . Sorry for that.
>

 These clients shouldn't include internal client processes like
 rebalance, self-heal daemons right? IIUC from [1], we should avoid
 changing times during rebalance and self-heals.

 Also what about fops generated from the underlying layers -
 getxattr/setxattr which may modify these time attributes?

>>>
>>> Since the time stamps are appended from master xlators like fuse , we
>>> will not have the timestamp for internal daemons as they don't have
>>> master xlator loaded. internal fops won't generate new timestamp , even
>>> if we are sending an internal fops from say dht, it will have only one
>>> time genrated by fuse. So I think this is fine.
>>>
>>
>> Okay. But since self-heal and snapview-server (atleast) seem to be using
>> gfapi, how can gfapi differentiate between these internal clients and other
>> applications (like NFS/SMB)? I thought we need intelligence on server-side
>> to identify such clients based on pid and then avoid updating timestamp
>> sent by them.
>>
>>
> Very good point. Considering we should be using gfapi in future for most
> of the internal processes, this becomes more important. Should we just add
> it as argument to glfs_init() options? If you set a flag during init, the
> ctime xattr is not generated for the fops which otherwise generate and send
> them.
>
>
>

Most internal clients are recognized by their special PIDs
(gf_special_pid_t) in various translators. We could store this PID
information in a global location and use that to determine whether
timestamp generation is needed. This way we will not be required to change
any api signature for distinguishing internal clients.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Maintainers 2.0 Proposal

[Vijay Bellur]

On Sat, Mar 18, 2017 at 7:27 AM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

>
>
> On Thu, Mar 16, 2017 at 7:42 AM, Vijay Bellur  wrote:
>
>> Hi All,
>>
>> We have been working on a proposal [1] to make the lifecycle management
>> of Gluster maintainers more structured. We intend to make the proposal
>> effective around 3.11 (May 2016).
>>
>> Please review the proposal and let us know your feedback. If you need
>> clarity on any existing aspect or feel the need for something additional in
>> the proposal, please feel free to let us know.
>>
>
>
>-
>
>It’s okay to drop a component if they are not able to find
>time/energy. Owners are requested to minimize disruption to the project by
>helping with transitions and announcing their intentions.
>
> How and to who should it be communicated that a owner/peer is doing a bad
> job and there are better alternatives for the component?
>
>
All feedback should be directed to the project and community leaders. At
this point in time, please direct any feedback (both positive and negative)
to Amye, Jeff and me.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Maintainers 2.0 Proposal

[Vijay Bellur]

On Sat, Mar 18, 2017 at 7:17 AM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

>
>
> On Sat, Mar 18, 2017 at 1:20 AM, Amar Tumballi 
> wrote:
>
>> I don't want to take the discussions in another direction, but want
>> clarity on few things:
>>
>> 1. Does maintainers means they are only reviewing/ merging patches?
>> 2. Should maintainers be responsible for answering ML / IRC questions
>> (well, they should focus more on documentation IMO).
>> 3. Who's responsibility is it to keep the gluster.org webpage? I
>> personally feel the responsibility should be well defined.
>> 4. Can a component have more than 1 owner ? 1 maintainers etc?
>>
>
> More than 1 maintainer is the best case we should aim for. I see EC
> benefit tremendously because of this. Work keeps moving because at least
> one of Xavi/I are available for discussions.
>


The number of maintainers will largely be a function of the complexity of
the component and the nature of ongoing work (features being designed,
patches for review, outstanding technical debt). The intent of this
exercise is not to disturb status quo for components where we do not have
problems.

Some of these questions should be clearly answered in document IMO.
>>
>
+1. We are evolving a project governance document and let us strive to
evolve as much clarity needed before making it effective.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Maintainers 2.0 Proposal

[Vijay Bellur]

Hi All,

We have been working on a proposal [1] to make the lifecycle management 
of Gluster maintainers more structured. We intend to make the proposal 
effective around 3.11 (May 2016).


Please review the proposal and let us know your feedback. If you need 
clarity on any existing aspect or feel the need for something additional 
in the proposal, please feel free to let us know.


Thanks!
Amye & Vijay

[1]  https://hackmd.io/s/SkwiZd4qe
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Demo in community meetings

[Vijay Bellur]

On Wed, Mar 15, 2017 at 12:33 PM, Vijay Bellur  wrote:

> On 03/14/2017 07:10 AM, Prasanna Kalever wrote:
>
>> Thanks for the opportunity.
>>
>> I will be happy to stream a demo on 'howto gluster-block' tomorrow.
>>
>> --
>> Prasanna
>>
>> On Mon, Mar 13, 2017 at 8:45 AM, Vijay Bellur  wrote:
>>
>>> Hi All,
>>>
>>> In the last meeting of maintainers, we discussed about reserving 15-30
>>> minutes in the community meeting for demoing new functionalities on
>>> anything
>>> related to Gluster. If you are working on something new or possess
>>> specialized knowledge of some intricate functionality, then this slot
>>> would
>>> be a great opportunity for sharing that with the community and obtaining
>>> real time feedback from seasoned Gluster folks in the meeting.
>>>
>>> Given that the slot is for 15-30 minutes, we would be able to accommodate
>>> 1-2 demos per meeting. This demo will happen over bluejeans and the URL
>>> would be available in the agenda for the meeting. If you are interested
>>> in
>>> kickstarting the demo series this week, please respond on this thread and
>>> let us know.
>>>
>>>
>
> Thank you Prasanna for your presentation and demo of gluster-block!
>
> Recording of the session can be found at [1].
>
>
Looks like the earlier link requires a login. Please use the new URL [2]
for accessing the sesion.

Thanks!
Vijay

[2] https://goo.gl/41mV3c
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Demo in community meetings

[Vijay Bellur]

On 03/14/2017 07:10 AM, Prasanna Kalever wrote:

Thanks for the opportunity.

I will be happy to stream a demo on 'howto gluster-block' tomorrow.

--
Prasanna

On Mon, Mar 13, 2017 at 8:45 AM, Vijay Bellur  wrote:

Hi All,

In the last meeting of maintainers, we discussed about reserving 15-30
minutes in the community meeting for demoing new functionalities on anything
related to Gluster. If you are working on something new or possess
specialized knowledge of some intricate functionality, then this slot would
be a great opportunity for sharing that with the community and obtaining
real time feedback from seasoned Gluster folks in the meeting.

Given that the slot is for 15-30 minutes, we would be able to accommodate
1-2 demos per meeting. This demo will happen over bluejeans and the URL
would be available in the agenda for the meeting. If you are interested in
kickstarting the demo series this week, please respond on this thread and
let us know.




Thank you Prasanna for your presentation and demo of gluster-block!

Recording of the session can be found at [1].

Regards,
Vijay

[1] https://goo.gl/Gbhrkd

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Demo in community meetings

[Vijay Bellur]

On Tue, Mar 14, 2017 at 7:10 AM, Prasanna Kalever 
wrote:

> Thanks for the opportunity.
>
> I will be happy to stream a demo on 'howto gluster-block' tomorrow.
>


Thank you, Prasanna. We will have the demo streamed at [1].

Regards,
Vijay

[1] https://bluejeans.com/102845329
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Dashboards

[Vijay Bellur]

On Mon, Mar 13, 2017 at 8:30 AM, Atin Mukherjee  wrote:

> This is super cool, Nigel!
>


+1. Thank you, Nigel!

-Vijay


>
> On Mon, Mar 13, 2017 at 4:40 PM, Nigel Babu  wrote:
>
>> Hello folks,
>>
>> We had this discussion a while ago about having dashboards inside Gerrit.
>> I tried to get it working and couldn't back then. I've finally got it
>> working.
>> It seems to be very useful for release maintainers after a release has
>> been
>> branched. I have two dashboards for 3.10 and 3.8, both of which are our
>> currently maintained releases. I've used the query that Shyam provided
>> for 3.10
>> and copied that over for 3.8 and master.
>>
>> The master dashboard is a little noisy because we tend not to abandon
>> merges.
>> This means there's plenty of patches over the years that are still lying
>> around. I could set an age limit for the query, but that will only make it
>> easier for patches to slip through the cracks. If you have a patch there
>> that
>> you don't want to pursue anymore, please abandon them.
>>
>> https://review.gluster.org/#/admin/projects/glusterfs,dashboards
>>
>> --
>> nigelb
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
>
> ~ Atin (atinm)
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Demo in community meetings

[Vijay Bellur]

Hi All,

In the last meeting of maintainers, we discussed about reserving 15-30
minutes in the community meeting for demoing new functionalities on
anything related to Gluster. If you are working on something new or possess
specialized knowledge of some intricate functionality, then this slot would
be a great opportunity for sharing that with the community and obtaining
real time feedback from seasoned Gluster folks in the meeting.

Given that the slot is for 15-30 minutes, we would be able to accommodate
1-2 demos per meeting. This demo will happen over bluejeans and the URL
would be available in the agenda for the meeting. If you are interested in
kickstarting the demo series this week, please respond on this thread and
let us know.

Thanks!
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Snapshot maintainer

[Vijay Bellur]

Hi Rajesh,

Thank you for your contributions to snapshot and other areas of Gluster! I
wish you luck in your future endeavors and look forward to your continued
presence in the community.

All,

Raghavendra Bhat has agreed to be the interim maintainer for both snapshots
and user serviceable snapshots.  Please work with Raghu for all activities
related to snapshots.

Regards,
Vijay


On Tue, Mar 7, 2017 at 5:42 AM, Rajesh Joseph  wrote:

> Hi all,
>
> I have taken an opportunity which will take considerable amount of my
> time. Therefore I may not be able to spend the desired time as a
> maintainer. Because of which I would require a new maintainer for
> snapshot to replace me. I will continue as a contributor to the
> feature and Gluster in general.
>
> Best Regards,
> Rajesh
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gogfapi improvements

[Vijay Bellur]

On Wed, Mar 8, 2017 at 3:56 PM, Ben Werthmann  wrote:

> Hello Gluster Integration,
>
> We've improved gogfapi and received the approval to make the changes
> public today. I've opened a PR with upstream. Please take a look if this is
> of interest.
>
> https://github.com/gluster/gogfapi/pull/22
>
>
>

This is cool and is useful for go gfapi bindings. Thank you for
contributing this PR!

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Untriaged FB-Forwardport bugs

[Vijay Bellur]

On Tue, Mar 7, 2017 at 7:27 AM, Niels de Vos  wrote:

> Hi Vijay,
>
> It seems that there are 50+ new bugs reported with the 'private bug'
> feature 'development whiteboard' set to FB-Forwardport. All of these
> bugs are filed against the 'core' component, which is clearly wrong for
> many of them. This causes quite a lot of work for the maintainers of teh
> core component that should be triaging these bugs. Many have fallen
> through and showed up on todays Bug Triaege meeting.
>
> We have decided not to triage the bugs during the meeting, and hope that
> the people working on these changes will triage the bugs themselves
> ASAP. Please follow the usual triaging procedures (and by extension the
> reporting guidelines) for the bugs listed below.
>
>   https://gluster.readthedocs.io/en/latest/Contributors-Guide/Bug-Triage/
>   https://gluster.readthedocs.io/en/latest/Contributors-
> Guide/Bug-Reporting-Guidelines/
>
>
Niels and others: My intent has been not to create additional work for any
of you. I am trying to expedite outstanding patches from Facebook as it is
one of our core focus areas for 3.11.  There is a significant backlog of
patches that need to be pulled in to master and I am automating as much as
possible to clear the backlog. Owing to automation there could be some more
rough edges than creating bugs manually. I appreciate any help you can
provide in overcoming the rough edges & in taking this activity to
completion.

I did spend a few minutes today to clean up the bugzilla reports. If you
would like to see further improvements in the reports, please let me know.
To get more karma from me, please go ahead and improve such bug reports on
your own :-).

Thanks!
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Build failed in Jenkins: regression-test-burn-in #2734

[Vijay Bellur]

Hey All,

A patch [1] that I pushed to master on behalf of Shreyas has caused our
regression burn-in tests to fail over the weekend. There is another patch
from Shreyas [2] that fixes the problem and is currently awaiting votes
from CentOS & NetBSD regressions. My bad that we did not end up getting
both these patches in the repo at the same time.

Once patch [2] is pulled in, we should be back to normal. Apologies for any
inconvenience caused!

Thanks,
Vijay

[1]  https://review.gluster.org/#/c/16797/

[2] https://review.gluster.org/#/c/16802/



On Sun, Mar 5, 2017 at 5:54 PM,  wrote:

> See  display/redirect>
>
> --
> [...truncated 13078 lines...]
> ok 39, LINENUM:121
> ok 40, LINENUM:122
> ok 41, LINENUM:123
> ok 42, LINENUM:124
> ok 43, LINENUM:127
> Failed 1/43 subtests
>
> Test Summary Report
> ---
> ./tests/bugs/distribute/bug-1161311.t (Wstat: 0 Tests: 43 Failed: 1)
>   Failed test:  27
> Files=1, Tests=43, 139 wallclock secs ( 0.02 usr  0.00 sys +  1.54 cusr
> 98.23 csys = 99.79 CPU)
> Result: FAIL
> End of test ./tests/bugs/distribute/bug-1161311.t
> 
> 
>
>
> Run complete
> 
> 
> Number of tests found: 221
> Number of tests selected for run based on pattern: 221
> Number of tests skipped as they were marked bad:   8
> Number of tests skipped because of known_issues:   4
> Number of tests that were run: 209
>
> 1 test(s) failed
> ./tests/bugs/distribute/bug-1161311.t
>
> 0 test(s) generated core
>
>
> Tests ordered by time taken, slowest to fastest:
> 
> 
> ./tests/basic/tier/tier.t  -  495 second
> ./tests/basic/ec/ec-12-4.t  -  359 second
> ./tests/basic/ec/ec-7-3.t  -  217 second
> ./tests/basic/ec/ec-6-2.t  -  183 second
> ./tests/basic/afr/entry-self-heal.t  -  162 second
> ./tests/basic/afr/split-brain-favorite-child-policy.t  -  161 second
> ./tests/basic/ec/ec-5-2.t  -  158 second
> ./tests/bugs/distribute/bug-1125824.t  -  157 second
> ./tests/basic/ec/ec-5-1.t  -  156 second
> ./tests/basic/afr/self-heal.t  -  146 second
> ./tests/bugs/distribute/bug-1161311.t  -  143 second
> ./tests/basic/ec/ec-4-1.t  -  132 second
> ./tests/basic/tier/legacy-many.t  -  126 second
> ./tests/basic/ec/ec-optimistic-changelog.t  -  124 second
> ./tests/basic/afr/self-heald.t  -  113 second
> ./tests/bugs/distribute/bug-1117851.t  -  110 second
> ./tests/bugs/core/bug-1110917.t  -  109 second
> ./tests/basic/ec/ec-3-1.t  -  102 second
> ./tests/basic/volume-snapshot-clone.t  -  86 second
> ./tests/basic/tier/new-tier-cmds.t  -  86 second
> ./tests/basic/afr/split-brain-heal-info.t  -  84 second
> ./tests/basic/afr/sparse-file-self-heal.t  -  83 second
> ./tests/basic/ec/ec-new-entry.t  -  81 second
> ./tests/basic/ec/heal-info.t  -  80 second
> ./tests/basic/afr/metadata-self-heal.t  -  79 second
> ./tests/basic/afr/split-brain-healing.t  -  77 second
> ./tests/basic/ec/ec-notify.t  -  75 second
> ./tests/basic/ec/ec-background-heals.t  -  74 second
> ./tests/bugs/bug-1368312.t  -  68 second
> ./tests/basic/quota.t  -  63 second
> ./tests/basic/afr/granular-esh/cli.t  -  63 second
> ./tests/basic/ec/self-heal.t  -  60 second
> ./tests/basic/uss.t  -  57 second
> ./tests/bugs/cli/bug-770655.t  -  56 second
> ./tests/basic/volume-snapshot.t  -  54 second
> ./tests/basic/tier/fops-during-migration-pause.t  -  52 second
> ./tests/basic/afr/quorum.t  -  52 second
> ./tests/basic/tier/frequency-counters.t  -  50 second
> ./tests/basic/mount-nfs-auth.t  -  47 second
> ./tests/basic/afr/inodelk.t  -  47 second
> ./tests/basic/ec/ec.t  -  45 second
> ./tests/basic/ec/ec-readdir.t  -  45 second
> ./tests/basic/afr/gfid-self-heal.t  -  42 second
> ./tests/basic/afr/arbiter.t  -  42 second
> ./tests/basic/ec/ec-cpu-extensions.t  -  41 second
> ./tests/basic/mpx-compat.t  -  39 second
> ./tests/basic/tier/tier-heald.t  -  37 second
> ./tests/bitrot/bug-1294786.t  -  36 second
> ./tests/bitrot/br-state-check.t  -  36 second
> ./tests/basic/tier/locked_file_migration.t  -  36 second
> ./tests/basic/afr/granular-esh/conservative-merge.t  -  36 second
> ./tests/basic/volume-snapshot-xml.t  -  35 second
> ./tests/basic/tier/unlink-during-migration.t  -  35 second
> ./tests/basic/geo-replication/marker-xattrs.t  -  32 second
> ./tests/bugs/core/bug-1402841.t-mt-dir-scan-race.t  -  28 second
> ./tests/basic/quota-ancestry-building.t  -  28 second
> ./tests/basic/afr/data-self-heal.t  -  28 second
> ./tests/basic/afr/arbiter-mount.t  -  28 second
> ./tests/bugs/cli/bug-1353156-get-state-cli-validations.t  -  27 second
> ./tests/basic/afr/split-brain-resolution.t  -  27 second
> ./tests/basic/afr/durability-off.t  -  27 second
> 

Re: [Gluster-devel] Solaris support ?

[Vijay Bellur]

On Wed, Feb 22, 2017 at 2:22 PM, Michael Scherer 
wrote:

> Hi,
>
> I see that we still have solaris code in the repo, and I was wondering
> if we should either:
> - drop it, because solaris is likely dead
>
> - start to at least verify compilation there, and so need to install
> some open solaris host somewhere
>
> Any opinions ?
>
>

Let us drop it. There's no point in carrying around that code anymore.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Reviews needed

[Vijay Bellur]

On Thu, Feb 16, 2017 at 3:43 AM, Xavier Hernandez 
wrote:

> Hi everyone,
>
> I would need some reviews if you have some time:
>
> A memory leak fix in fuse:
> * Patch already merged in master and 3.10
> * Backport to 3.9: https://review.gluster.org/16402
> * Backport to 3.8: https://review.gluster.org/16403
>
> A safe fallback for dynamic code generation in EC:
> * Master: https://review.gluster.org/16614
>
> A fix for incompatibilities with FreeBSD:
> * Master: https://review.gluster.org/16417
>
> A fix for FreeBSD's statvfs():
> * Patch already merged in master
> * Backport to 3.10: https://review.gluster.org/16631
> * Backport to 3.9: https://review.gluster.org/16632
> * Backport to 3.8: https://review.gluster.org/16634
>
> I also have two reviews for 3.7 but I think it won't have any new
> releases, right ?
>
>
That is right. We can also avoid additional backports to 3.9 as it is going
to be EOL once 3.10 is out.

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Storage/posix: syscalls done holding inode->lock

[Vijay Bellur]

On Mon, Feb 6, 2017 at 4:30 AM, Raghavendra Gowdappa 
wrote:

> Hi all,
>
> Storage/posix does syscalls on backend filesystem holding inode->lock.
> This is bad as it is a lock global to the inode and can cause unnecessary
> contention with unrelated translators doing unrelated operations (like
> inode_ctx_get etc). I've discussed one such issue in [2]. A quick git grep
> on "inode->lock" in storage/posix gave following results:
>
> * posix_writev -
>   GLUSTERFS_WRITE_IS_APPEND - looks like used by arbiter/afr.
>   GLUSTERFS_WRITE_UPDATE_ATOMIC - looks like used by shard
> * posix_fd_ctx_get - resolves gfid handle (which can involve multiple
> readlinks and lstats) in inode->lock. Though the cost is only once when
> fd-ctx is freshly created.
> * code that maintains pgfid xattrs - executed in various dentry
> modification fops like mkdir, create, mknod, unlink, rename, link etc
> * code that uses GET_LINK_COUNT - looks like used by shard and EC. Note
> that this code is executed during rename/unlink.
> * posix_create_link_if_gfid_exists - looks like used by afr entry selfheal
> * posix_(f)xattrop - various xlators like afr, marker during different
> fops.
>
> The question here is can we've synchronization using a lock visible only
> to storage/posix so that contention is localized (like [1])?
>
>
I think the answer depends on the degree of isolation required across
threads operating on the same inode. If the operations being done within
"inode->lock" do not cause any side effects elsewhere in the xlator stack,
we should be able to replace inode->lock with a more local lock.  At the
outset it looks like we should be able to synchronize using a smaller lock
for most cases. A more careful analysis is needed to determine if there are
scenarios where inode->lock helps.

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Creating new options for multiple gluster versions

[Vijay Bellur]

On Tue, Jan 31, 2017 at 2:36 AM, Xavier Hernandez 
wrote:

> Hi Atin,
>
> On 31/01/17 05:45, Atin Mukherjee wrote:
>
>>
>>
>> On Mon, Jan 30, 2017 at 9:02 PM, Xavier Hernandez > > wrote:
>>
>> Hi Atin,
>>
>> On 30/01/17 15:25, Atin Mukherjee wrote:
>>
>>
>>
>> On Mon, Jan 30, 2017 at 7:30 PM, Xavier Hernandez
>> mailto:xhernan...@datalab.es>
>> >>
>>
>> wrote:
>>
>> Hi,
>>
>> I'm wondering how a new option needs to be created to be
>> available
>> to different versions of gluster.
>>
>> When a new option is created for 3.7 for example, it needs
>> to have a
>> GD_OP_VERSION referencing the next 3.7 release. This ensures
>> that
>> there won't be any problem with previous versions.
>>
>> However what happens with 3.8 ?
>>
>> 3.8.0 is greater than any 3.7.x, however the new option won't
>> be
>> available until the next 3.8 release. How this needs to be
>> handled ?
>>
>>
>> I'd discourage to backport any new volume options from mainline
>> to the
>> stable releases branches like 3.7 & 3.8. This creates a lot of
>> backward
>> compatibility issues w.r.t clients. Any new option is actually
>> an RFE
>> and supposed to be slated for only upcoming releases.
>>
>>
>> Even if it's needed to solve an issue in all versions ?
>>
>> For example, a hardcoded timeout is seen to be insufficient in some
>> configurations, so it needs to be increased, but increasing it will
>> be too much for many of the environments where the current timeout
>> has worked fine. It could even be not enough for other environments
>> still not tried, needed a future increase.
>>
>> With a new option, this can be solved case by case and only when
>> needed.
>>
>> How can this be solved ?
>>
>>
>> Hi Xavi,
>>
>> Let me try to explain this a bit in detail. A new option with an
>> op-version say 30721 (considering 3.7.21 is the next update of 3.7 which
>> is the oldest active branch) is introduced in mainline and then the same
>> is backported to 3.7 (slated for 3.7.21) &  3.8 branch (slated for
>> 3.8.9).  Now say if an user forms a cluster of three nodes with gluster
>> versions as 3.7.21, 3.8.9 & 3.8.8 respectively and tries to set this
>> option, volume set would always fail as in 3.8.8 this option is not
>> defined. Also any client running with 3.8 version would see a
>> compatibility issue here. Also the op-version number of the new option
>> has to be same across different release branches.
>>
>
> Thanks for the explanation. This confirms what I already thought. So the
> question is: now that 3.10 has already been branched, does it mean that any
> new option won't be available for LTS users until 3.12 is released ? I
> think this is not acceptable, specially for changes intended to fix an
> issue, not introducing new features.
>
>
>> With the current form of op-version management, I don't think this can
>> be solved, the only way is to ask users to upgrade to the latest.
>>
>
> As I said, someone using 3.10 LTS won't be able to upgrade until 3.12 is
> released. What would we say to them when we add a new option to 3.11 ?
>
> Maybe we should add a new kind of option that causes no failure if not
> recognized. They are simply ignored. Many options do not cause any visible
> functional change, so they could be defined even if some nodes of the
> cluster don't recognize them (for example performance improvement options
> or some timeout values).
>


I like this idea. This will give us some flexibility for defining options.

-Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel