[Gluster-devel] Maintainers 2.0 Proposal

[Vijay Bellur]

Hi All,

We have been working on a proposal [1] to make the lifecycle management 
of Gluster maintainers more structured. We intend to make the proposal 
effective around 3.11 (May 2016).


Please review the proposal and let us know your feedback. If you need 
clarity on any existing aspect or feel the need for something additional 
in the proposal, please feel free to let us know.


Thanks!
Amye & Vijay

[1]  https://hackmd.io/s/SkwiZd4qe
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Consistent time attributes (ctime, atime and mtime) across replica set and distribution set

[Pranith Kumar Karampuri]

On Wed, Mar 15, 2017 at 11:31 PM, Soumya Koduri  wrote:

> Hi Rafi,
>
> I haven't thoroughly gone through design. But have few comments/queries
> which I have posted inline for now .
>
> On 02/28/2017 01:11 PM, Mohammed Rafi K C wrote:
>
>> Thanks for the reply , Comments are inline
>>
>>
>>
>> On 02/28/2017 12:50 PM, Niels de Vos wrote:
>>
>>> On Tue, Feb 28, 2017 at 11:21:55AM +0530, Mohammed Rafi K C wrote:
>>>
 Hi All,


 We discussed the problem $subject in the mail thread [1]. Based on the
 comments and suggestions I will summarize the design (Made as points for
 simplicity.)


 1) As part of each fop, top layer will generate a time stamp and pass it
 to the down along with other param.

 1.1) This will bring a dependency for NTP synced clients along with
 servers

>>> What do you mean with "top layer"? Is this on the Gluster client, or
>>> does the time get inserted on the bricks?
>>>
>> It is the top layer (master xlator) in client graph like fuse, gfapi,
>> nfs . My mistake I should have mentioned . Sorry for that.
>>
>
> These clients shouldn't include internal client processes like rebalance,
> self-heal daemons right? IIUC from [1], we should avoid changing times
> during rebalance and self-heals.
>
> Also what about fops generated from the underlying layers -
> getxattr/setxattr which may modify these time attributes?
>
>
>>
>>
>>> I think we should not require a hard dependency on NTP, but have it
>>> strongly suggested. Having a synced time in a clustered environment is
>>> always helpful for reading and matching logs.
>>>
>> Agreed, but if we go with option 1 where we generate time from client,
>> then time will not be in sync if not done with NTP.
>>
>>
>>
>>
>>> 1.2) There can be a diff in time if the fop stuck in the xlator for
 various reason, for ex: because of locks.

>>> Or just slow networks? Blocking (mandatory?) locks should be handled
>>> correctly. The time a FOP is blocked can be long.
>>>
>> True, the questions can this be included in timestamp valie, because if
>> it generated from say fuse then when it reaches to the brick the time
>> may have moved ahead. what do you think about it ?
>>
>>
>>
>>> 2) On the server posix layer stores the value in the memory (inode ctx)
 and will sync the data periodically to the disk as an extended attr

>>> Will you use any timer thread for asynchronous update?
>
>
  2.1) of course sync call also will force it. And fop comes for an
 inode which is not linked, we do the sync immediately.

>>> Does it need to be in the posix layer?
>>>
>>
>> You mean storing the time attr ? then it need not be , protocol/server
>> is also another candidate but I feel posix is ahead in the race ;) .
>>
>
> I agree with Shyam and Niels that posix layer doesn't seem right. Since
> having this support comes with performance cost, how about a separate
> xlator (which shall be optional)?
>
>
>>
>>
>>> 3) Each time when inodes are created or initialized it read the data
 from disk and store it.


 4) Before setting to inode_ctx we compare the timestamp stored and the
 timestamp received, and only store if the stored value is lesser than
 the current value.

>>> If we choose not to set this attribute for self-heal/rebalance (as
> stated above) daemons, we would need special handling for the requests sent
> by them (i.e, to heal this time attribute as well on the destination
> file/dir).
>
>

 5) So in best case data will be stored and retrieved from the memory. We
 replace the values in iatt with the values in inode_ctx.


 6) File ops that changes the parent directory attr time need to be
 consistent across all the distributed directories across the subvolumes.
 (for eg: a create call will change ctime and mtime of parent dir)

  6.1) This has to handle separately because we only send the fop to
 the hashed subvolume.

  6.2) We can asynchronously send the timeupdate setattr fop to the
 other subvoumes and change the values for parent directory if the file
 fops is successful on hashed subvolume.

>>>
> The same needs to be handled even during DHT directory healing right?
>
>
  6.3) This will have a window where the times are inconsistent
 across dht subvolume (Please provide your suggestions)

>>> Isn't this the same problem for 'normal' AFR volumes? I guess self-heal
>>> needs to know how to pick the right value for the [cm]time xattr.
>>>
>>
>> Yes and need to heal. Both self heal and dht. But till then there can be
>> difference in values.
>>
>
> Is this design targetting to synchronize only ctime/mtime? If 'atime' is
> also considered , as the read/stat done by AFR shall modify atime only on
> the first subvol, even AFR xlator needs to take care of updating other
> subvols. Same goes with EC as well.
>

atime is updated on open which 

Re: [Gluster-devel] add glusterfs support into xfstests

[Niels de Vos]

Hi Zorro,

Many thanks for doing this! I'm adding gluster-devel@gluster.org as that
is the list where all Gluster developers are active. At this moment I
can not think of anything specific (but it is rather late). I'll go
through your patch tomorrow and will think about it a little more.

Thanks again,
Niels


On Thu, Mar 16, 2017 at 01:11:34AM +0800, Zorro Lang wrote:
> Hi Niels,
> 
> We already contributed many patches to xfstests, most of them are
> XFS related. Due to there're more and more FUSE bugs will be merged
> in future RHEL kernel, I decide to try to make xfstests support
> glusterfs test.
> 
> The first patch as below (reviewing):
> http://marc.info/?l=fstests=148959701527695=2
> 
> It will add basic GlusterFS support, we still need long time to
> modify some case problems which will cause failures on glusterfs.
> 
> As you know, I only know how to use "mount -t glusterfs $IP:vol",
> so this patch only to do this mount and test. Is there anything
> else you feel important to glusterfs? Likes your project about
> NFS on glusterfs, is there any difference when use it?
> 
> I'm glad to hear your suggestions as usual :)
> 
> Thanks,
> Zorro


signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] About DHT layout

[Tahereh Fattahi]

Thanks a lot, but I need more details. Can you tell me exactly which
function assigns the hash to directories on bricks? at the end of
dht_function the dht_layout->list does not have the value of hash, so where
this assignment is done?

On Wed, Mar 15, 2017 at 8:11 PM, Ankit Raj  wrote:

>
>
> On Wed, Mar 15, 2017 at 8:43 PM, Tahereh Fattahi 
> wrote:
>
>> Hi
>>
>
> DHT creates directories on all the bricks in the volume. And, their
> range varies from  to  0x and and each brick is assigned a
> specific subset of this range. Also, the directory creation and the
>setting of layout is part of mkdir operation.
>
> I want to know when and where the layout of a directory is created?
>>
>
>
>> I read more and the code of dht_mkdir and the functions were called  in
>> this function but I could not find the primary part (setting the list field
>> in dht_layout structure), can anyone help me?
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Demo in community meetings

[Vijay Bellur]

On Wed, Mar 15, 2017 at 12:33 PM, Vijay Bellur  wrote:

> On 03/14/2017 07:10 AM, Prasanna Kalever wrote:
>
>> Thanks for the opportunity.
>>
>> I will be happy to stream a demo on 'howto gluster-block' tomorrow.
>>
>> --
>> Prasanna
>>
>> On Mon, Mar 13, 2017 at 8:45 AM, Vijay Bellur  wrote:
>>
>>> Hi All,
>>>
>>> In the last meeting of maintainers, we discussed about reserving 15-30
>>> minutes in the community meeting for demoing new functionalities on
>>> anything
>>> related to Gluster. If you are working on something new or possess
>>> specialized knowledge of some intricate functionality, then this slot
>>> would
>>> be a great opportunity for sharing that with the community and obtaining
>>> real time feedback from seasoned Gluster folks in the meeting.
>>>
>>> Given that the slot is for 15-30 minutes, we would be able to accommodate
>>> 1-2 demos per meeting. This demo will happen over bluejeans and the URL
>>> would be available in the agenda for the meeting. If you are interested
>>> in
>>> kickstarting the demo series this week, please respond on this thread and
>>> let us know.
>>>
>>>
>
> Thank you Prasanna for your presentation and demo of gluster-block!
>
> Recording of the session can be found at [1].
>
>
Looks like the earlier link requires a login. Please use the new URL [2]
for accessing the sesion.

Thanks!
Vijay

[2] https://goo.gl/41mV3c
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] About DHT layout

[Ankit Raj]

On Wed, Mar 15, 2017 at 8:43 PM, Tahereh Fattahi 
wrote:

> Hi
>

DHT creates directories on all the bricks in the volume. And, their
range varies from  to  0x and and each brick is assigned a
specific subset of this range. Also, the directory creation and the
   setting of layout is part of mkdir operation.

I want to know when and where the layout of a directory is created?
>


> I read more and the code of dht_mkdir and the functions were called  in
> this function but I could not find the primary part (setting the list field
> in dht_layout structure), can anyone help me?
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Demo in community meetings

[Vijay Bellur]

On 03/14/2017 07:10 AM, Prasanna Kalever wrote:

Thanks for the opportunity.

I will be happy to stream a demo on 'howto gluster-block' tomorrow.

--
Prasanna

On Mon, Mar 13, 2017 at 8:45 AM, Vijay Bellur  wrote:

Hi All,

In the last meeting of maintainers, we discussed about reserving 15-30
minutes in the community meeting for demoing new functionalities on anything
related to Gluster. If you are working on something new or possess
specialized knowledge of some intricate functionality, then this slot would
be a great opportunity for sharing that with the community and obtaining
real time feedback from seasoned Gluster folks in the meeting.

Given that the slot is for 15-30 minutes, we would be able to accommodate
1-2 demos per meeting. This demo will happen over bluejeans and the URL
would be available in the agenda for the meeting. If you are interested in
kickstarting the demo series this week, please respond on this thread and
let us know.




Thank you Prasanna for your presentation and demo of gluster-block!

Recording of the session can be found at [1].

Regards,
Vijay

[1] https://goo.gl/Gbhrkd

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel