Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On 10/29/2015 10:26 PM, Jeff Darcy wrote: On October 29, 2015 at 8:42:50 PM, Shyam (srang...@redhat.com) wrote: I assume this is about infra changes (as the first 2 points are for some reason squashed in my reader). I think what you state is infra (or other non-experimental) code impact due to changes by experimental/inprogress code, should be dealt with clearly and carefully so as to not impact regular functionality. In which case I *agree* and do not mean otherwise. I think this sort of goes back to what Niels commented on my squashing a .patch and proposed using #define EXPERIMENTAL in this thread (or such methods). Sort of, although I would prefer that the distinction be run-time instead of compile-time whenever possible. 3. All experimental functionality, whether in its own translator or otherwise, should be controlled by an option which is off by default. Ah! I think this is something akin to the "#define EXPERIMENTAL" suggestion and non-experimental code impact I guess, right? I think so. Also, since I wasn’t clear before, I think there should be *separate* options per feature, not one blanket “experimental” option. Agreed For example, if you want to play with DHT you’d need to: (1) Install the gluster-experimental RPM (2) Tweak the glusterd script or volfile to allow experimental features (3) Set cluster.dht2 (or whatever) on your volume Note the absence of steps to download, hand-edit, or build anything yourself. I think that’s key: no risk if you don’t go out of your way to enable experimental code, but you don’t have to be a full-time Gluster developer to walk on the wild side. Agreed. One more question, should we package all experimental code in the future, or things that reach a certain level of experimental maturity? As an example, say DHT2 has 3 FOPs only implemented when we do *some release*, should we package it, or leave it behind? Asking as I am unsure of the course, or maybe a consideration for the future. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On 10/29/2015 01:42 AM, Avra Sengupta wrote: My 2 cents on this: The decision we take on this should have certain rationale, and I see two key things affecting that decision. 1. How much of the code we are planning to write now, is going to make it to the final product. If we are sure that a sizeable amount of code we are writing now, is gonna change over the course of time, then it makes sense to have that kinda development in experimental branch. 2. Is the code we are planning to deliver meddle with existing infrastructure. If so, then it should definitely go into experimental Now analyzing NSR based on the above two constraints : 1. We definitely plan to use more than 80% of the code we write now, and plan to go about it in an incremental module by module kinda way. 2. We hardly have any overlap with existing infrastructure, and we can easily integrate with the current glusterd now, and move on to Glusterd 2, as and when it is ready for consumption. Based on the above analysis, I would say NSR can go right into master, and we can easily make sure that it's not built as part of any release. Now what NSR would follow, isn;t necessary for other translators to follow. For eg. I would think Glusterd2 would have to be in experimental coz it might meddle with current glusterd (but Atin would know better). The point being, the decision we take need not be a collective decision for all components, that would be enforced as a process, but rather should be a decision taken by individual components based on merit. Will code that NSR puts up in master be ready to ship when 3.8 is branched? I ask the above, as I think we need a *process*, and not an open ended "put it where you want option", for the following reasons, - Something that ships experimental is not to be consumed in production, so till the point an/any effort is ready it can be housed in experimental - It is not about how much of infra code is impacted, or how much code would change, it is about readiness of the functionality - The intention is also to keep such WIP xlators segregated in master - It is easy to control the "don't build/ship" directive for everything under experimental, rather than make these decisions at a per directory/xlator/module level - It is a clear message for anyone picking up these pieces to deploy and play with etc. Coming to DHT2, irrespective of reason (1) above, all other reasons for which NSR can stay outside of experimental holds good for DHT2 as well. I would leave others to comment if NSR gets into experimental or not, and what is the due diligence we need in this case. So are we going ahead with experimental as a process? I am punting this to Vijay and Jeff :) On 10/28/2015 08:32 PM, Shyam wrote: Sending this along again. - Are we decided on *experimental*? - If so, what else needs attention in the patch below? - (Re)views please... (views as in "What are your views on this?", not "Have you viewed this?" ;) ) Shyam On 10/12/2015 02:18 PM, Shyam wrote: In an effort to push this along, update the change with suggested edits and comments till now, request a review and further comments so that we make this official at some (sooner) point in time. http://review.gluster.org/#/c/12321/2 ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On 10/29/2015 07:29 PM, Jeff Darcy wrote: On October 29, 2015 at 9:12:46 AM, Shyam (srang...@redhat.com) wrote: Will code that NSR puts up in master be ready to ship when 3.8 is branched? Do we know when 3.8 will be branched? This is an example not absolute, what I mean is when the next release is made. I ask the above, as I think we need a *process*, and not an open ended "put it where you want option", for the following reasons, - Something that ships experimental is not to be consumed in production, so till the point an/any effort is ready it can be housed in experimental - It is not about how much of infra code is impacted, or how much code would change, it is about readiness of the functionality I disagree. From an operational perspective, unavoidable impact (not just on infrastructure but on any existing functionality) is *the* thing we want to avoid. Experimental code that sits on disk, but isn’t even loaded into memory without an explicit request to enable it, carries little risk. Experimental code that’s loaded and executed every time Gluster is run carries more risk, even if it’s just one line. I assume this is about infra changes (as the first 2 points are for some reason squashed in my reader). I think what you state is infra (or other non-experimental) code impact due to changes by experimental/inprogress code, should be dealt with clearly and carefully so as to not impact regular functionality. In which case I *agree* and do not mean otherwise. I think this sort of goes back to what Niels commented on my squashing a .patch and proposed using #define EXPERIMENTAL in this thread (or such methods). - The intention is also to keep such WIP xlators segregated in master - It is easy to control the "don't build/ship" directive for everything under experimental, rather than make these decisions at a per directory/xlator/module level They’re likely to make those decisions at the finer granularity anyway. Most people will probably only want to try out one experimental translator at a time. Making them edit two makefiles (to enable “experimental” and then a specific translator within it) instead of just one doesn’t seem to get us very far. Either way they’ll have to edit the specfile, and they’ll see a list of all the experimental translators they could enable. - It is a clear message for anyone picking up these pieces to deploy and play with etc. If having to edit makefiles and specfiles by hand isn’t enough, there are other things we could do to send such a clear message. For example, we could require a special configure flag or glusterd startup option to enable management support for experimental features - not just whole translators but options within existing translators as well. Coming to DHT2, irrespective of reason (1) above, all other reasons for which NSR can stay outside of experimental holds good for DHT2 as well. Perhaps. I think that’s pretty clearly true for the DHT2 translator itself. For the associated posix changes it’s less clearly true, but the modified version could go in storage/posix2 as easily as experimental/posix or experimental/posix2. Yes, posix2 is where the new posix code would land, hence the comment on DHT2 being mostly similar in nature. IMO the main reason to have an “experimental” directory is one not mentioned yet - to put them in a separate RPM. This is not the same as your “don’t build/ship” point above because they’ll get *built* anyway. I think I should word my response more clearly, as "don't build/ship" is taken literally. The choice of experimental as a separate entity, makes some of the choices presented below easier to implement/follow IMHO, which is what I am getting at. Another thing I am getting at is, we *should* define a clear way to do such things, and not leave it open ended, which is where we seem to be headed below. They’ll just be separately installable - or not, for people who aren’t interested in experimental code. Then we could combine that with the special glusterd startup option mentioned above to make the wole process of enabling or disabling experimental translators pretty seamless. Install the RPM, tweak the option, and you can use experimental code. Otherwise you can’t, and you get a nice clear error message if you try. So, here's what I think we should do (right now - subject to change). 1. We should create an "experimental" directory, and update makefiles accordingly. I am going to point back to the patch around this here, as it contains a README as well, which we can put these specifics into, http://review.gluster.org/#/c/12321/2 We can further that, or create a new one, I am fine either way. 2. The main Gluster 4.0 translators - dht2+posix, nsr* - should go into the experimental directory and the specfile should put those translators into a new RPM. 3. All experimental functionality, whether in its own translator or otherwise,
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On October 29, 2015 at 9:12:46 AM, Shyam (srang...@redhat.com) wrote: > Will code that NSR puts up in master be ready to ship when 3.8 is > branched? Do we know when 3.8 will be branched? > I ask the above, as I think we need a *process*, and not an open ended > "put it where you want option", for the following reasons, - Something > that ships experimental is not to be consumed in production, so till > the point an/any effort is ready it can be housed in experimental - It > is not about how much of infra code is impacted, or how much code > would change, it is about readiness of the functionality I disagree. From an operational perspective, unavoidable impact (not just on infrastructure but on any existing functionality) is *the* thing we want to avoid. Experimental code that sits on disk, but isn’t even loaded into memory without an explicit request to enable it, carries little risk. Experimental code that’s loaded and executed every time Gluster is run carries more risk, even if it’s just one line. > - The intention is also to keep such WIP xlators segregated in master > - It is easy to control the "don't build/ship" directive for > everything under experimental, rather than make these decisions at a > per directory/xlator/module level They’re likely to make those decisions at the finer granularity anyway. Most people will probably only want to try out one experimental translator at a time. Making them edit two makefiles (to enable “experimental” and then a specific translator within it) instead of just one doesn’t seem to get us very far. Either way they’ll have to edit the specfile, and they’ll see a list of all the experimental translators they could enable. > - It is a clear message for anyone picking up these pieces to deploy > and play with etc. If having to edit makefiles and specfiles by hand isn’t enough, there are other things we could do to send such a clear message. For example, we could require a special configure flag or glusterd startup option to enable management support for experimental features - not just whole translators but options within existing translators as well. > Coming to DHT2, irrespective of reason (1) above, all other reasons > for which NSR can stay outside of experimental holds good for DHT2 as > well. Perhaps. I think that’s pretty clearly true for the DHT2 translator itself. For the associated posix changes it’s less clearly true, but the modified version could go in storage/posix2 as easily as experimental/posix or experimental/posix2. IMO the main reason to have an “experimental” directory is one not mentioned yet - to put them in a separate RPM. This is not the same as your “don’t build/ship” point above because they’ll get *built* anyway. They’ll just be separately installable - or not, for people who aren’t interested in experimental code. Then we could combine that with the special glusterd startup option mentioned above to make the wole process of enabling or disabling experimental translators pretty seamless. Install the RPM, tweak the option, and you can use experimental code. Otherwise you can’t, and you get a nice clear error message if you try. So, here's what I think we should do (right now - subject to change). 1. We should create an "experimental" directory, and update makefiles accordingly. 2. The main Gluster 4.0 translators - dht2+posix, nsr* - should go into the experimental directory and the specfile should put those translators into a new RPM. 3. All experimental functionality, whether in its own translator or otherwise, should be controlled by an option which is off by default. 4. Configuring an experimental feature should require a special glusterd flag (plus installation of the experimental RPM). ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On October 29, 2015 at 8:42:50 PM, Shyam (srang...@redhat.com) wrote: > I assume this is about infra changes (as the first 2 points are for > some reason squashed in my reader). I think what you state is infra > (or other non-experimental) code impact due to changes by > experimental/inprogress code, should be dealt with clearly and > carefully so as to not impact regular functionality. In which case I > *agree* and do not mean otherwise. > > I think this sort of goes back to what Niels commented on my squashing > a .patch and proposed using #define EXPERIMENTAL in this thread (or > such methods). Sort of, although I would prefer that the distinction be run-time instead of compile-time whenever possible. > > 3. All experimental functionality, whether in its own translator or > > otherwise, should be controlled by an option which is off by > > default. > > Ah! I think this is something akin to the "#define EXPERIMENTAL" > suggestion and non-experimental code impact I guess, right? I think so. Also, since I wasn’t clear before, I think there should be *separate* options per feature, not one blanket “experimental” option. For example, if you want to play with DHT you’d need to: (1) Install the gluster-experimental RPM (2) Tweak the glusterd script or volfile to allow experimental features (3) Set cluster.dht2 (or whatever) on your volume Note the absence of steps to download, hand-edit, or build anything yourself. I think that’s key: no risk if you don’t go out of your way to enable experimental code, but you don’t have to be a full-time Gluster developer to walk on the wild side. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
Sending this along again. - Are we decided on *experimental*? - If so, what else needs attention in the patch below? - (Re)views please... (views as in "What are your views on this?", not "Have you viewed this?" ;) ) Shyam On 10/12/2015 02:18 PM, Shyam wrote: In an effort to push this along, update the change with suggested edits and comments till now, request a review and further comments so that we make this official at some (sooner) point in time. http://review.gluster.org/#/c/12321/2 ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
My 2 cents on this: The decision we take on this should have certain rationale, and I see two key things affecting that decision. 1. How much of the code we are planning to write now, is going to make it to the final product. If we are sure that a sizeable amount of code we are writing now, is gonna change over the course of time, then it makes sense to have that kinda development in experimental branch. 2. Is the code we are planning to deliver meddle with existing infrastructure. If so, then it should definitely go into experimental Now analyzing NSR based on the above two constraints : 1. We definitely plan to use more than 80% of the code we write now, and plan to go about it in an incremental module by module kinda way. 2. We hardly have any overlap with existing infrastructure, and we can easily integrate with the current glusterd now, and move on to Glusterd 2, as and when it is ready for consumption. Based on the above analysis, I would say NSR can go right into master, and we can easily make sure that it's not built as part of any release. Now what NSR would follow, isn;t necessary for other translators to follow. For eg. I would think Glusterd2 would have to be in experimental coz it might meddle with current glusterd (but Atin would know better). The point being, the decision we take need not be a collective decision for all components, that would be enforced as a process, but rather should be a decision taken by individual components based on merit. On 10/28/2015 08:32 PM, Shyam wrote: Sending this along again. - Are we decided on *experimental*? - If so, what else needs attention in the patch below? - (Re)views please... (views as in "What are your views on this?", not "Have you viewed this?" ;) ) Shyam On 10/12/2015 02:18 PM, Shyam wrote: In an effort to push this along, update the change with suggested edits and comments till now, request a review and further comments so that we make this official at some (sooner) point in time. http://review.gluster.org/#/c/12321/2 ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
In an effort to push this along, update the change with suggested edits and comments till now, request a review and further comments so that we make this official at some (sooner) point in time. http://review.gluster.org/#/c/12321/2 ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On 10/09/2015 12:07 AM, Atin Mukherjee wrote: First of all my apologies for not going through the meeting blog before sending my thoughts on how we plan to maintain GlusterD 2.0 [1]. This approach seems fine to me as long as we don't touch any existing xlators. How do we handle cases where other xlators get impacted by certain changes. Are we going to copy the whole translator in xlators/experimental and start working on it? Nope, we should send a change request for that xlator as a separate commit when possible. The counter example to this is, point (4) below (where DHT2 needs a bit of change in glusterd, but ...). I suggest such changes be maintained as .patch files inside the xlator, till a point when this can be merged is decided. Instead of all this wouldn't it be simpler to have development under a separate branch say "4.0-unstable" and we could disable CI on this branch till it becomes stable? Are we worried about pulling in the changes from this to master once the branch becomes stable? I guess the worry is *bulk* changes appearing in master (as per meeting minutes). I share the same concern as well (on bulk changes), but I am unsure of review stringency on experimental, as things will evolve here, than each commit be ready for a clean review from day 1. So, this is an open confusion in my head as well, as when we want to move an xlator from experimental to suported, what would be the criteria? Would we not be doing bulk reviews then as well? What do others think? This is just my thought and I would like to get a clarity on this. Thanks, Atin [1] http://www.gluster.org/pipermail/gluster-devel/2015-October/046872.html On 10/08/2015 11:35 PM, Shyam wrote: Hi, On checking yesterday's gluster meeting AIs and (later) reading the minutes, for DHT2 here is what I gather and propose to do for $SUBJECT. Feel free to add/negate any plans. (This can also be discussed at [2]) --- 1) Create a directory under the glusterfs master branch as follows, ./xlators/*experimental*/dht2 ./xlators/*experimental*/posix2 See patch request at [2] All code, design documents (work products in general) would go into this directory. 2) Code that compiles and does not cause CI failures could *potentially* be merged with very few DHT2 dev folks assent. There would possibly be no CI integration till we get something working, so merges would be based on compile passing initially. Soon there would be an attempt at getting unit testing integrated, so that code being submitted is not abysmally horrendous 3) Common framework code changes (if any) would be presented as a separate commit request 4) (Big problem) DHT2 requires glusterd changes to create a volume as DHT2 and not DHT, this would be maintained as a .patch in the dht2 directory as above. This is so that people can play with DHT2 volumes if interested. Integration of this piece either comes with glusterd 2.0 or based on time lines of other events, in the current version of glusterd. (if you are interested in seeing the current version of this patch, go here [1]) --- If there is some key disagreement on certain points like (2) above, then we would need to bring in DHT2 code in parts so that it makes sense. This is fine too, just that we would have 2 repos till we reach a point of maturity in development. --- *Some issues with the approach:* A) We need to ensure we do not ship xlators compiled from the experimental directory B) We need to possibly add a buddy maintainer for experimental translator owners, who can help with the process of merging their changes. C) I am not sure how this helps the review process, as initially xlator development can be iffy and so we do not expect reviews to be stringent. Later when we want to move this out of the experimental category, how do we review the same now, and what actions do we take to ensure quality? Won't we have the same bulk code review issue? --- Shameless plug: For quality and if an xlator plays well with other parts of gluster the distaf framework of testing against possible graphs and access protocols can be of immense help. Shyam [1] https://github.com/ShyamsundarR/glusterfs/commit/663eeb98f6a51384c8745b8882e7c6c4f7b58a7c [2] http://review.gluster.org/#/c/12321/1 ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On Friday 09 October 2015 10:39 PM, Shyam wrote: On 10/09/2015 11:26 AM, Jeff Darcy wrote: My position is that we should maximize visibility for other developers by doing all work on review.gluster.org. If it doesn't affect existing tests, it should even go on master. This includes: * Minor changes (e.g. list.h or syncop.* in http://review.gluster.org/#/c/8913/) * Refactoring that doesn't change functionality (e.g. all of http://review.gluster.org/#/c/9411/) * Translators that no existing test will enable (e.g. DHT2) It's not hard to ensure that experimental translators get built but not shipped, just by tweaking the specfile. I think it's something to do with "ghost" but maybe someone who actually knows can just answer off the top of their head before I spend 10x as much time investigating. The really sticky case is incompatible changes to permanent infrastructure, such as GlusterD 2. My preference for those to be on review.gluster.org as well, but on a branch other than master. It's tempting to make other things dependent on GlusterD 2 and put them on the branch as well, but IMO that temptation should be avoided. Periodic merges from master onto the branch *will* become a time sink, *especially* if other people are following the advice above to make other changes on master. That's exactly what happened with NSR before, and I don't think it will be any different this time or with DHT2. It's really not *that* much work to make something compatible with GlusterD 1 as well, and/or to make it selectable via an option. In the long run, it's likely to be less work than either constant branch management or a big-bang merge at the end. Overall I am fine with the "experimental" on master, I think nothing avoids a review problem better than having things in master. When something move out of experimental, I think we should have had enough eyes on the code, to make that last move less painful than what it is today, i.e a big merge request. So in short my vote is a +1 for the "experimental" manner of approaching this, (esp. for DHT2). Anyway, start of this would be this patch: http://review.gluster.org/#/c/12321 Thanks Shyam, this seems like the right approach to me for all ongoing development. Over time we could also establish graduation criterion from experimental to mainline. I wonder if glusterd2 could also be a different directory in experimental/. We could add a new configure option, say something like --enable-glusterd2, that compiles & installs glusterd2 instead of the existing glusterd. Thoughts? Regards, Vijay ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
On 10/09/2015 11:26 AM, Jeff Darcy wrote: My position is that we should maximize visibility for other developers by doing all work on review.gluster.org. If it doesn't affect existing tests, it should even go on master. This includes: * Minor changes (e.g. list.h or syncop.* in http://review.gluster.org/#/c/8913/) * Refactoring that doesn't change functionality (e.g. all of http://review.gluster.org/#/c/9411/) * Translators that no existing test will enable (e.g. DHT2) It's not hard to ensure that experimental translators get built but not shipped, just by tweaking the specfile. I think it's something to do with "ghost" but maybe someone who actually knows can just answer off the top of their head before I spend 10x as much time investigating. The really sticky case is incompatible changes to permanent infrastructure, such as GlusterD 2. My preference for those to be on review.gluster.org as well, but on a branch other than master. It's tempting to make other things dependent on GlusterD 2 and put them on the branch as well, but IMO that temptation should be avoided. Periodic merges from master onto the branch *will* become a time sink, *especially* if other people are following the advice above to make other changes on master. That's exactly what happened with NSR before, and I don't think it will be any different this time or with DHT2. It's really not *that* much work to make something compatible with GlusterD 1 as well, and/or to make it selectable via an option. In the long run, it's likely to be less work than either constant branch management or a big-bang merge at the end. Overall I am fine with the "experimental" on master, I think nothing avoids a review problem better than having things in master. When something move out of experimental, I think we should have had enough eyes on the code, to make that last move less painful than what it is today, i.e a big merge request. So in short my vote is a +1 for the "experimental" manner of approaching this, (esp. for DHT2). Anyway, start of this would be this patch: http://review.gluster.org/#/c/12321 ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
> I wonder if glusterd2 could also be a different directory in > experimental/. We could add a new configure option, say something like > --enable-glusterd2, that compiles & installs glusterd2 instead of the > existing glusterd. Thoughts? It might be a bit painful. Firstly, anything that involves configure.ac and its cronies is likely to induce a certain amount of nausea. Secondly, glusterd2 has a bunch of new dependencies that are not currently satisfied on our regression test machines (or most developers'). It's not impossible, and most of those obstacles need to be overcome eventually, but I think I'd rather keep glusterd2 developers focused on the glusterd code itself for now and defer work on that other stuff until later. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
My position is that we should maximize visibility for other developers by doing all work on review.gluster.org. If it doesn't affect existing tests, it should even go on master. This includes: * Minor changes (e.g. list.h or syncop.* in http://review.gluster.org/#/c/8913/) * Refactoring that doesn't change functionality (e.g. all of http://review.gluster.org/#/c/9411/) * Translators that no existing test will enable (e.g. DHT2) It's not hard to ensure that experimental translators get built but not shipped, just by tweaking the specfile. I think it's something to do with "ghost" but maybe someone who actually knows can just answer off the top of their head before I spend 10x as much time investigating. The really sticky case is incompatible changes to permanent infrastructure, such as GlusterD 2. My preference for those to be on review.gluster.org as well, but on a branch other than master. It's tempting to make other things dependent on GlusterD 2 and put them on the branch as well, but IMO that temptation should be avoided. Periodic merges from master onto the branch *will* become a time sink, *especially* if other people are following the advice above to make other changes on master. That's exactly what happened with NSR before, and I don't think it will be any different this time or with DHT2. It's really not *that* much work to make something compatible with GlusterD 1 as well, and/or to make it selectable via an option. In the long run, it's likely to be less work than either constant branch management or a big-bang merge at the end. ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted
First of all my apologies for not going through the meeting blog before sending my thoughts on how we plan to maintain GlusterD 2.0 [1]. This approach seems fine to me as long as we don't touch any existing xlators. How do we handle cases where other xlators get impacted by certain changes. Are we going to copy the whole translator in xlators/experimental and start working on it? Instead of all this wouldn't it be simpler to have development under a separate branch say "4.0-unstable" and we could disable CI on this branch till it becomes stable? Are we worried about pulling in the changes from this to master once the branch becomes stable? This is just my thought and I would like to get a clarity on this. Thanks, Atin [1] http://www.gluster.org/pipermail/gluster-devel/2015-October/046872.html On 10/08/2015 11:35 PM, Shyam wrote: > Hi, > > On checking yesterday's gluster meeting AIs and (later) reading the > minutes, for DHT2 here is what I gather and propose to do for $SUBJECT. > Feel free to add/negate any plans. > > (This can also be discussed at [2]) > > --- > 1) Create a directory under the glusterfs master branch as follows, > ./xlators/*experimental*/dht2 > ./xlators/*experimental*/posix2 > > See patch request at [2] > > All code, design documents (work products in general) would go into this > directory. > > 2) Code that compiles and does not cause CI failures could *potentially* > be merged with very few DHT2 dev folks assent. > > There would possibly be no CI integration till we get something working, > so merges would be based on compile passing initially. Soon there would > be an attempt at getting unit testing integrated, so that code being > submitted is not abysmally horrendous > > 3) Common framework code changes (if any) would be presented as a > separate commit request > > 4) (Big problem) DHT2 requires glusterd changes to create a volume as > DHT2 and not DHT, this would be maintained as a .patch in the dht2 > directory as above. This is so that people can play with DHT2 volumes if > interested. Integration of this piece either comes with glusterd 2.0 or > based on time lines of other events, in the current version of glusterd. > (if you are interested in seeing the current version of this patch, go > here [1]) > --- > > If there is some key disagreement on certain points like (2) above, then > we would need to bring in DHT2 code in parts so that it makes sense. > This is fine too, just that we would have 2 repos till we reach a point > of maturity in development. > > --- > *Some issues with the approach:* > A) We need to ensure we do not ship xlators compiled from the > experimental directory > > B) We need to possibly add a buddy maintainer for experimental > translator owners, who can help with the process of merging their changes. > > C) I am not sure how this helps the review process, as initially xlator > development can be iffy and so we do not expect reviews to be stringent. > Later when we want to move this out of the experimental category, how do > we review the same now, and what actions do we take to ensure quality? > Won't we have the same bulk code review issue? > --- > > Shameless plug: For quality and if an xlator plays well with other parts > of gluster the distaf framework of testing against possible graphs and > access protocols can be of immense help. > > Shyam > > [1] > https://github.com/ShyamsundarR/glusterfs/commit/663eeb98f6a51384c8745b8882e7c6c4f7b58a7c > > [2] http://review.gluster.org/#/c/12321/1 > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://www.gluster.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
