Re: Proposal: create a separate ZooKeeper release artifact for recipes
Thanks Jordan. Want to start a new thread on this? THis has already dragged on for a while. Can you also include - what happens to existing recipes? Would this new sub project include the already existing recipes and port them using Curator? thanks mahadev On Mon, Dec 12, 2011 at 11:06 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: While we're not interested in submitting Curator to the Incubator, I've prepared a proposal for including Curator as a ZooKeeper sub-project or inclusion in the main project. Here's the proposal (for commenting, etc.): Curator, a ZooKeeper Recipe Proposal == Abstract == Users of Apache ZooKeeper would greatly benefit from having a high quality implementation of common recipes included with the ZooKeeper distribution. Curator is that implementation. == Proposal == Apache ZooKeeper should produce a new artifact for recipes/applications. This artifact should either be a ZooKeeper sub-project or top-level part of ZooKeeper itself. For Java (and possibly other JVM based languages), the Netflix Curator project should be the artifact. == Background == ZooKeeper consists of server software and client software. The client implementations that are part of the ZooKeeper distribution are very low level and difficult to use correctly. Further, implementing the recipes listed in the ZooKeeper documentation is non-trivial and involves deep knowledge of ZooKeeper best practices and edge cases. == Rationale == The existing clients for ZooKeeper are difficult to use and are limited. Further, correct usage of ZooKeeper is non-trivial and non-obvious. Users of ZooKeeper are mostly interested in the recipes/applications and are not likely interested in becoming experts in the minutiae of correct ZooKeeper client usage. What they want is a simple way to use the recipes. Curator is directed at this goal. == Current Status == Curator is an active open source project hosted at Github (https://github.com/netflix/curator). It has no dependencies other than industry standard libraries. It provides implementations for all recipes listed on the ZooKeeper recipes doc as well as a high level framework for using ZooKeeper that simplifies most of the low level housekeeping normally required. Curator has been open since October, 2011 and has been reviewed by several active members of the ZooKeeper community. Netflix is a strong proponent of open source and is supportive of providing the community with this project. == Core Developers == The initial set of committers are all employees of Netflix, Inc. The lead developer is Jordan Zimmerman (jzimmer...@netflix.com), Senior Platform Engineer at Netflix, Inc. == Alignment == The current ZooKeeper distribution has an obvious hole in regard to recipes/applications. Curator's minimal dependencies and use of standard ZooKeeper components, recipe algorithms and best practices make it a natural compliment to the ZooKeeper distribution. == Known Risks == Orphan-ing of Curator: ZooKeeper has become a major component at Netflix. Curator is the abstraction being used. Open Source Experience: Netflix has major experience with open source. The committers of Curator have long experience with open source. Homogenous/Salaried Developers: The current committers are all salaried employees of Netflix. This proposal would help by having a more diverse set of committers. Ties to Apache products: The initial codebase relies heavily on existing Apache technologies and will continue to do so. == Documentation == Comprehensive documentation is at https://github.com/Netflix/curator/wiki == Initial Source == The source code currently is hosted at Github at: https://github.com/Netflix/curator == External Dependencies == Runtime: * Apache ZooKeeper * Google Guava Test: * Test NG * Apache Commons Math * Mockito * Javaassist Build: * Apache Maven == Initial Committers == Jordan Zimmerman (Netflix, Inc.) Jay Zarfoss (Netflix, Inc.) Jerome Boulon (Netflix, Inc.)
Re: Proposal: create a separate ZooKeeper release artifact for recipes
While we're not interested in submitting Curator to the Incubator, I've prepared a proposal for including Curator as a ZooKeeper sub-project or inclusion in the main project. Here's the proposal (for commenting, etc.): Curator, a ZooKeeper Recipe Proposal == Abstract == Users of Apache ZooKeeper would greatly benefit from having a high quality implementation of common recipes included with the ZooKeeper distribution. Curator is that implementation. == Proposal == Apache ZooKeeper should produce a new artifact for recipes/applications. This artifact should either be a ZooKeeper sub-project or top-level part of ZooKeeper itself. For Java (and possibly other JVM based languages), the Netflix Curator project should be the artifact. == Background == ZooKeeper consists of server software and client software. The client implementations that are part of the ZooKeeper distribution are very low level and difficult to use correctly. Further, implementing the recipes listed in the ZooKeeper documentation is non-trivial and involves deep knowledge of ZooKeeper best practices and edge cases. == Rationale == The existing clients for ZooKeeper are difficult to use and are limited. Further, correct usage of ZooKeeper is non-trivial and non-obvious. Users of ZooKeeper are mostly interested in the recipes/applications and are not likely interested in becoming experts in the minutiae of correct ZooKeeper client usage. What they want is a simple way to use the recipes. Curator is directed at this goal. == Current Status == Curator is an active open source project hosted at Github (https://github.com/netflix/curator). It has no dependencies other than industry standard libraries. It provides implementations for all recipes listed on the ZooKeeper recipes doc as well as a high level framework for using ZooKeeper that simplifies most of the low level housekeeping normally required. Curator has been open since October, 2011 and has been reviewed by several active members of the ZooKeeper community. Netflix is a strong proponent of open source and is supportive of providing the community with this project. == Core Developers == The initial set of committers are all employees of Netflix, Inc. The lead developer is Jordan Zimmerman (jzimmer...@netflix.com), Senior Platform Engineer at Netflix, Inc. == Alignment == The current ZooKeeper distribution has an obvious hole in regard to recipes/applications. Curator's minimal dependencies and use of standard ZooKeeper components, recipe algorithms and best practices make it a natural compliment to the ZooKeeper distribution. == Known Risks == Orphan-ing of Curator: ZooKeeper has become a major component at Netflix. Curator is the abstraction being used. Open Source Experience: Netflix has major experience with open source. The committers of Curator have long experience with open source. Homogenous/Salaried Developers: The current committers are all salaried employees of Netflix. This proposal would help by having a more diverse set of committers. Ties to Apache products: The initial codebase relies heavily on existing Apache technologies and will continue to do so. == Documentation == Comprehensive documentation is at https://github.com/Netflix/curator/wiki == Initial Source == The source code currently is hosted at Github at: https://github.com/Netflix/curator == External Dependencies == Runtime: * Apache ZooKeeper * Google Guava Test: * Test NG * Apache Commons Math * Mockito * Javaassist Build: * Apache Maven == Initial Committers == Jordan Zimmerman (Netflix, Inc.) Jay Zarfoss (Netflix, Inc.) Jerome Boulon (Netflix, Inc.)
Re: Proposal: create a separate ZooKeeper release artifact for recipes
To me it is difficult to assess the various options without seeing a proposal. If you feel strongly about it being a zookeeper sub-project, I would suggest to write a proposal following the incubator guidelines (http://incubator.apache.org/guides/proposal.html) and let the pmc discuss and vote. If the zookeeper pmc does not accept it, then I'm sure you can submit it pretty much verbatim to incubator. If there is interest, there are lots of samples here: http://wiki.apache.org/incubator/FrontPage -Flavio On Dec 6, 2011, at 3:11 AM, Camille Fournier wrote: I'm sure there is. But I still stand behind the idea of this as an incubator. Personally, I am not comfortable endorsing this project as the be-all-end-all zk client. I think we have to be truly comfortable supporting it as such if we pull it in as part of our code base, no matter how tangential we might keep it, and I for one am not on board with that just yet. Maybe in a few months if we saw a significant uptick in adoption by our user base, we could evaluate it as such. But right now I think it's premature to take this code base on. C On Mon, Dec 5, 2011 at 8:59 PM, Ted Dunning ted.dunn...@gmail.com wrote: IF we are talking analogies, there are SolR / Lucene, Mahout Collections / Mahout, PiggyBank / Pig that go the other way. You will be able to add many more as well, starting with Lucene / Hadoop and so on. My thought is that it really has to do most with common development interests. The projects that became TLP's had divergent communities and those that stuck together had more overlapping communities. In this case, I see lots of overlap because I have interests on both sides. Other people may care only about one side or the other, but I suspect that we actually have a spectrum rather than a dichotomy. On Mon, Dec 5, 2011 at 5:35 PM, Camille Fournier cami...@apache.org wrote: I vote incubator. It seems to be to zookeeper as hector is to cassandra. From my phone On Dec 5, 2011 7:13 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Mon, Dec 5, 2011 at 3:33 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. We'd want 1 (me) at minimum. 2 would be better as I have a backup here at Netflix. Let's see what the rest of the ZK world says. flavio junqueira research scientist f...@yahoo-inc.com direct +34 93-183-8828 avinguda diagonal 177, 8th floor, barcelona, 08018, es phone (408) 349 3300fax (408) 349 3301
Re: Proposal: create a separate ZooKeeper release artifact for recipes
i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do you want to build a community around Apache Curator or are you just looking to push the source into ZK and move on to other things? Patrick On 11/30/11 5:42 PM, Ted Dunning ted.dunn...@gmail.com wrote: Separator committer lists are generally frowned on by Apache (ref Lucene/Solr and also Mahout). They also are essentially unnecessary. It isn't that big a deal to have a single committer list and depend on reversion to enforce community standards. Core ZK is review-then-commit and should continue to be. Add-on modules that are relatively independent of the core can quite reasonably be commit-then-review or whatever level of care is implied by module's content. The review can include a determination of who should actually do the commit. Moreover, projects can easily have more than one artifact. Mahout has three independent artifacts (mahout-math, mahout-collections and the core stuff with examples and integration code). That works well. There are different sets of committers who are typically
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do you want to build a community around Apache Curator or are you just looking to push the source into ZK and move on to other things? Patrick On 11/30/11 5:42 PM, Ted Dunning ted.dunn...@gmail.com wrote: Separator committer lists are generally frowned on by Apache (ref Lucene/Solr and also Mahout). They also are essentially unnecessary. It isn't that big a deal to have a single committer list and depend on reversion to enforce community standards. Core ZK is review-then-commit and should continue to be. Add
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Incubator is the Apache recommended way to handle this. We could be the sponsor, with the intent that once Curator graduates from the incubator we would accept it http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor anyone could join the curator effort, including existing ZK committers/contributors. if curator felt it should be a tlp (at time of graduation) I think that would be fine too. I would be happy to Champion/Mentor this effort. Patrick On Mon, Dec 5, 2011 at 11:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do
Re: Proposal: create a separate ZooKeeper release artifact for recipes
sounds great. do you think they would broaden their scope to all of the zk recipes rather than just their own? ben On Mon, Dec 5, 2011 at 11:11 AM, Patrick Hunt ph...@apache.org wrote: Incubator is the Apache recommended way to handle this. We could be the sponsor, with the intent that once Curator graduates from the incubator we would accept it http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor anyone could join the curator effort, including existing ZK committers/contributors. if curator felt it should be a tlp (at time of graduation) I think that would be fine too. I would be happy to Champion/Mentor this effort. Patrick On Mon, Dec 5, 2011 at 11:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer
Re: Proposal: create a separate ZooKeeper release artifact for recipes
+1... mahadev On Mon, Dec 5, 2011 at 11:42 AM, Benjamin Reed br...@apache.org wrote: +1 On Mon, Dec 5, 2011 at 11:11 AM, Patrick Hunt ph...@apache.org wrote: Incubator is the Apache recommended way to handle this. We could be the sponsor, with the intent that once Curator graduates from the incubator we would accept it http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor anyone could join the curator effort, including existing ZK committers/contributors. if curator felt it should be a tlp (at time of graduation) I think that would be fine too. I would be happy to Champion/Mentor this effort. Patrick On Mon, Dec 5, 2011 at 11:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've
Re: Proposal: create a separate ZooKeeper release artifact for recipes
-1 I don't think that a full incubator process is needed or even useful here. Incubator is ONE process in Apache process for this, but a contrib artifact is another way to do it. This isn't a sub-project. This is additional ZK software. It just isn't the core stuff. This is more akin to Solr being added to Lucene than it is like Mahout spinning out of Lucene ultimately as a top level project. On Mon, Dec 5, 2011 at 11:58 AM, Mahadev Konar maha...@hortonworks.comwrote: +1... mahadev On Mon, Dec 5, 2011 at 11:42 AM, Benjamin Reed br...@apache.org wrote: +1 On Mon, Dec 5, 2011 at 11:11 AM, Patrick Hunt ph...@apache.org wrote: Incubator is the Apache recommended way to handle this. We could be the sponsor, with the intent that once Curator graduates from the incubator we would accept it http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor anyone could join the curator effort, including existing ZK committers/contributors. if curator felt it should be a tlp (at time of graduation) I think that would be fine too. I would be happy to Champion/Mentor this effort. Patrick On Mon, Dec 5, 2011 at 11:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Ted, can you talk more about this? How do you see this working? FYI - I had a nice lunch with Patrick today. Personally (and Netflixily) I'm not very interested in Incubator. The recipes are useless without ZooKeeper itself so I can't imagine recipes ever becoming a TLP. Patrick and I talked about going the sub-project route (though he prefers Incubator). My desires are: * Get widespread use/visibility for Curator * Get additional committers to Curator * Help ZK users by making quality recipe implementations readily available -JZ On 12/5/11 12:36 PM, Ted Dunning ted.dunn...@gmail.com wrote: -1 I don't think that a full incubator process is needed or even useful here. Incubator is ONE process in Apache process for this, but a contrib artifact is another way to do it. This isn't a sub-project. This is additional ZK software. It just isn't the core stuff. This is more akin to Solr being added to Lucene than it is like Mahout spinning out of Lucene ultimately as a top level project. On Mon, Dec 5, 2011 at 11:58 AM, Mahadev Konar maha...@hortonworks.comwrote: +1... mahadev On Mon, Dec 5, 2011 at 11:42 AM, Benjamin Reed br...@apache.org wrote: +1 On Mon, Dec 5, 2011 at 11:11 AM, Patrick Hunt ph...@apache.org wrote: Incubator is the Apache recommended way to handle this. We could be the sponsor, with the intent that once Curator graduates from the incubator we would accept it http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sp onsor anyone could join the curator effort, including existing ZK committers/contributors. if curator felt it should be a tlp (at time of graduation) I think that would be fine too. I would be happy to Champion/Mentor this effort. Patrick On Mon, Dec 5, 2011 at 11:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Hi Ted. Incubator has a number of benefits for new projects; helping them learn the ropes of Apache, setup infrastructure, build community, create their own Apache blessed release, etc... That's a big part of the responsibility that IPMC members take on when a project enters incubation, something we'd (ZKPMC) have to do ourselves in the approach you outline. Would you be able to commit that sort of time? wrt examples: MRUnit is an incubator project, it was a contrib of Hadoop. Hadoop recently worked to shed itself of all (most?) contribs. This is similar no? Apache Felix is another example (in the extreme, it has 20 subs) http://felix.apache.org/site/index.html Doug has always told me that you want to be guided by community more than anything. Keep two projects together if they are the same community, otw separating them is ok (he strongly favors incubator over sub). Patrick On Mon, Dec 5, 2011 at 12:36 PM, Ted Dunning ted.dunn...@gmail.com wrote: -1 I don't think that a full incubator process is needed or even useful here. Incubator is ONE process in Apache process for this, but a contrib artifact is another way to do it. This isn't a sub-project. This is additional ZK software. It just isn't the core stuff. This is more akin to Solr being added to Lucene than it is like Mahout spinning out of Lucene ultimately as a top level project. On Mon, Dec 5, 2011 at 11:58 AM, Mahadev Konar maha...@hortonworks.comwrote: +1... mahadev On Mon, Dec 5, 2011 at 11:42 AM, Benjamin Reed br...@apache.org wrote: +1 On Mon, Dec 5, 2011 at 11:11 AM, Patrick Hunt ph...@apache.org wrote: Incubator is the Apache recommended way to handle this. We could be the sponsor, with the intent that once Curator graduates from the incubator we would accept it http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor anyone could join the curator effort, including existing ZK committers/contributors. if curator felt it should be a tlp (at time of graduation) I think that would be fine too. I would be happy to Champion/Mentor this effort. Patrick On Mon, Dec 5, 2011 at 11:06 AM, Ted Dunning ted.dunn...@gmail.com wrote: Ben, I think that you are right on target with this. We need to have a wider community of developers in order to have the bandwidth to handle recipes and the recipes really do need separate release cycles. Independent releases should be not much of an issue given how compatible ZK versions tend to be. On Mon, Dec 5, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote: i also agreed that it would be great to build a community around curator especially if it could pull in other zk recipes. i've felt for a while that the core developers don't have the bandwidth to really keep track of recipe development and recipes are a very important aspect of zk. so, having a self sustaining community of developers focused on that would be awesome. being able to have a separate release cycle is also important i think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do you want to build a community around Apache Curator or are you just looking to push the source into ZK and move on to other things? Patrick On 11/30/11 5:42 PM, Ted Dunning ted.dunn...@gmail.com wrote: Separator committer lists are generally frowned on by Apache (ref Lucene/Solr and also Mahout). They also are essentially unnecessary. It isn't that big a deal to have a single committer list and depend on reversion to enforce community standards. Core ZK is review-then-commit and should continue to be. Add-on modules that are relatively independent of the core can quite reasonably be commit-then-review or whatever level of care is implied by module's content. The review can include a determination of who should actually do the commit. Moreover, projects can easily have more than one artifact. Mahout has three independent artifacts (mahout-math, mahout-collections and the core stuff with examples and integration code). That works well. There are different sets of committers who are typically the ones who work on different components. The overall level of diligence required in Mahout is much lower than for Zookeeper as you might expect, but the principle that multiple artifacts are fine is still the same. Likewise, the idea of committers with specialized expertise also
Re: Proposal: create a separate ZooKeeper release artifact for recipes
think. (i may be reading too much into the proposal though :) ben On Mon, Dec 5, 2011 at 7:14 AM, Flavio Junqueira f...@yahoo-inc.com wrote: I suppose you're just trying to sense if others think that it would be a good idea to have it as a subproject. If so, I'm in favor of putting a proposal together, +1. -Flavio On Dec 2, 2011, at 8:45 PM, Patrick Hunt wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do you want to build a community around Apache Curator or are you just looking to push the source into ZK and move on to other things? Patrick On 11/30/11 5:42 PM, Ted Dunning ted.dunn...@gmail.com wrote: Separator committer lists are generally frowned on by Apache (ref Lucene/Solr and also Mahout). They also are essentially unnecessary. It isn't that big a deal to have a single committer list and depend on reversion to enforce community standards. Core ZK is review-then-commit and should continue to be. Add-on modules that are relatively independent of the core can quite reasonably be commit-then-review or whatever level of care is implied by module's content. The review can include a determination of who should actually do the commit. Moreover, projects can easily have more than one artifact. Mahout has three independent artifacts (mahout-math, mahout-collections and the core stuff with examples and integration code). That works
Re: Proposal: create a separate ZooKeeper release artifact for recipes
I don't think that is necessary. On Mon, Dec 5, 2011 at 2:44 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: I understand the limitations (being tied to ZK server's release schedule), etc.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
How would that work? On 12/5/11 2:49 PM, Ted Dunning ted.dunn...@gmail.com wrote: I don't think that is necessary. On Mon, Dec 5, 2011 at 2:44 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: I understand the limitations (being tied to ZK server's release schedule), etc.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
There would be two artifacts. The recipes artifact would have a particular version of ZK as a dependency. There might be point releases of the recipes to backport features. It is customary with tightly coupled things like this to synchronize major version numbers. With ZK, major and minor might be appropriate. That doesn't affect the schedule, however. On Mon, Dec 5, 2011 at 2:55 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: How would that work? On 12/5/11 2:49 PM, Ted Dunning ted.dunn...@gmail.com wrote: I don't think that is necessary. On Mon, Dec 5, 2011 at 2:44 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: I understand the limitations (being tied to ZK server's release schedule), etc.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
I see - this would be done in Ivy. OK by me. So, to summarize, there are 3 possibilities: * Push Curator into ZooKeeper proper under a new recipes artifact. Curator would get n committers from Netflix with the understanding that their focus is on Curator * Start a sub-project ala BookKeeper for ZooKeeper with its own committer list * Put Curator into the Incubator and follow that road Ted (and now I) prefer the first option. I believe Patrick prefers the third option but would accept the second. How do the others feel? -JZ On 12/5/11 3:06 PM, Ted Dunning ted.dunn...@gmail.com wrote: There would be two artifacts. The recipes artifact would have a particular version of ZK as a dependency. There might be point releases of the recipes to backport features. It is customary with tightly coupled things like this to synchronize major version numbers. With ZK, major and minor might be appropriate. That doesn't affect the schedule, however. On Mon, Dec 5, 2011 at 2:55 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: How would that work? On 12/5/11 2:49 PM, Ted Dunning ted.dunn...@gmail.com wrote: I don't think that is necessary. On Mon, Dec 5, 2011 at 2:44 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: I understand the limitations (being tied to ZK server's release schedule), etc.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Some nuances. Overall, I like the way you are working for consensus. On Mon, Dec 5, 2011 at 3:13 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: I see - this would be done in Ivy. OK by me. ZK is moving to maven, I would prefer to just jump recipes to there directly. I can set up the project initially. So, to summarize, there are 3 possibilities: * Push Curator into ZooKeeper proper under a new recipes artifact. Curator would get n committers from Netflix with the understanding that their focus is on Curator Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. * Start a sub-project ala BookKeeper for ZooKeeper with its own committer list * Put Curator into the Incubator and follow that road Ted (and now I) prefer the first option. I believe Patrick prefers the third option but would accept the second. How do the others feel? I don't really think that we have heard from Patrick on option 1. I think he was thinking I was suggesting option 2 and I agree with him that is sub-optimal.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
ZK is moving to maven, I would prefer to just jump recipes to there directly. I can set up the project initially. Better for me. Curator is Maven already. -JZ On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Some nuances. Overall, I like the way you are working for consensus. On Mon, Dec 5, 2011 at 3:13 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: I see - this would be done in Ivy. OK by me. ZK is moving to maven, I would prefer to just jump recipes to there directly. I can set up the project initially. So, to summarize, there are 3 possibilities: * Push Curator into ZooKeeper proper under a new recipes artifact. Curator would get n committers from Netflix with the understanding that their focus is on Curator Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. * Start a sub-project ala BookKeeper for ZooKeeper with its own committer list * Put Curator into the Incubator and follow that road Ted (and now I) prefer the first option. I believe Patrick prefers the third option but would accept the second. How do the others feel? I don't really think that we have heard from Patrick on option 1. I think he was thinking I was suggesting option 2 and I agree with him that is sub-optimal.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. We'd want 1 (me) at minimum. 2 would be better as I have a backup here at Netflix. -JZ
Re: Proposal: create a separate ZooKeeper release artifact for recipes
On Mon, Dec 5, 2011 at 3:33 PM, Jordan Zimmerman jzimmer...@netflix.comwrote: On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. We'd want 1 (me) at minimum. 2 would be better as I have a backup here at Netflix. Let's see what the rest of the ZK world says.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
I vote incubator. It seems to be to zookeeper as hector is to cassandra. From my phone On Dec 5, 2011 7:13 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Mon, Dec 5, 2011 at 3:33 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. We'd want 1 (me) at minimum. 2 would be better as I have a backup here at Netflix. Let's see what the rest of the ZK world says.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
-- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do you want to build a community around Apache Curator or are you just looking to push the source into ZK and move on to other things? Patrick On 11/30/11 5:42 PM, Ted Dunning ted.dunn...@gmail.com wrote: Separator committer lists are generally frowned on by Apache (ref Lucene/Solr and also Mahout). They also are essentially unnecessary. It isn't that big a deal to have a single committer list and depend on reversion to enforce community standards. Core ZK is review-then-commit and should continue to be. Add-on modules that are relatively independent of the core can quite reasonably be commit-then-review or whatever level of care is implied by module's content. The review can include a determination of who should actually do the commit. Moreover, projects can easily have more than one artifact. Mahout has three independent artifacts (mahout-math, mahout-collections and the core stuff with examples and integration code). That works well. There are different sets of committers who are typically the ones who work on different components. The overall level of diligence required in Mahout is much lower than for Zookeeper as you might expect, but the principle that multiple artifacts are fine is still the same. Likewise, the idea of committers with specialized expertise also applies here. There is a tradition in Zookeeper of having a very high bar for committership, but experience in some other
Re: Proposal: create a separate ZooKeeper release artifact for recipes
IF we are talking analogies, there are SolR / Lucene, Mahout Collections / Mahout, PiggyBank / Pig that go the other way. You will be able to add many more as well, starting with Lucene / Hadoop and so on. My thought is that it really has to do most with common development interests. The projects that became TLP's had divergent communities and those that stuck together had more overlapping communities. In this case, I see lots of overlap because I have interests on both sides. Other people may care only about one side or the other, but I suspect that we actually have a spectrum rather than a dichotomy. On Mon, Dec 5, 2011 at 5:35 PM, Camille Fournier cami...@apache.org wrote: I vote incubator. It seems to be to zookeeper as hector is to cassandra. From my phone On Dec 5, 2011 7:13 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Mon, Dec 5, 2011 at 3:33 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. We'd want 1 (me) at minimum. 2 would be better as I have a backup here at Netflix. Let's see what the rest of the ZK world says.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
I'm sure there is. But I still stand behind the idea of this as an incubator. Personally, I am not comfortable endorsing this project as the be-all-end-all zk client. I think we have to be truly comfortable supporting it as such if we pull it in as part of our code base, no matter how tangential we might keep it, and I for one am not on board with that just yet. Maybe in a few months if we saw a significant uptick in adoption by our user base, we could evaluate it as such. But right now I think it's premature to take this code base on. C On Mon, Dec 5, 2011 at 8:59 PM, Ted Dunning ted.dunn...@gmail.com wrote: IF we are talking analogies, there are SolR / Lucene, Mahout Collections / Mahout, PiggyBank / Pig that go the other way. You will be able to add many more as well, starting with Lucene / Hadoop and so on. My thought is that it really has to do most with common development interests. The projects that became TLP's had divergent communities and those that stuck together had more overlapping communities. In this case, I see lots of overlap because I have interests on both sides. Other people may care only about one side or the other, but I suspect that we actually have a spectrum rather than a dichotomy. On Mon, Dec 5, 2011 at 5:35 PM, Camille Fournier cami...@apache.org wrote: I vote incubator. It seems to be to zookeeper as hector is to cassandra. From my phone On Dec 5, 2011 7:13 PM, Ted Dunning ted.dunn...@gmail.com wrote: On Mon, Dec 5, 2011 at 3:33 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: On 12/5/11 3:29 PM, Ted Dunning ted.dunn...@gmail.com wrote: Initially, n would be a very small number (possibly even 0, but that seems unnecessary). The number could increase over time. We'd want 1 (me) at minimum. 2 would be better as I have a backup here at Netflix. Let's see what the rest of the ZK world says.
Re: Proposal: create a separate ZooKeeper release artifact for recipes
I think it sounds like a great idea and a nice nucleus for a ZK applications area. On Fri, Dec 2, 2011 at 11:45 AM, Patrick Hunt ph...@apache.org wrote: Somehow this got off list, Jordan and I just noticed, summary so far: -- Forwarded message -- From: Patrick Hunt ph...@apache.org Date: Fri, Dec 2, 2011 at 9:56 AM Subject: Re: Proposal: create a separate ZooKeeper release artifact for recipes To: Jordan Zimmerman jzimmer...@netflix.com Did you mean to reply just to me? I think this is a great idea btw, just need to work out the mechanics of getting it integrated and get the others on board. Would it help to talk f2f? On Thu, Dec 1, 2011 at 6:20 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: From talking to folks here (and my own feeling), I'd like to have Curator get as wide use as possible. Some sort of sanction from the ZooKeeper team would do that. I don't see a lot of downside to getting a community around it either. Sure, it's nice to be the only guy I need to please, but the benefits of a community outweigh that. I think the beauty of folding it in to ZooKeeper is that the structure is already there: Jira, website, etc. I (and probably one other from Netflix) could take on the role of managing the Curator portion so that it doesn't burden you folks. I'm doing that internally at Netflix anyway. Frankly, it sounds like fun (famous last words). -Jordan On 12/1/11 5:36 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 2:05 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: The feeling here is that it would be nice to push Curator into ZK. However, we'd still like to contribute to it. If you don't want separate contributor lists, I could be a ZK contributor but explicitly only work on Curator. Of course, we're open to your suggestions on this as well. Don't get me wrong, I don't mind having one list of contributors/committers for ZK/Curator, I'm personally fine with having one. If you notice in my original proposal I lean towards that direction (single set of committers with multiple artifacts) I see a benefit to having something like Curator available to users, open sourcing it on GH accomplishes this. Bringing something to Apache means building a community around it though, not just making the code open source. This is analogous to when ZK itself was on sourceforge, eventually moving to Apache. Here's the issue: building community adds overhead, which you might not be willing to sign up for. I'm not sure we (zk community) can take on this additional cost alone. Hence my question -- I _wouldn't_ want to you leave, that's the point. We'd need some set of core Curator contributors who would continue to work on Curator, answer questions, shepherd new contributor/committers/etc... If that doesn't happen the code will get imported, see some use, but never really reach it's full potential. Hope that helps. LMK. Patrick On 12/1/11 2:01 PM, Patrick Hunt ph...@apache.org wrote: On Thu, Dec 1, 2011 at 1:26 PM, Jordan Zimmerman jzimmer...@netflix.com wrote: I've spoken about this with the appropriate folks here at Netflix and we're interested in pursuing this. What are the next steps? Hi Jordan, what are your intentions. Do you want to build a community around Apache Curator or are you just looking to push the source into ZK and move on to other things? Patrick On 11/30/11 5:42 PM, Ted Dunning ted.dunn...@gmail.com wrote: Separator committer lists are generally frowned on by Apache (ref Lucene/Solr and also Mahout). They also are essentially unnecessary. It isn't that big a deal to have a single committer list and depend on reversion to enforce community standards. Core ZK is review-then-commit and should continue to be. Add-on modules that are relatively independent of the core can quite reasonably be commit-then-review or whatever level of care is implied by module's content. The review can include a determination of who should actually do the commit. Moreover, projects can easily have more than one artifact. Mahout has three independent artifacts (mahout-math, mahout-collections and the core stuff with examples and integration code). That works well. There are different sets of committers who are typically the ones who work on different components. The overall level of diligence required in Mahout is much lower than for Zookeeper as you might expect, but the principle that multiple artifacts are fine is still the same. Likewise, the idea of committers with specialized expertise also applies here. There is a tradition in Zookeeper of having a very high bar for committership, but experience in some other projects indicates that this doesn't actually help quality or security all that much and it can be argued to decrease community involvement a bit. On Wed, Nov 30, 2011 at 5:11 PM, Patrick Hunt ph
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Curator has been getting a lot of interest and was just officially
announced by Netflix:
http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html
I chatted briefly with @adrianco @chad_walters @randgalt (Jordan
Zimmerman) on twitter and the idea of moving Curator into
Apache/ZooKeeper came up:
https://twitter.com/#!/search/phunt%20curator
I wanted to restart this thread as a way of discussing this idea in
more depth, and to gauge the interest of both communities. I
personally think this would be a great thing: both for users and for
the development community itself. Thoughts?
Re the mechanics. See my original post below. Seems to me we (ZK PMC)
could sponsor Curator as an incubator project with the intent to bring
it to ZK as either a subproject (it's own committer list) or as a
separate release artifact of the ZK TLP itself. Or just shortcut the
incubator process altogether. My preference would be to go incubator
first. At some point we would deprecate the existing ZK recipes (ala
Curator had suitable replacements).
Patrick
On Fri, Jul 22, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote:
while i agree with the sentiment of not fragmenting the zookeeper
community and recipe committers also moving into core development, i
also think it would be good if a strong community of developers grew
up around zookeeper recipes. to do that they need a sense of identity,
and i'm not sure if they can get that with this proposal.
on the other hand the proposal does address all of the other issues i
have with recipe maintenance. the key is to grow a committer base.
ben
On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt ph...@apache.org wrote:
During the recent developer meetup we discussed how to more
effectively grow the recipes.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting
The concerns seemed to be around managing changes, community growth,
and releases. We discussed a number of approaches:
* move recipes into apache-extras
* or github
* or incubator
* or sub-project
each of these has it's good and bad points. One alternative that I put
forth was to create a separate release artifact for recipes as part of
the ZooKeeper project itself. What would this look like?
* zookeeper/trunk/src/recipes would move to
zookeeper/recipes/{trunk,branches,tags,...}
* there would be distinct/separate releases of zookeeper and
zookeeper-recipes artifacts
** these releases would not necessarily be synchronized, although we'd
want to ensure that particular versions of each release work together
** these releases would go through our standard release process - ie
creation of release candidate by a committer and approval by the PMC
** recipes would no longer be included in zookeeper (core) releases
* continue to use the ZOOKEEPER jira, with recipes component
* space in the web and wiki sites for recipes specific pages
* recipes continue to use the std zookeeper mailing lists
* commits to recipes would continue to use our regular commit process
(jira, rtc, etc...)
* we would accept new ZooKeeper committers with the intent that they
focus on their area of expertise, whether this be core (zookeeper
trunk) or recipes.
This last item is the more interesting I think, the other issues are
all pretty straightforward. Although moving to sub (or out entirely)
would allow us to have a separate commit list it would fragment the
ZooKeeper community. I also believe that over time committers to
recipes would more likely get experience with core and start
contributing there as well. The only issue really, if you want to call
it that, is that we'd have committers with guardrails implied. ie
someone with more expertise in recipes than core. However there are
many other Apache projects that take this approach, and do it
successfully. Regardless RTC allows for review both before and after a
change is committed.
What do you think?
Patrick
Re: Proposal: create a separate ZooKeeper release artifact for recipes
My initial opinion would be to make it a separate artifact - something
akin to the blessed implementation. What changes would need to be made to
bring it into the incubator? My main concern is that this is an active
service inside of Netflix. I imagine Netflix would fork it internally so
as not to burden our releases.
-Jordan
On 11/30/11 5:11 PM, Patrick Hunt ph...@apache.org wrote:
Curator has been getting a lot of interest and was just officially
announced by Netflix:
http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.
html
I chatted briefly with @adrianco @chad_walters @randgalt (Jordan
Zimmerman) on twitter and the idea of moving Curator into
Apache/ZooKeeper came up:
https://twitter.com/#!/search/phunt%20curator
I wanted to restart this thread as a way of discussing this idea in
more depth, and to gauge the interest of both communities. I
personally think this would be a great thing: both for users and for
the development community itself. Thoughts?
Re the mechanics. See my original post below. Seems to me we (ZK PMC)
could sponsor Curator as an incubator project with the intent to bring
it to ZK as either a subproject (it's own committer list) or as a
separate release artifact of the ZK TLP itself. Or just shortcut the
incubator process altogether. My preference would be to go incubator
first. At some point we would deprecate the existing ZK recipes (ala
Curator had suitable replacements).
Patrick
On Fri, Jul 22, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote:
while i agree with the sentiment of not fragmenting the zookeeper
community and recipe committers also moving into core development, i
also think it would be good if a strong community of developers grew
up around zookeeper recipes. to do that they need a sense of identity,
and i'm not sure if they can get that with this proposal.
on the other hand the proposal does address all of the other issues i
have with recipe maintenance. the key is to grow a committer base.
ben
On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt ph...@apache.org wrote:
During the recent developer meetup we discussed how to more
effectively grow the recipes.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperM
eeting
The concerns seemed to be around managing changes, community growth,
and releases. We discussed a number of approaches:
* move recipes into apache-extras
* or github
* or incubator
* or sub-project
each of these has it's good and bad points. One alternative that I put
forth was to create a separate release artifact for recipes as part of
the ZooKeeper project itself. What would this look like?
* zookeeper/trunk/src/recipes would move to
zookeeper/recipes/{trunk,branches,tags,...}
* there would be distinct/separate releases of zookeeper and
zookeeper-recipes artifacts
** these releases would not necessarily be synchronized, although we'd
want to ensure that particular versions of each release work together
** these releases would go through our standard release process - ie
creation of release candidate by a committer and approval by the PMC
** recipes would no longer be included in zookeeper (core) releases
* continue to use the ZOOKEEPER jira, with recipes component
* space in the web and wiki sites for recipes specific pages
* recipes continue to use the std zookeeper mailing lists
* commits to recipes would continue to use our regular commit process
(jira, rtc, etc...)
* we would accept new ZooKeeper committers with the intent that they
focus on their area of expertise, whether this be core (zookeeper
trunk) or recipes.
This last item is the more interesting I think, the other issues are
all pretty straightforward. Although moving to sub (or out entirely)
would allow us to have a separate commit list it would fragment the
ZooKeeper community. I also believe that over time committers to
recipes would more likely get experience with core and start
contributing there as well. The only issue really, if you want to call
it that, is that we'd have committers with guardrails implied. ie
someone with more expertise in recipes than core. However there are
many other Apache projects that take this approach, and do it
successfully. Regardless RTC allows for review both before and after a
change is committed.
What do you think?
Patrick
Re: Proposal: create a separate ZooKeeper release artifact for recipes
Separator committer lists are generally frowned on by Apache (ref
Lucene/Solr and also Mahout).
They also are essentially unnecessary. It isn't that big a deal to have a
single committer list and depend on reversion to enforce community
standards. Core ZK is review-then-commit and should continue to be.
Add-on modules that are relatively independent of the core can quite
reasonably be commit-then-review or whatever level of care is implied by
module's content. The review can include a determination of who should
actually do the commit.
Moreover, projects can easily have more than one artifact. Mahout has
three independent artifacts (mahout-math, mahout-collections and the core
stuff with examples and integration code). That works well. There are
different sets of committers who are typically the ones who work on
different components. The overall level of diligence required in Mahout is
much lower than for Zookeeper as you might expect, but the principle that
multiple artifacts are fine is still the same. Likewise, the idea of
committers with specialized expertise also applies here.
There is a tradition in Zookeeper of having a very high bar for
committership, but experience in some other projects indicates that this
doesn't actually help quality or security all that much and it can be
argued to decrease community involvement a bit.
On Wed, Nov 30, 2011 at 5:11 PM, Patrick Hunt ph...@apache.org wrote:
Curator has been getting a lot of interest and was just officially
announced by Netflix:
http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html
I chatted briefly with @adrianco @chad_walters @randgalt (Jordan
Zimmerman) on twitter and the idea of moving Curator into
Apache/ZooKeeper came up:
https://twitter.com/#!/search/phunt%20curator
I wanted to restart this thread as a way of discussing this idea in
more depth, and to gauge the interest of both communities. I
personally think this would be a great thing: both for users and for
the development community itself. Thoughts?
Re the mechanics. See my original post below. Seems to me we (ZK PMC)
could sponsor Curator as an incubator project with the intent to bring
it to ZK as either a subproject (it's own committer list) or as a
separate release artifact of the ZK TLP itself. Or just shortcut the
incubator process altogether. My preference would be to go incubator
first. At some point we would deprecate the existing ZK recipes (ala
Curator had suitable replacements).
Patrick
On Fri, Jul 22, 2011 at 10:59 AM, Benjamin Reed br...@apache.org wrote:
while i agree with the sentiment of not fragmenting the zookeeper
community and recipe committers also moving into core development, i
also think it would be good if a strong community of developers grew
up around zookeeper recipes. to do that they need a sense of identity,
and i'm not sure if they can get that with this proposal.
on the other hand the proposal does address all of the other issues i
have with recipe maintenance. the key is to grow a committer base.
ben
On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt ph...@apache.org wrote:
During the recent developer meetup we discussed how to more
effectively grow the recipes.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting
The concerns seemed to be around managing changes, community growth,
and releases. We discussed a number of approaches:
* move recipes into apache-extras
* or github
* or incubator
* or sub-project
each of these has it's good and bad points. One alternative that I put
forth was to create a separate release artifact for recipes as part of
the ZooKeeper project itself. What would this look like?
* zookeeper/trunk/src/recipes would move to
zookeeper/recipes/{trunk,branches,tags,...}
* there would be distinct/separate releases of zookeeper and
zookeeper-recipes artifacts
** these releases would not necessarily be synchronized, although we'd
want to ensure that particular versions of each release work together
** these releases would go through our standard release process - ie
creation of release candidate by a committer and approval by the PMC
** recipes would no longer be included in zookeeper (core) releases
* continue to use the ZOOKEEPER jira, with recipes component
* space in the web and wiki sites for recipes specific pages
* recipes continue to use the std zookeeper mailing lists
* commits to recipes would continue to use our regular commit process
(jira, rtc, etc...)
* we would accept new ZooKeeper committers with the intent that they
focus on their area of expertise, whether this be core (zookeeper
trunk) or recipes.
This last item is the more interesting I think, the other issues are
all pretty straightforward. Although moving to sub (or out entirely)
would allow us to have a separate commit list it would fragment the
ZooKeeper community. I also
Re: Proposal: create a separate ZooKeeper release artifact for recipes
while i agree with the sentiment of not fragmenting the zookeeper
community and recipe committers also moving into core development, i
also think it would be good if a strong community of developers grew
up around zookeeper recipes. to do that they need a sense of identity,
and i'm not sure if they can get that with this proposal.
on the other hand the proposal does address all of the other issues i
have with recipe maintenance. the key is to grow a committer base.
ben
On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt ph...@apache.org wrote:
During the recent developer meetup we discussed how to more
effectively grow the recipes.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting
The concerns seemed to be around managing changes, community growth,
and releases. We discussed a number of approaches:
* move recipes into apache-extras
* or github
* or incubator
* or sub-project
each of these has it's good and bad points. One alternative that I put
forth was to create a separate release artifact for recipes as part of
the ZooKeeper project itself. What would this look like?
* zookeeper/trunk/src/recipes would move to
zookeeper/recipes/{trunk,branches,tags,...}
* there would be distinct/separate releases of zookeeper and
zookeeper-recipes artifacts
** these releases would not necessarily be synchronized, although we'd
want to ensure that particular versions of each release work together
** these releases would go through our standard release process - ie
creation of release candidate by a committer and approval by the PMC
** recipes would no longer be included in zookeeper (core) releases
* continue to use the ZOOKEEPER jira, with recipes component
* space in the web and wiki sites for recipes specific pages
* recipes continue to use the std zookeeper mailing lists
* commits to recipes would continue to use our regular commit process
(jira, rtc, etc...)
* we would accept new ZooKeeper committers with the intent that they
focus on their area of expertise, whether this be core (zookeeper
trunk) or recipes.
This last item is the more interesting I think, the other issues are
all pretty straightforward. Although moving to sub (or out entirely)
would allow us to have a separate commit list it would fragment the
ZooKeeper community. I also believe that over time committers to
recipes would more likely get experience with core and start
contributing there as well. The only issue really, if you want to call
it that, is that we'd have committers with guardrails implied. ie
someone with more expertise in recipes than core. However there are
many other Apache projects that take this approach, and do it
successfully. Regardless RTC allows for review both before and after a
change is committed.
What do you think?
Patrick
