Re: HDFS-1623 branching strategy

2011-07-30 Thread Aaron T. Myers
Hey everyone, Just a heads up - I just created the branch in SVN and called it, rather uncreatively, HDFS-1623. -- Aaron T. Myers Software Engineer, Cloudera On Mon, Jul 18, 2011 at 3:24 PM, Aaron T. Myers a...@cloudera.com wrote: Hey everyone, Since it seems that several people are

Re: HDFS-1623 branching strategy

2011-07-19 Thread Steve Loughran
On 08/07/11 22:03, sanjay Radia wrote: On Jul 8, 2011, at 12:52 PM, Allen Wittenauer wrote: On Jul 8, 2011, at 12:22 PM, Suresh Srinivas wrote: Thanks Aaron for sending the details from the meeting that Sanjay had arranged. What meeting? After we posted the design document,

Re: HDFS-1623 branching strategy

2011-07-19 Thread Konstantin Shvachko
Sanjay, 1. I don't see any comments in HDFS-1623 since may 26. And I don't see any updates to the design document. While HA jiras are being committed directly to trunk. And I personally don't like what is being committed. Possible because I don't understand the approach. 2. Whatever the

Re: HDFS-1623 branching strategy

2011-07-19 Thread Suresh Srinivas
CTR or RTC I am seriously considering to revert HDFS-2141 because it is changing the terminology used in several design documents ans publications, which people got used to, and which will create more confusion. Konstantin, you could comment this on jira. This is not a branching issue. This is

Re: HDFS-1623 branching strategy

2011-07-13 Thread sanjay Radia
On Jul 11, 2011, at 3:50 PM, Eli Collins wrote: On Mon, Jul 11, 2011 at 3:05 PM, Jakob Homan jgho...@gmail.com wrote: Eli wrote: This is the *branch* policy, which if I understand correctly, the branch maintainer sets. Thanks, Eli I do not understand the new role branch

Re: HDFS-1623 branching strategy

2011-07-13 Thread Eli Collins
On Wed, Jul 13, 2011 at 5:40 PM, sanjay Radia san...@hortonworks.com wrote: On Jul 11, 2011, at 3:50 PM, Eli Collins wrote: On Mon, Jul 11, 2011 at 3:05 PM, Jakob Homan jgho...@gmail.com wrote: Eli wrote: This is the *branch* policy, which if I understand correctly, the branch

Re: HDFS-1623 branching strategy

2011-07-12 Thread Aaron T. Myers
On Mon, Jul 11, 2011 at 1:28 PM, Suresh Srinivas sur...@hortonworks.comwrote: My preference is to review before commit. Reviewing incremental changes is much more effective than reviewing one mega patch during merge of a branch. I certainly am not in favor of only reviewing a mega-patch

Re: HDFS-1623 branching strategy

2011-07-11 Thread sanjay Radia
Clearly we cannot get all the changes required for HA into 23. We would like to get some changes in that will allow one to build a HA solution before a 23.x release. What we want to discuss is what are the changes worth getting in into the current 0.23. Further many of the changes are

Re: HDFS-1623 branching strategy

2011-07-11 Thread Suresh Srinivas
Sorry for the late reply... My concern is not about creating a branch. Using a branch is some thing we have been doing for a long time for feature development and it is effective. Infact I am eager for branch for HA to be created, to start submitting the patches I am currently working on. Here

Re: HDFS-1623 branching strategy

2011-07-11 Thread Eli Collins
On Fri, Jul 8, 2011 at 3:40 PM, Jakob Homan jgho...@gmail.com wrote: I also have concerns.  While those working on the 1073 branch haven't abused the commit-then-review process, the fact remains that the final merge to trunk is equivalent to a 1.3MB patch that makes far-reaching changes to the

Re: HDFS-1623 branching strategy

2011-07-11 Thread Eli Collins
On Mon, Jul 11, 2011 at 1:28 PM, Suresh Srinivas sur...@hortonworks.com wrote: Sorry for the late reply... My concern is not about creating a branch. Using a branch is some thing we have been doing for a long time for feature development and it is effective. Infact I am eager for branch for

Re: HDFS-1623 branching strategy

2011-07-11 Thread Jakob Homan
Eli wrote: Each change was done individually with it's own jira, patch, and review. People can review the sub-tasks if they don't want to look at the entire patch. The majority of the changes were reviewed before they were committed, when that wasn't the case review feedback was incorporated

Commit then Review (was Re: HDFS-1623 branching strategy)

2011-07-11 Thread Arun C Murthy
On Jul 11, 2011, at 3:05 PM, Jakob Homan wrote: The question at hand is does the community want to go ahead with the 1073 model, or not? My preference is not, but since some in the community like it, I'd suggest making a small tweak to bring it closer in line with the intent of our bylaws,

Re: Commit then Review (was Re: HDFS-1623 branching strategy)

2011-07-11 Thread Arun C Murthy
On Jul 11, 2011, at 3:52 PM, Arun C Murthy wrote: However, blocking RTC en-masse is just counter-productive. We should let folks doing the work decide how best they want to do the work. To be clear: I support RTC for trunk and more stringent reviews for merge. Yet, all these need to be

Re: HDFS-1623 branching strategy

2011-07-11 Thread Jakob Homan
Deep sigh. I'm not blocking RTC on the branch. The branch maintainer gets to decide that. I'm not personally a big fan of it, but since it's not my branch, it's not my call. Either way the final patch to trunk goes by the normal rules that any patch to trunk follows. This is where I

Re: HDFS-1623 branching strategy

2011-07-11 Thread Eli Collins
On Mon, Jul 11, 2011 at 4:05 PM, Jakob Homan jgho...@gmail.com wrote: Deep sigh.  I'm not blocking RTC on the branch.  The branch maintainer gets to decide that.  I'm not personally a big fan of it, but since it's not my branch, it's not my call. Either way the final patch to trunk goes by

Re: HDFS-1623 branching strategy

2011-07-11 Thread Jakob Homan
s/RTC/CTR/g. oy. On Mon, Jul 11, 2011 at 4:05 PM, Jakob Homan jgho...@gmail.com wrote: Deep sigh.  I'm not blocking RTC on the branch.  The branch maintainer gets to decide that.  I'm not personally a big fan of it, but since it's not my branch, it's not my call. Either way the final patch to

Re: HDFS-1623 branching strategy

2011-07-11 Thread Eli Collins
On Mon, Jul 11, 2011 at 4:12 PM, Eli Collins e...@cloudera.com wrote: On Mon, Jul 11, 2011 at 4:05 PM, Jakob Homan jgho...@gmail.com wrote: Deep sigh.  I'm not blocking RTC on the branch.  The branch maintainer gets to decide that.  I'm not personally a big fan of it, but since it's not my

Re: HDFS-1623 branching strategy

2011-07-09 Thread Aaron T. Myers
Hi Jakob, My apologies for taking so long to get back to you - I've been traveling the last few days. On Fri, Jul 8, 2011 at 3:40 PM, Jakob Homan jgho...@gmail.com wrote: The tweak I would make to the suggested process is to require that the final trunk merge have a higher threshold to

Re: HDFS-1623 branching strategy

2011-07-08 Thread Dhruba Borthakur
+1, this sounds like a very good approach. -dhruba On Thu, Jul 7, 2011 at 4:33 PM, Eli Collins e...@cloudera.com wrote: +1 Sounds like this has worked well for the MR2 branch as well. Thanks for volunteering to maintain the branch. On Thu, Jul 7, 2011 at 1:43 PM, Aaron T. Myers

Re: HDFS-1623 branching strategy

2011-07-08 Thread sanjay Radia
On Jul 8, 2011, at 12:52 PM, Allen Wittenauer wrote: On Jul 8, 2011, at 12:22 PM, Suresh Srinivas wrote: Thanks Aaron for sending the details from the meeting that Sanjay had arranged. What meeting? After we posted the design document, some HDFS contributors who

Re: HDFS-1623 branching strategy

2011-07-08 Thread Doug Cutting
On 07/08/2011 12:22 PM, Suresh Srinivas wrote: This is a critical feature for HDFS. This proposal is not exactly what I had envisioned. Why don’t we meet again to discuss and come up with a proposal for branching and development-process? Suresh, What are your concerns with this proposal?

Re: HDFS-1623 branching strategy

2011-07-08 Thread Jakob Homan
I also have concerns. While those working on the 1073 branch haven't abused the commit-then-review process, the fact remains that the final merge to trunk is equivalent to a 1.3MB patch that makes far-reaching changes to the guts of the NN. Asking the reviewers most experienced in this code to

Re: HDFS-1623 branching strategy

2011-07-08 Thread Dhruba Borthakur
Hi suresh, Can you pl explain how we can modify Aaron's proposal to fit your needs? It would be nice to do the HA development in its own branch, is it not? -dhruba On Fri, Jul 8, 2011 at 2:22 PM, Suresh Srinivas srini30...@gmail.comwrote: Thanks Aaron for sending the details from the meeting

HDFS-1623 branching strategy

2011-07-07 Thread Aaron T. Myers
Hello everyone, This has been informally mentioned before, but I think it's best to be completely transparent/explicit about this. We (Sanjay, Suresh, Todd, Eli, myself, and anyone else who wants to help) intend to do the work for HDFS-1623 (High Availability Framework for HDFS NN) on a

Re: HDFS-1623 branching strategy

2011-07-07 Thread Todd Lipcon
Sounds good to me. I think this strategy has worked well on the HDFS-1073 branch -- allowed development to be quite rapid, and at this point all but a couple trivial patches have been explicitly reviewed by a committer (and the others implicitly reviewed since later patches touched the same code