Re: Release plans
On Tue, Feb 16, 2010 at 2:12 PM, Eli Collins wrote: > Hey guys, > > What are the current plans for the 21 release? > Doesn't seem like there is much interest going by the deafening silence Eli. Did it come up last night at the HUG? > The roadmap on the wiki [1] says "minor releases are made regularly, > every few months." The 21 branch was created 5 months ago, and the > release dates in jira for the various core projects have all passed. I > know some projects, eg HBase, would like to get users on more recent > bits. What are the next steps for rolling a release? > Taking a look at blockers -- assuming to make a release, we just need to clear blockers and roll a candidate -- it seems like common/core and hdfs have but a few but mapreduce has a bunch. Too many. Does anyone detect activity in the blocker issues? I've not made a review. Perhaps someone already has? St.Ack
Re: Release plans
On Thu, Feb 18, 2010 at 10:54 AM, Stack wrote: > On Tue, Feb 16, 2010 at 2:12 PM, Eli Collins wrote: >> Hey guys, >> >> What are the current plans for the 21 release? >> > Doesn't seem like there is much interest going by the deafening silence Eli. > > Did it come up last night at the HUG? > It did, Owen gave an update in his talk and there was some follow on discussion. The question of whether it makes sense to rebase 21 on trunk came up, since 21 has gotten really out of date since it was branched. What do you and others think of rebasing the 21 branch on trunk once security is feature complete next month, and then targeting an initial release a month after that? The rationale being that 21 branch is old, has not been getting the backports it needs, and the community should be releasing relatively recent bits. In the mean time we can focus on 21/trunk blockers. It doesn't look like there are a ton more trunk blockers than 21 blockers. I know people, HBase in particular, have been waiting for a 21 release for a while so delaying further is a pain, and we should consider rolling what we've got, the hope would be that the 21 branch would be more widely useful if it has more recent features and fixes. >> The roadmap on the wiki [1] says "minor releases are made regularly, >> every few months." The 21 branch was created 5 months ago, and the >> release dates in jira for the various core projects have all passed. I >> know some projects, eg HBase, would like to get users on more recent >> bits. What are the next steps for rolling a release? >> > > Taking a look at blockers -- assuming to make a release, we just need > to clear blockers and roll a candidate -- it seems like common/core > and hdfs have but a few but mapreduce has a bunch. Too many. > > Does anyone detect activity in the blocker issues? I've not made a > review. Perhaps someone already has? A decent number of the MR ones are patch available and quite a few others are documentation related, nothing too scary. The core/hdfs ones are pretty doable. We should make another pass to confirm that they're all actually blockers. I'm happy to start working on the blockers if that will help push the release forward. Thanks, Eli
Re: Release plans
Eli Collins wrote: What are the current plans for the 21 release? Personally, I'd rather re-branch 21 from trunk in early March. I believe Y!'s security changes will be feature-complete in trunk by then. Symlinks and end-to-end Avro should also hopefully be committed by then. I'd then propose we roll a 0.21.0 (alpha) release one month later, in early April. Henceforth, I'd like to see the project: - set branch dates at 6-month intervals - roll alpha releases one month after branch dates - roll bugfix releases as needed thereafter At the onset of each six-month period we'd also need to decide what sort of release we'll make, major or minor. (Minor releases allow new features, but remove no deprecated APIs. Only major releases are permitted to remove previously deprecated APIs. All pre-1.0 releases are major.) We'd also designate a release master for each release, to drive the process: making the branches, rolling the artifacts, and calling the votes, etc. Do we have any release master candidate volunteers for 0.21? Doug
Re: Release plans
On 2/18/10 11:29 AM, "Doug Cutting" wrote: > Eli Collins wrote: >> What are the current plans for the 21 release? > > Personally, I'd rather re-branch 21 from trunk in early March. I > believe Y!'s security changes will be feature-complete in trunk by then. > Symlinks and end-to-end Avro should also hopefully be committed by then. > > I'd then propose we roll a 0.21.0 (alpha) release one month later, in > early April. +1
Re: Release plans
I have seen/heard about people using the 0.21 branch. Won't it disturbing to them that a new 0.21 branch that they might sync to in the future will have a whole bunch of new code/features that they were not testing with earlier? I thought that there is some benefit in rolling a 0.22 sometime in the future and then marking 0.21 as "not for production usage". thanks, dhruba On Thu, Feb 18, 2010 at 11:29 AM, Doug Cutting wrote: > Eli Collins wrote: > >> What are the current plans for the 21 release? >> > > Personally, I'd rather re-branch 21 from trunk in early March. I believe > Y!'s security changes will be feature-complete in trunk by then. Symlinks > and end-to-end Avro should also hopefully be committed by then. > > I'd then propose we roll a 0.21.0 (alpha) release one month later, in early > April. > > Henceforth, I'd like to see the project: > - set branch dates at 6-month intervals > - roll alpha releases one month after branch dates > - roll bugfix releases as needed thereafter > > At the onset of each six-month period we'd also need to decide what sort of > release we'll make, major or minor. (Minor releases allow new features, but > remove no deprecated APIs. Only major releases are permitted to remove > previously deprecated APIs. All pre-1.0 releases are major.) > > We'd also designate a release master for each release, to drive the > process: making the branches, rolling the artifacts, and calling the votes, > etc. Do we have any release master candidate volunteers for 0.21? > > Doug > -- Connect to me at http://www.facebook.com/dhruba
Re: Release plans
On Feb 18, 2010, at 11:29 AM, Doug Cutting wrote: My presentation basically said that: * Yahoo is currently running Yahoo's 0.20.8 on all 25,000 of our nodes. * We have made substantial numbers of feature back ports from trunk into our branch. Including: * improved Capacity scheduler * run MapReduce tasks as the real user * Hadoop 0.21 isn't stable yet and still has 28 blockers. * Furthermore, there are missing back ports that have been fixed in trunk but not 0.21. * We have a very tight timeline for adding security, which we can't slip * feature complete in february * deploy to first integration cluster in april * completely deployed on all clusters in august * To reduce risk, we are back porting security in to a branch of Yahoo's 0.20 branch. * By the time we are ready for the next version after security, the current 0.21 will be too stale to be interesting * So Yahoo will probably skip straight from Yahoo 0.20 with security to 0.22 in 2011. * Someone outside of Yahoo needs to step up and start addressing the blockers to get 0.21 released. Personally, I'd rather re-branch 21 from trunk in early March. Rather than set a date, I think it is time to move to feature-based releases. We'd need to vote on the feature set, but looking for security, end-to-end avro, and symlinks seems like a reasonable list. That will avoid the large rush of commits the last week of the deadline, which has been counter productive. -- Owen
Re: Release plans
> > * Someone outside of Yahoo needs to step up and start addressing the > blockers to get 0.21 released. > Thanks for the update on the state of the world at Yahoo!, Owen--very helpful. I think the community will step up to knock down some of the blockers once we resolve what should be in the 0.21 release: the current branch, or a rebase on trunk. Do you/Yahoo! have a preference on that front? Rather than set a date, I think it is time to move to feature-based > releases. We'd need to vote on the feature set, but looking for security, > end-to-end avro, and symlinks seems like a reasonable list. That will avoid > the large rush of commits the last week of the deadline, which has been > counter productive. > Could someone give an example of a successful open source project that follows a feature-based release cycle? From the research I've done, the regular drumbeat of time-based releases seem to be more conducive to project health. Always interested to hear otherwise. The example Owen cites of "the large rush of commits the last week of the deadline" is certainly a good argument in favor of feature-based releases; I'm curious to hear more. Thanks, Jeff
Re: Release plans
On Feb 18, 2010, at 11:28 AM, Eli Collins wrote: On Thu, Feb 18, 2010 at 10:54 AM, Stack wrote: On Tue, Feb 16, 2010 at 2:12 PM, Eli Collins wrote: Hey guys, What are the current plans for the 21 release? Doesn't seem like there is much interest going by the deafening silence Eli. Did it come up last night at the HUG? It did, Owen gave an update in his talk and there was some follow on discussion. The question of whether it makes sense to rebase 21 on trunk came up, since 21 has gotten really out of date since it was branched. What do you and others think of rebasing the 21 branch on trunk once security is feature complete next month, and then targeting an initial release a month after that? The rationale being that 21 branch is old, has not been getting the backports it needs, and the community should be releasing relatively recent bits. In the mean time we can focus on 21/trunk blockers. It doesn't look like there are a ton more trunk blockers than 21 blockers. I know people, HBase in particular, have been waiting for a 21 release for a while so delaying further is a pain, and we should consider rolling what we've got, the hope would be that the 21 branch would be more widely useful if it has more recent features and fixes. If the community want the append features fast then we are better off using the original 21 branch. Basing a 21 off trunk will mean that there are more features with a new set of blockers and and hence will take longer to fix. sanjay
Re: Release plans
Owen O'Malley wrote: Rather than set a date, I think it is time to move to feature-based releases. The problem with feature-based releases is that one feature can delay releases arbitrarily, so a single feature can hold all others hostage. Rather I think we need to be willing to remove or disable a feature in a branch if it proves problematic. If an addition causes regressions that are not resolved promptly then that addition might be removed or disabled. This levels the playing field: features that doesn't cause regressions can be made available in a release in a timely, predictable manner and are not held back by more problematic features. Doug
Re: Release plans
On Feb 18, 2010, at 3:06 PM, Jeff Hammerbacher wrote: I think the community will step up to knock down some of the blockers once we resolve what should be in the 0.21 release: the current branch, or a rebase on trunk. Do you/Yahoo! have a preference on that front? Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the timing, Yahoo will probably never deploy it. (It take months to push a release through QA and production testing, as I wrote the security release will hit the pipeline this year (code complete in february, first integration cluster in april, on all production clusters by august). Yahoo can't handle another big release until january 2011. Personally, I'd prefer to rebase 0.21, especially after we have the Maven story straightened out. Generating good poms would be a huge win for downstream projects. One big concern is that backwards incompatibility is a big cost. Especially if 0.21 (like 0.19) never gets wide deployment, I'd like to start a vote that we don't make any API incompatible in 0.22 relative to 0.20. -- Owen
Re: Release plans
Thanks Owen, that's useful information. It sounds like the API incompatibility vote can be a separate issue. Do we have consensus around rebasing on 0.21? Anyone already testing on 0.21 who would be upset if the current branch were to be retired? On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley wrote: > On Feb 18, 2010, at 3:06 PM, Jeff Hammerbacher wrote: > > I think the community will step up to knock down some of the >> blockers once we resolve what should be in the 0.21 release: the current >> branch, or a rebase on trunk. Do you/Yahoo! have a preference on that >> front? >> > > Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the > timing, Yahoo will probably never deploy it. (It take months to push a > release through QA and production testing, as I wrote the security release > will hit the pipeline this year (code complete in february, first > integration cluster in april, on all production clusters by august). Yahoo > can't handle another big release until january 2011. > > Personally, I'd prefer to rebase 0.21, especially after we have the Maven > story straightened out. Generating good poms would be a huge win for > downstream projects. > > One big concern is that backwards incompatibility is a big cost. Especially > if 0.21 (like 0.19) never gets wide deployment, I'd like to start a vote > that we don't make any API incompatible in 0.22 relative to 0.20. > > -- Owen >
Re: Release plans
On Thu, Feb 18, 2010 at 5:19 PM, Jeff Hammerbacher wrote: > Thanks Owen, that's useful information. It sounds like the API > incompatibility vote can be a separate issue. > > Do we have consensus around rebasing on 0.21? Anyone already testing on 0.21 > who would be upset if the current branch were to be retired? > One question I have is what to do with the JIRA tickets - we probably need some kind of bulk edit, right? Anything that's currently resolved with fix version 22 needs to be changed over to fix version 21? > On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley wrote: > >> On Feb 18, 2010, at 3:06 PM, Jeff Hammerbacher wrote: >> >> I think the community will step up to knock down some of the >>> blockers once we resolve what should be in the 0.21 release: the current >>> branch, or a rebase on trunk. Do you/Yahoo! have a preference on that >>> front? >>> >> >> Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the >> timing, Yahoo will probably never deploy it. (It take months to push a >> release through QA and production testing, as I wrote the security release >> will hit the pipeline this year (code complete in february, first >> integration cluster in april, on all production clusters by august). Yahoo >> can't handle another big release until january 2011. >> >> Personally, I'd prefer to rebase 0.21, especially after we have the Maven >> story straightened out. Generating good poms would be a huge win for >> downstream projects. >> >> One big concern is that backwards incompatibility is a big cost. Especially >> if 0.21 (like 0.19) never gets wide deployment, I'd like to start a vote >> that we don't make any API incompatible in 0.22 relative to 0.20. >> >> -- Owen >> >
Re: Release plans
Todd Lipcon wrote: One question I have is what to do with the JIRA tickets - we probably need some kind of bulk edit, right? Anything that's currently resolved with fix version 22 needs to be changed over to fix version 21? Yes, that should be done if we decide to rebase the 0.21 branch. Doug
Re: Release plans
On 2/18/2010 5:19 PM, Jeff Hammerbacher wrote: Do we have consensus around rebasing on 0.21? Anyone already testing on 0.21 who would be upset if the current branch were to be retired? Rebasing 0.21 will further delay the release. In current 0.21 branch there is some 28 blockers, which will take a couple of weeks to fix. The rebased 0.21 will add to this more issues, and therefore more time. Based on the experience I had time to stabilize the release is measured in months rather than weeks. --Konstantin On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley wrote: > On Feb 18, 2010, at 3:06 PM, Jeff Hammerbacher wrote: > >I think the community will step up to knock down some of the >> blockers once we resolve what should be in the 0.21 release: the current >> branch, or a rebase on trunk. Do you/Yahoo! have a preference on that >> front? >> > > Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the > timing, Yahoo will probably never deploy it. (It take months to push a > release through QA and production testing, as I wrote the security release > will hit the pipeline this year (code complete in february, first > integration cluster in april, on all production clusters by august). Yahoo > can't handle another big release until january 2011. > > Personally, I'd prefer to rebase 0.21, especially after we have the Maven > story straightened out. Generating good poms would be a huge win for > downstream projects. > > One big concern is that backwards incompatibility is a big cost. Especially > if 0.21 (like 0.19) never gets wide deployment, I'd like to start a vote > that we don't make any API incompatible in 0.22 relative to 0.20. > > -- Owen >
Re: Release plans
On Thu, Feb 18, 2010 at 6:08 PM, Konstantin Shvachko wrote: > On 2/18/2010 5:19 PM, Jeff Hammerbacher wrote: >> >> Do we have consensus around rebasing on 0.21? Anyone already testing on >> 0.21 >> who would be upset if the current branch were to be retired? > > Rebasing 0.21 will further delay the release. > In current 0.21 branch there is some 28 blockers, > which will take a couple of weeks to fix. > The rebased 0.21 will add to this more issues, and therefore > more time. Based on the experience I had time to stabilize > the release is measured in months rather than weeks. I agree that we're probably talking months rather than weeks. However, I see a lot of the stabilization time as a fixed cost regardless of the number of changes. Certainly there is an O(n) component too, but I don't think the stabilization time of a rebased 21 is double the stabilization time of current 21. Maybe more like 30% more? Do you disagree? -Todd > --Konstantin > > >> >> On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley wrote: >> >>> > On Feb 18, 2010, at 3:06 PM, Jeff Hammerbacher wrote: >>> > >>> > I think the community will step up to knock down some of the >> blockers once we resolve what should be in the 0.21 release: the >> current >> branch, or a rebase on trunk. Do you/Yahoo! have a preference on >> that >> front? >> >>> >>> > >>> > Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the >>> > timing, Yahoo will probably never deploy it. (It take months to push a >>> > release through QA and production testing, as I wrote the security >>> > release >>> > will hit the pipeline this year (code complete in february, first >>> > integration cluster in april, on all production clusters by august). >>> > Yahoo >>> > can't handle another big release until january 2011. >>> > >>> > Personally, I'd prefer to rebase 0.21, especially after we have the >>> > Maven >>> > story straightened out. Generating good poms would be a huge win for >>> > downstream projects. >>> > >>> > One big concern is that backwards incompatibility is a big cost. >>> > Especially >>> > if 0.21 (like 0.19) never gets wide deployment, I'd like to start a >>> > vote >>> > that we don't make any API incompatible in 0.22 relative to 0.20. >>> > >>> > -- Owen >>> > > >
Re: Release plans
If we include symlinks, security and Avro in 0.21, then what's the feature set for 0.22? Do we have any big items planned? Thanks, Tomer On Thu, Feb 18, 2010 at 6:12 PM, Todd Lipcon wrote: > On Thu, Feb 18, 2010 at 6:08 PM, Konstantin Shvachko > wrote: > > On 2/18/2010 5:19 PM, Jeff Hammerbacher wrote: > >> > >> Do we have consensus around rebasing on 0.21? Anyone already testing on > >> 0.21 > >> who would be upset if the current branch were to be retired? > > > > Rebasing 0.21 will further delay the release. > > In current 0.21 branch there is some 28 blockers, > > which will take a couple of weeks to fix. > > The rebased 0.21 will add to this more issues, and therefore > > more time. Based on the experience I had time to stabilize > > the release is measured in months rather than weeks. > > I agree that we're probably talking months rather than weeks. However, > I see a lot of the stabilization time as a fixed cost regardless of > the number of changes. Certainly there is an O(n) component too, but I > don't think the stabilization time of a rebased 21 is double the > stabilization time of current 21. Maybe more like 30% more? Do you > disagree? > > -Todd > > > > --Konstantin > > > > > >> > >> On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley > wrote: > >> > >>> > On Feb 18, 2010, at 3:06 PM, Jeff Hammerbacher wrote: > >>> > > >>> >I think the community will step up to knock down some of the > > >> blockers once we resolve what should be in the 0.21 release: the > >> current > >> branch, or a rebase on trunk. Do you/Yahoo! have a preference on > >> that > >> front? > >> > >>> > >>> > > >>> > Yahoo doesn't care. Even if we rebase the 0.21 branch, because of > the > >>> > timing, Yahoo will probably never deploy it. (It take months to push > a > >>> > release through QA and production testing, as I wrote the security > >>> > release > >>> > will hit the pipeline this year (code complete in february, first > >>> > integration cluster in april, on all production clusters by august). > >>> > Yahoo > >>> > can't handle another big release until january 2011. > >>> > > >>> > Personally, I'd prefer to rebase 0.21, especially after we have the > >>> > Maven > >>> > story straightened out. Generating good poms would be a huge win for > >>> > downstream projects. > >>> > > >>> > One big concern is that backwards incompatibility is a big cost. > >>> > Especially > >>> > if 0.21 (like 0.19) never gets wide deployment, I'd like to start a > >>> > vote > >>> > that we don't make any API incompatible in 0.22 relative to 0.20. > >>> > > >>> > -- Owen > >>> > > > > > > -- Tomer Shiran Director of Product Management | MapR Technologies (www.mapr.com) | 650-804-8657
Re: Release plans
On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley wrote: > Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the > timing, Yahoo will probably never deploy it. (It take months to push a > release through QA and production testing, as I wrote the security release > will hit the pipeline this year (code complete in february, first > integration cluster in april, on all production clusters by august). Yahoo > can't handle another big release until january 2011. I haven't done a survey but my guess is that the HBase crew would favor putting out the current 0.21 branch as 0.21 rather than rebasing. To us, hdfs and its flush, in testing, runs great. I'm with Konstantin and Sanjay that a release is pretty close and that a rebasing pulling in new features means more months even allowing for the Todd stabilizing constant. If the project is not getting any heavy-duty Y!-fu till ~January 2011, that sounds like hadoop 0.23 time, rather than 0.22, to me, that is if we want to get the project back on 6-month cycles again (> 1 year between major releases ain't healthy). St.Ack
Re: Release plans
On Thu, Feb 18, 2010 at 8:36 PM, Stack wrote: > On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley wrote: >> Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the >> timing, Yahoo will probably never deploy it. (It take months to push a >> release through QA and production testing, as I wrote the security release >> will hit the pipeline this year (code complete in february, first >> integration cluster in april, on all production clusters by august). Yahoo >> can't handle another big release until january 2011. > > I haven't done a survey but my guess is that the HBase crew would > favor putting out the current 0.21 branch as 0.21 rather than > rebasing. To us, hdfs and its flush, in testing, runs great. I'm > with Konstantin and Sanjay that a release is pretty close and that a > rebasing pulling in new features means more months even allowing for > the Todd stabilizing constant. Given that Yahoo and others plan to skip 21, and the HBase crowd could actually use 21 and have invested in the current branch's contents, it sounds like we should try to get the current branch released. Anyone object? Thanks, Eli
Re: Release plans
+1 from me. I agree with Stack; faster release cycles will help keep the project focused and get code testing and soaking in more environments. - Aaron On Fri, Feb 19, 2010 at 1:44 PM, Eli Collins wrote: > On Thu, Feb 18, 2010 at 8:36 PM, Stack wrote: > > On Thu, Feb 18, 2010 at 5:02 PM, Owen O'Malley > wrote: > >> Yahoo doesn't care. Even if we rebase the 0.21 branch, because of the > >> timing, Yahoo will probably never deploy it. (It take months to push a > >> release through QA and production testing, as I wrote the security > release > >> will hit the pipeline this year (code complete in february, first > >> integration cluster in april, on all production clusters by august). > Yahoo > >> can't handle another big release until january 2011. > > > > I haven't done a survey but my guess is that the HBase crew would > > favor putting out the current 0.21 branch as 0.21 rather than > > rebasing. To us, hdfs and its flush, in testing, runs great. I'm > > with Konstantin and Sanjay that a release is pretty close and that a > > rebasing pulling in new features means more months even allowing for > > the Todd stabilizing constant. > > Given that Yahoo and others plan to skip 21, and the HBase crowd could > actually use 21 and have invested in the current branch's contents, it > sounds like we should try to get the current branch released. Anyone > object? > > Thanks, > Eli >
Re: Release plans
Eli Collins wrote: Given that Yahoo and others plan to skip 21, and the HBase crowd could actually use 21 and have invested in the current branch's contents, it sounds like we should try to get the current branch released. My concern is more about when the next branch from trunk will be made. If we embrace a six-month trunk branch cycle, then the next branch from trunk should be created soon, regardless of whether we call it 21 or 22. Also note that this decision has compatibility implications, unless we decide to, post-fact, declare that 0.20 was actually a major release (1.0) and that 0.21 (1.1) and 0.22 (1.2) are minor, feature releases, that may add deprecations and features but may not remove any or otherwise change APIs incompatibly. Doug
Re: Release plans
On Fri, Feb 19, 2010 at 1:58 PM, Doug Cutting wrote: > Eli Collins wrote: >> >> Given that Yahoo and others plan to skip 21, and the HBase crowd could >> actually use 21 and have invested in the current branch's contents, it >> sounds like we should try to get the current branch released. > > My concern is more about when the next branch from trunk will be made. If we > embrace a six-month trunk branch cycle, then the next branch from trunk > should be created soon, regardless of whether we call it 21 or 22. > Does there need to be a dependency? We could still branch 22 once security, avro, etc are in, even if that's only a couple months from now. Minor releases are supposed to come out in a matter of months anyway, since we haven't released a major version yet. > Also note that this decision has compatibility implications, unless we > decide to, post-fact, declare that 0.20 was actually a major release (1.0) > and that 0.21 (1.1) and 0.22 (1.2) are minor, feature releases, that may add > deprecations and features but may not remove any or otherwise change APIs > incompatibly. Could the vote on whether 22 is backwards compatible with 20 be independent of what we call 1.0? Guess it depends on what type of backwards compatibility we're talking about (eg API vs wire). Given that already branched 21 a while back, it seems like we should be able to release it, regardless of how a compatibility vote goes. Thanks, Eli
Re: Release plans
Eli Collins wrote: My concern is more about when the next branch from trunk will be made. Does there need to be a dependency? No. I just wanted to note that, from my point of view, releasing from the existing 0.21 branch is not sufficient, that, regardless, we still need to release from trunk soon, and we need a schedule going forward for when future branches from trunk will be made. Aside from resolving compatibility issues (more on that below) what we perhaps need are one or two release master volunteers, for a release based on trunk and perhaps for a release based on the 0.21 branch. Could the vote on whether 22 is backwards compatible with 20 be independent of what we call 1.0? We minimally need to declare whether releases are major or minor. The convention has been that all 0.X releases are major, 1.0 is major, 1.1, 1.2, etc. are minor, 2.0 is major, 2.1, 2.2, etc are minor and so on. We could amend this convention, but things might get confusing. For example, we could declare that we just have a sequence of numerically increasing releases, some major, some minor, but that you can't tell which is which from the number. Or we could, like Sun did with Java, move the decimal point, and say that 0.20 is just 2.0, and that, instead of 0.21 our next release is 2.1. Or we could, as I suggested in my previous message, retroactively consider 0.20 to be 1.0, and then make our next release 1.1. None are particularly attractive options. Doug
Re: Release plans
Hadoop 2 EE Version 5? On Fri, Feb 19, 2010 at 5:49 PM, Doug Cutting wrote: > Eli Collins wrote: > >> My concern is more about when the next branch from trunk will be made. >>> >> >> Does there need to be a dependency? >> > > No. I just wanted to note that, from my point of view, releasing from the > existing 0.21 branch is not sufficient, that, regardless, we still need to > release from trunk soon, and we need a schedule going forward for when > future branches from trunk will be made. > > Aside from resolving compatibility issues (more on that below) what we > perhaps need are one or two release master volunteers, for a release based > on trunk and perhaps for a release based on the 0.21 branch. > > > Could the vote on whether 22 is backwards compatible with 20 be >> independent of what we call 1.0? >> > > We minimally need to declare whether releases are major or minor. The > convention has been that all 0.X releases are major, 1.0 is major, 1.1, 1.2, > etc. are minor, 2.0 is major, 2.1, 2.2, etc are minor and so on. We could > amend this convention, but things might get confusing. For example, we > could declare that we just have a sequence of numerically increasing > releases, some major, some minor, but that you can't tell which is which > from the number. Or we could, like Sun did with Java, move the decimal > point, and say that 0.20 is just 2.0, and that, instead of 0.21 our next > release is 2.1. Or we could, as I suggested in my previous message, > retroactively consider 0.20 to be 1.0, and then make our next release 1.1. > None are particularly attractive options. > > Doug >
Re: Release plans
On Fri, Feb 19, 2010 at 2:49 PM, Doug Cutting wrote: > Eli Collins wrote: >>> >>> My concern is more about when the next branch from trunk will be made. >> >> Does there need to be a dependency? > > No. I just wanted to note that, from my point of view, releasing from the > existing 0.21 branch is not sufficient, that, regardless, we still need to > release from trunk soon, and we need a schedule going forward for when > future branches from trunk will be made. > Agreed. Releasing a 21 won't settle the release process question or tell us when the first major release is. Maybe we could get 21 behind us and have a separate discussion covering those. >> Could the vote on whether 22 is backwards compatible with 20 be >> independent of what we call 1.0? > > We minimally need to declare whether releases are major or minor. I was assuming 21 would be another minor release, didn't hear otherwise when it was branched. I didn't interpret the compatibility vote as a suggestion that we should retroactively consider 20 to be the first major release, but rather a suggestion for voting that we don't remove APIs in 22 that were deprecated in 20. Owen, please correct me if I'm wrong. Thanks, Eli
Re: Release plans
Eli Collins wrote: Maybe we could get 21 behind us and have a separate discussion covering those. I'm personally more interested in that discussion than in what's currently in the 0.21 branch, but others, like Stack, may reasonably be of the opposite persuasion. So rather than putting one before the other, can we pursue both in parallel? I was assuming 21 would be another minor release, didn't hear otherwise when it was branched. We've never officially had a minor release. All of our releases have been either major or bugfix. A minor release adds features but does not remove any deprecated APIs. So making 0.21 a minor release would be a change in the rules. Have no deprecations in fact been removed between the 0.20 and 0.21 branches? Perhaps we can agree that, after 0.20, no deprecations will be removed until 1.0, that all pre-1.0 releases after 0.20 are now minor instead of major, that 0.20 was effectively 1.0 but we're not going to rename it, and that 1.0 will effectively be 2.0, etc. Could that work? Doug
Re: Release plans
On 2/19/10 3:38 PM, "Doug Cutting" wrote: > Perhaps we can agree that, after 0.20, no deprecations will be removed > until 1.0, that all pre-1.0 releases after 0.20 are now minor instead of > major, that 0.20 was effectively 1.0 but we're not going to rename it, > and that 1.0 will effectively be 2.0, etc. Could that work? FWIW: I'm still very much opposed to any 1.0 that isn't ABI-wire compatible going forward.
Re: Release plans
On Fri, Feb 19, 2010 at 3:38 PM, Doug Cutting wrote: > Eli Collins wrote: >> >> Maybe we could get 21 behind >> us and have a separate discussion covering those. > > I'm personally more interested in that discussion than in what's currently > in the 0.21 branch, but others, like Stack, may reasonably be of the > opposite persuasion. So rather than putting one before the other, can we > pursue both in parallel? > Don't want to hold up the release process, version, compatibility discussion, or have it hold up work on the 21 release (most 21 blockers are probably trunk blockers so probably not much wasted effort either way). >> I was assuming 21 would be another minor release, didn't hear >> otherwise when it was branched. > > We've never officially had a minor release. All of our releases have been > either major or bugfix. A minor release adds features but does not remove > any deprecated APIs. So making 0.21 a minor release would be a change in > the rules. > Good to know, the versioning scheme (major.minor.point) outlined on the wiki makes it seem like there's only been minor and point releases. Can we make a decision on basing 21 on the current branch and if it's decided that 22 can't remove stuff that was in 20 we'll go back and do the necessary additions on 21 and trunk? Suspect that decision will take a lot more back and forth, but needs to conclude before 21 is released. Thanks, Eli
Re: Release plans
On Fri, Feb 19, 2010 at 4:23 PM, Eli Collins wrote: > Can we make a decision on basing 21 on the current branch and if it's > decided that 22 can't remove stuff that was in 20 we'll go back and do > the necessary additions on 21 and trunk? Suspect that decision will > take a lot more back and forth, but needs to conclude before 21 is > released. > Lets. Regards 0.21/current-branch release, as has been suggested above, first we need to figure the release master. No release master, no release. If we have a release master, then I suggest we vote on current branch being released as 0.21 as soon as the blockers are cleared. I don't think we need muddy the above vote with whether or not 0.21 maintains API combatibility with 0.20. IMO, it must (because Y! want to have the 0.20 API in place when January 2011 rolls around). This makes 0.21 a "minor" release -- something we've not done before (For the record, I also had a misunderstanding that what we were doing up to this was major and patch only). So, part of the release process would involve ensuring no removed deprecations, etc. As DC has been saying, this requirement that releases between now and January 2011 not change APIs makes 0.20 retroactively into a "major" release. 0.20 is the release where major shifted left in our versioning scheme and minor releases came into play. 0.21 and 0.22 will be minor releases. Can we just acknowledge this fact, that there was a step at version 0.20, update the wiki around versioning -- its currently wrong anyways as Elis' points out -- and just move on? Going back and calling 0.20 a 1.0 seems more apt to create confusion and besides, I'm with Allen that hadoop 1.0 needs wire compatibility before the 1.0 can roll around. Thanks, St.Ack P.S. +1 on branching as soon as avro and security are in, etc.
Re: Release plans
Well, since someone has to get the ball rolling as far as release masters, I'll nominate Stack and/or someone hbase related for 0.21 with the primary goal of being "soon"? They get a big win from append and others will gain from the expanded mapreduce lib, better schedulers, etc. There are a lot of new features and some major changes (project split) already in the 0.21 branch, so IMO it's worth considering a release with minimal backports, rather than make binding decisions about 0.22 before 0.21 is even in the wild. -Jay PS sorry Stack On Feb 20, 2010, at 5:04 PM, Stack wrote: On Fri, Feb 19, 2010 at 4:23 PM, Eli Collins wrote: Can we make a decision on basing 21 on the current branch and if it's decided that 22 can't remove stuff that was in 20 we'll go back and do the necessary additions on 21 and trunk? Suspect that decision will take a lot more back and forth, but needs to conclude before 21 is released. Lets. Regards 0.21/current-branch release, as has been suggested above, first we need to figure the release master. No release master, no release. If we have a release master, then I suggest we vote on current branch being released as 0.21 as soon as the blockers are cleared. I don't think we need muddy the above vote with whether or not 0.21 maintains API combatibility with 0.20. IMO, it must (because Y! want to have the 0.20 API in place when January 2011 rolls around). This makes 0.21 a "minor" release -- something we've not done before (For the record, I also had a misunderstanding that what we were doing up to this was major and patch only). So, part of the release process would involve ensuring no removed deprecations, etc. As DC has been saying, this requirement that releases between now and January 2011 not change APIs makes 0.20 retroactively into a "major" release. 0.20 is the release where major shifted left in our versioning scheme and minor releases came into play. 0.21 and 0.22 will be minor releases. Can we just acknowledge this fact, that there was a step at version 0.20, update the wiki around versioning -- its currently wrong anyways as Elis' points out -- and just move on? Going back and calling 0.20 a 1.0 seems more apt to create confusion and besides, I'm with Allen that hadoop 1.0 needs wire compatibility before the 1.0 can roll around. Thanks, St.Ack P.S. +1 on branching as soon as avro and security are in, etc.
RE: Release plans
I'm sorry for breaking into your discussion as an outsider, but I'm very curious about the security features you are planning to roll out in March. Where can I find information about this? Best regards, Evert Lammerts > -Original Message- > From: Jay Booth [mailto:jaybo...@gmail.com] > Sent: maandag 22 februari 2010 5:55 > To: general@hadoop.apache.org > Cc: general@hadoop.apache.org > Subject: Re: Release plans > > Well, since someone has to get the ball rolling as far as release > masters, I'll nominate Stack and/or someone hbase related for 0.21 > with the primary goal of being "soon"? They get a big win from append > and others will gain from the expanded mapreduce lib, better > schedulers, etc. There are a lot of new features and some major > changes (project split) already in the 0.21 branch, so IMO it's worth > considering a release with minimal backports, rather than make binding > decisions about 0.22 before 0.21 is even in the wild. > > -Jay > > PS sorry Stack > > On Feb 20, 2010, at 5:04 PM, Stack wrote: > > > On Fri, Feb 19, 2010 at 4:23 PM, Eli Collins > wrote: > >> Can we make a decision on basing 21 on the current branch and if > it's > >> decided that 22 can't remove stuff that was in 20 we'll go back and > >> do > >> the necessary additions on 21 and trunk? Suspect that decision will > >> take a lot more back and forth, but needs to conclude before 21 is > >> released. > >> > > > > Lets. > > > > Regards 0.21/current-branch release, as has been suggested above, > > first we need to figure the release master. No release master, no > > release. If we have a release master, then I suggest we vote on > > current branch being released as 0.21 as soon as the blockers are > > cleared. > > > > I don't think we need muddy the above vote with whether or not 0.21 > > maintains API combatibility with 0.20. IMO, it must (because Y! want > > to have the 0.20 API in place when January 2011 rolls around). This > > makes 0.21 a "minor" release -- something we've not done before (For > > the record, I also had a misunderstanding that what we were doing up > > to this was major and patch only). So, part of the release process > > would involve ensuring no removed deprecations, etc. > > > > As DC has been saying, this requirement that releases between now and > > January 2011 not change APIs makes 0.20 retroactively into a "major" > > release. 0.20 is the release where major shifted left in our > > versioning scheme and minor releases came into play. 0.21 and 0.22 > > will be minor releases. Can we just acknowledge this fact, that there > > was a step at version 0.20, update the wiki around versioning -- its > > currently wrong anyways as Elis' points out -- and just move on? > > Going back and calling 0.20 a 1.0 seems more apt to create confusion > > and besides, I'm with Allen that hadoop 1.0 needs wire compatibility > > before the 1.0 can roll around. > > > > Thanks, > > St.Ack > > P.S. +1 on branching as soon as avro and security are in, etc. smime.p7s Description: S/MIME cryptographic signature
Re: Release plans
On Feb 19, 2010, at 3:14 PM, Eli Collins wrote: On Fri, Feb 19, 2010 at 2:49 PM, Doug Cutting wrote: Eli Collins wrote: My concern is more about when the next branch from trunk will be made. Does there need to be a dependency? No. I just wanted to note that, from my point of view, releasing from the existing 0.21 branch is not sufficient, that, regardless, we still need to release from trunk soon, and we need a schedule going forward for when future branches from trunk will be made. Agreed. Releasing a 21 won't settle the release process question or tell us when the first major release is. Maybe we could get 21 behind us and have a separate discussion covering those. Could the vote on whether 22 is backwards compatible with 20 be independent of what we call 1.0? We minimally need to declare whether releases are major or minor. I was assuming 21 would be another minor release, didn't hear otherwise when it was branched. I didn't interpret the compatibility vote as a suggestion that we should retroactively consider 20 to be the first major release, but rather a suggestion for voting that we don't remove APIs in 22 that were deprecated in 20. Agreed. Rather than renaming versions, lets vote on the compatibility rules for 22: 22 is API compatible with 20. Owen, please correct me if I'm wrong. Thanks, Eli
Re: Release plans
Sanjay Radia wrote: Rather than renaming versions, lets vote on the compatibility rules for 22: 22 is API compatible with 20. I think if we elect to go this way then we should make the rule more general, i.e.: all releases after 0.20 are to be API-compatible with 0.20 until we state otherwise (probably 1.0). If we both release 0.21 as currently branched and we branch 0.22 from trunk in a few weeks, then we might want to have an API-compatible 0.23 too. Doug
