Getting a release out is critical. Otherwise, IMO, the project is dead but for the stiffening.
Thanks Tom for stepping up to play the RM role for a 0.21. Regarding Steve's call for what we can offer Tom to help along the release, the little flea hbase can test its use case on 0.21.0 candidates and we can probably take on a few of the HDFS blockers. I also like Steve's suggestion of a council to figure what makes the 0.21 cut (We're talking security and avro in 0.22, not 0.21 right?). Allen in his note raises another issue beyond the release blockage that I believe warrants further discussion. The "forks" maintained by the big contributors currently cloud (undermine?) the Apache release and the amount and pain involved patch wrangling is a friction on forward progress especially as versions deviate further. Perhaps this state is inevitable when the stakes are this high, where there are new releases rolled out across thousands of machines carrying biz-critical data that cannot fail. Having the Apache project release reliably on a schedule should help especially if posted fixes get reviewed and committed. Formally adopting the stable/unstable labeling could help too. I'd be interested in more discussion of the latter point and in what the community thinks we can do to address the current, IMO, stasis-making set of circumstances. Thanks, St.Ack On Thu, Mar 25, 2010 at 6:22 AM, Steve Loughran <ste...@apache.org> wrote: > Tom White wrote: >> >> I agree that getting the release process restarted is of utmost >> importance to the project. To help make that happen I'm happy to >> volunteer to be a release manager for the next release. This will be >> the first release post-split, so there will undoubtedly be some issues >> to work out. I think the focus should be on getting an alpha release >> out, so I suggest we create a new 0.21 branch from trunk, then spend >> time fixing blockers (which will be a superset of the existing 0.21 >> blockers). >> >> Cheers, >> Tom > > My thoughts > > * The installed base creates its own inertia: if you have 2PB of data you > care about, you don't want to be bleeding edge. > * That installed base creates resistance to getting patches back in. I think > everyone -myself included -has stuff they want to get into the system, but > everyone who doesn't see the need for a feature is nervous. > * the branches reassure people of stability, but increase the cost of > changes and fixes too. There's more pressure to backport stuff, this makes > big reorgs hard. > * It makes takeup of new features (like the new FS apis) harder. You have to > consider how long 0.20.x will stay around, so focus on the stuff that's > there. > * I worry that the partioning of the project is making inertia worse too, > harder to co-ordinate changes across the code, and the code is still tightly > coupled enough that matters. > > My suggestons > +1 to the idea of stable/unstable, though I'd like to get some stuff into > 0.22, and with avro and security, that's going to be pretty traumatic too. > the move from 0.20 to 0.21 will be much less painful > +1 to Tom being release manager. > +1 to some session where everyone brings their patches up to date and we > push them into trunk, to bring the branches back in line. If you want to do > some session in the bay area then perhaps those of us outside it can skype > in or IPC, do a distributed triage run and really work hard to get stuff in > and working. We should do this before cutting the 0.21 branch, I will sort > my stuff out asap > > Now, what cluster time -real and virtual- to people have to offer Tom? I may > -repeat may- be able to sort out some OpenCirrus machines, or transient VMs > with limited per-VM storage in my little 1000 node datacentre, though > networking complexity gets in the way there. He won't have direct access to > it. > > -Steve > >> >> On Wed, Mar 24, 2010 at 1:27 PM, Brian Bockelman <bbock...@cse.unl.edu> >> wrote: >>> >>> Hey Allen, >>> >>> Your post provoked a few thoughts: >>> 1) Hadoop is a large, but relatively immature project (as in, there's >>> still a lot of major features coming down the pipe). If we wait to release >>> on features, especially when there are critical bugs, we end up with a large >>> number of patches between releases. This ends up encouraging custom patch >>> sets and custom distributions. >>> 2) The barrier for patch acceptance is high, especially for opportunistic >>> developers. This is a good thing for code quality, but for getting patches >>> in a timely manner. This means that there are a lot of 'mostly good' >>> patches out there in JIRA which have not landed. This again encourages >>> folks to develop their own custom patch sets. >>> 3) We make only bugfixes for past minor releases, meaning the stable >>> Apache release is perpetually behind in features, even features that are not >>> core. >>> >>> Not sure how to best fix these things. One possibility: >>> a) Have a stable/unstable series (0.19.x is unstable, 0.20.x is stable, >>> 0.21.x is unstable). For the unstable releases, lower the bar for code >>> acceptance for less-risky patches. >>> b) Combined with a a time-based release for bugfixes (and non-dangerous >>> features?) in order to keep the feature releases "fresh". >>> >>> (a) aims to tackle problems (1) and (2). (b) aims to tackle (3). >>> >>> This might not work for everything. If I had a goal, it would be to >>> decrease the number of active distributions from 3 to 2 - otherwise you end >>> up spending far too much time consensus building. >>> >>> Just a thought from an outside, relatively content observer, >>> >>> Brian >>> >>> On Mar 24, 2010, at 1:38 PM, Allen Wittenauer wrote: >>> >>>> On 3/15/10 9:06 AM, "Owen O'Malley" <o...@yahoo-inc.com> wrote: >>>>> >>>>> From our 21 experience, it looks like our old release strategy is >>>>> failing. >>>> >>>> Maybe this is a dumb question but... Are we sure it isn't the >>>> community >>>> failing? >>>> >>>> From where I stand, the major committers (PMC?) have essentially >>>> forked >>>> Hadoop into three competing source trees. No one appears to be >>>> dedicated to >>>> helping the community release because the focus is on their own tree. >>>> Worse >>>> yet, two of these trees are publicly available with both sides pushing >>>> their >>>> own tree as vastly superior (against each other and against the official >>>> Apache branded one). >>>> >>>> What are the next steps in getting this resolved? Is >>>> Hadoop-as-we-know-it essentially dead? What is going to prevent the >>>> fiasco >>>> that is 0.21 from impacting 0.22? >>>> >>>> For me personally, I'm more amused than upset that 0.21 hasn't been >>>> released. But I'm less happy that there appears to be a focus on >>>> feature >>>> additions rather than getting some of the 0.21 blockers settled (I'm >>>> assuming here that most of the 0.21 blockers apply to 0.22 as well). >>>> >>>> I don't think retroactively declaring 0.20 as 1.0 is going to make the >>>> situation any better. [In fact, I believe it will make it worse, since >>>> it >>>> gives an external impression that 0.20 is somehow stable at all levels. >>>> We >>>> all know this isn't true.] >>> > >
