Just so that everyone is on the same page w.r.t the compatibility between 20.2 & 20.203 (don't think this is documented anywhere yet)..
The aim of the team working on Hadoop Security at Yahoo! was to make Hadoop *secure*, and with *minimal* disruption to existing apps. I can't think of any major user-facing incompatibilities other than the users having to run a 'kinit' when they are working with a secure Hadoop cluster (of course the admins need to do more work in order to set up a secure cluster). Also, security can switched off, and all the other enhancements (job limits, etc.) are still available.. As per users/Operations/Solutions at Yahoo!, 20.security was one of the smoothest upgrades ever. On 5/10/11 2:28 PM, "Todd Lipcon" <t...@cloudera.com> wrote: On Tue, May 10, 2011 at 12:41 PM, Scott Carey <sc...@richrelevance.com>wrote: > > As an observer, this is a very important observation. Sure, the default > is that dot releases are bugfix-onl. But exceptions to these rules are > sometimes required and often beneficial to the health of the project. > Performance enhancements, minor features, and other items are sometimes > very low risk and the barrier to getting them to users earlier should be > lower. > I agree whole-heartedly. > These issues are the sort of things that get into non-Apache releases > quickly and drive the community away from the Apache release. Its been > well proven through those vehicles that back-porting minor features and > improvements from trunk to an old release can be done safely. However, one shouldn't understate the difficulty of agreeing on the risk-reward tradeoff here. While risk is mostly technical, reward may vary widely based on the userbase or organization. For example, everyone would agree that security was a very risky feature to add to 20, with known backward compatibilities and a lot of fallout. For some people (both CDH and YDH), the security features were an absolute necessity on a tight timeline, so the risk-reward decision was clear -- I've heard from many users, though, that they saw none of the reward from security and wished they hadn't had to endure the resulting changes and bugs within the 0.20 series. Another example is the 0.20-append patch series, which is indispensable for the HBase community but seen as overly risky by those who do not use HBase. So, while I'm in favor of "sustaining" release series like 0.20-security in theory, I also think we need a clear inclusion criteria for such branches. As I said in a previous email, the criteria used to be "low risk compatible bug fixes only" with a vote process for any exceptions. 0.20-security is obviously entirely different, but as yet remains undefined (it's way more than just "security"). -Todd -- Todd Lipcon Software Engineer, Cloudera
