On 1/5/10 11:29 AM, "Todd Lipcon" <t...@cloudera.com> wrote: > 1) Although we certainly do not guarantee wire compatibility between minor > versions (0.20 -> 0.21) have we previously implied wire compatibility > between bugfix releases?
IIRC, it has been implied and was a goal but not officially written anywhere public that I know of. > 2) Is the above something we *should* be guaranteeing already? A) From an ops perspective, the lack of compatibility between even minor releases is a pain. B) Most folks with even slightly complex environments are likely patching Hadoop. A good chunk of those are likely in ways that breaks compatibility. [For example, we're working on a TCP buffer patch for HDFS to fix what we suspect is a latency problem. Does it break compat? Maybe.] > 3) If we haven't guaranteed the above, how many users think we have? (ie how > do we correctly call out this fact in the 0.20.2 release notes in such a way > that no one gets surprised). I suspect most folks don't even know that micros are incompatible until they suddenly realize that distcp doesn't work. > I can imagine plenty of organizations where a > lockstep upgrade between client and server is difficult, and we should make > sure that cluster operators know it will be necessary. Since it wasn't > necessary between 0.20.0 and 0.20.1, or various 0.18 releases, people may > have grown used to non-lockstep upgrades. I can easily see many organizations only upgrading when it breaks due to the Hadoop binaries being spread far and wide and under the control of many different departments without any sort of centralized management. Despite the development model, I doubt few ever do just a mapred or hdfs upgrade, so any change in the stack will likely trigger a full Hadoop upgrade. > 4) If the above are problems, would it be worth considering a patch for > branch-20 that provides a client that is compatible with either, based on > the datanode protocol version number of the server? It seems like a bit of > scary complexity, but wanted to throw it out there. Everyone knows I don't mind playing devil's advocate :), so let me ask the obvious question: Bugs are bad, etc, etc, but is it so critical that it has to be in the 0.20 branch at all? I'd rather see the community spend cycles on 0.21 than worrying about 0.20 given that we're fast approaching the 1yr birthday of 0.20.0....
