If Hadoop refuses the changes before we release, we can change the default back.
On May 11, 2016, at 6:50 PM, Gary Helmling <[email protected]> wrote: >> >> >> I was trying to avoid the below oft-repeated pattern at least for the case >> of critical developments: >> >> + New feature arrives after much work by developer, reviewers and testers >> accompanied by fanfare (blog, talks). >> + Developers and reviewers move on after getting it committed or it gets >> hacked into a deploy so it works in a frankenstein form >> + It sits in our code base across one or more releases marked as optional, >> 'experimental' >> + The 'experimental' bleamish discourages its exercise by users >> + The feature lags, rots >> + Or, the odd time, we go ahead and enable it as default in spite of the >> fact it was never tried when experimental. >> >> Distributed Log Replay sat in hbase across a few major versions. Only when >> the threat of our making an actual release with it on by default did it get >> serious attention where it was found flawed and is now being actively >> purged. This was after it made it past reviews, multiple attempts at >> testing at scale, and so on; i.e. we'd done it all by the book. The time in >> an 'experimental' state added nothing. > Those are all valid concerns as well. It's certainly a pattern that we've > seen repeated. That's also a broader concern I have about the farther we > push out 2.0, then the less exercised master is. > > I don't really know how best to balance this with concerns about user > stability. Enabling by default in master would certainly be a forcing > function and would help it get more testing before release. I hear that > argument. But I'm worried about the impact after release, where something > as simple as a bug-fix point release upgrade of Hadoop could result in > runtime breakage of an HBase install. Will this happen in practice? I > don't know. It seems unlikely that the private variable names being used > for example would change in a point release. But we're violating the > abstraction that Hadoop provides us which guarantees such breakage won't > occur. > > >>> Yes. 2.0 is a bit out there so we have some time to iron out issues is >> the >> thought. Yes, it could push out delivery of 2.0. > Having this on by default in an unreleased master doesn't actually worry me > that much. It's just the question of what happens when we do release. At > that point, this discussion will be ancient history and I don't think we'll > give any renewed consideration to what the impact of this change might be. > Ideally it would be great to see this work in HDFS by that point and for > that HDFS version this becomes a non-issue. > > >> >> I think the discussion here has been helpful. Holes have been found (and >> plugged), the risk involved has gotten a good airing out here on dev, and >> in spite of the back and forth, one of our experts in good standing is >> still against it being on by default. >> >> If you are not down w/ the arguments, I'd be fine not making it the >> default. >> St.Ack > > I don't think it's right to block this by myself, since I'm clearly in the > minority. Since others clearly support this change, have at it. > > But let me pose an alternate question: what if HDFS flat out refuses to > adopt this change? What are our options then with this already shipping as > a default? Would we continue to endure breakage due to the use of HDFS > private internals? Do we switch the default back? Do we do something else? > > Thanks for the discussion.
