So, given your stated concerns, Sean, the correctness-in-face-of-failure
tests is the only WIP thing remaining that you'd like to see, right? Are
there any other issues that you can think of now that would prevent you
from +1'ing?
Regarding the move to a "higher-bar for new features" by the community,
I'll stand back because I know that I have not been following closely
enough to comment here. If this is a community decision that everyone is
on board with, great. I honestly have not followed discussions closely
enough to understand whether this is a "we'd like to do this" or "we are
doing this".
This leads me to asking how can we address the concern on the amorphous
state of what "HBase-2.0" and how to avoid this blocking contributions
to 2.0. It seems like there's a chicken-and-egg problem with being
unable to determine the expected level of testing while there's no
concrete roadmap for 2.0. To me, it seems premature to raise the bar
higher in this case (because there is no firm date where continued work
must be completed by to avoid blocking the release), but, again, I type
this with great hesitance (as I have not be diligent with my inbox).
Vladimir Rodionov wrote:
Sean,
* have docs
Agree. We have a doc and backup is the most documented feature :), we will
release it shortly to Apache.
* have sunny-day correctness tests
Feature has close to 60 test cases, which run for approx 30 min. We can
add more, if community do not mind :)
* have correctness-in-face-of-failure tests
Any examples of these tests in existing features? In works, we have a clear
understanding of what should be done by the time of 2.0 release.
That is very close goal for us, to verify IT monkey for existing code.
* don't rely on things outside of HBase for normal operation (okay for
advanced operation)
We do not.
Enormous time has been spent already on the development and testing the
feature, it has passed our internal tests and many rounds of code reviews
by HBase committers. We do not mind if someone from HBase community
(outside of HW) will review the code, but it will probably takes forever to
wait for volunteer?, the feature is quite large (1MB+ cumulative patch)
2.0 branch is full of half baked features, most of them are in active
development, therefore I am not following you here, Sean? Why HBASE-7912 is
not good enough yet to be integrated into 2.0 branch?
-Vlad
On Wed, Sep 7, 2016 at 8:23 AM, Sean Busbey<bus...@apache.org> wrote:
On Tue, Sep 6, 2016 at 10:36 PM, Josh Elser<josh.el...@gmail.com> wrote:
So, the answer to Sean's original question is "as robust as snapshots
presently are"? (independence of backup/restore failure tolerance from
snapshot failure tolerance)
Is this just a question WRT context of the change, or is it means for a
veto
from you, Sean? Just trying to make sure I'm following along adequately.
I'd say ATM I'm -0, bordering on -1 but not for reasons I can articulate
well.
Here's an attempt.
We've been trying to move, as a community, towards minimizing risk to
downstream folks by getting "complete enough for use" gates in place
before we introduce new features. This was spurred by a some features
getting in half-baked and never making it to "can really use" status
(I'm thinking of distributed log replay and the zk-less assignment
stuff, I don't recall if there was more).
The gates, generally, included things like:
* have docs
* have sunny-day correctness tests
* have correctness-in-face-of-failure tests
* don't rely on things outside of HBase for normal operation (okay for
advanced operation)
As an example, we kept the MOB work off in a branch and out of master
until it could pass these criteria. The big exemption we've had to
this was the hbase-spark integration, where we all agreed it could
land in master because it was very well isolated (the slide away from
including docs as a first-class part of building up that integration
has led me to doubt the wisdom of this decision).
We've also been treating inclusion in a "probably will be released to
downstream" branches as a higher bar, requiring
* don't moderately impact performance when the feature isn't in use
* don't severely impact performance when the feature is in use
* either default-to-on or show enough demand to believe a non-trivial
number of folks will turn the feature on
The above has kept MOB and hbase-spark integration out of branch-1,
presumably while they've "gotten more stable" in master from the odd
vendor inclusion.
Are we going to have a 2.0 release before the end of the year? We're
coming up on 1.5 years since the release of version 1.0; seems like
it's about time, though I haven't seen any concrete plans this year.
Presuming we are going to have one by the end of the year, it seems a
bit close to still be adding in "features that need maturing" on the
branch.
The lack of a concrete plan for 2.0 keeps me from considering these
things blocker at the moment. But I know first hand how much trouble
folks have had with other features that have gone into downstream
facing releases without robustness checks (i.e. replication), and I'm
concerned about what we're setting up if 2.0 goes out with this
feature in its current state.
