Michael,
Its in HBASE-7912
This is tip of git log:
commit a072f6f49a26a7259ff2aaef6cb56d85eb592482
Author: Frank Welsch <fwel...@jps.net>
Date: Fri Sep 23 18:00:42 2016 -0400
HBASE-16574 Book updates for backup and restore
commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e
Author: tedyu <yuzhih...@gmail.com>
Date: Wed Oct 5 16:29:40 2016 -0700
HBASE-16727 Backup refactoring: remove MR dependencies from HMaster
(Vladimir Rodionov)
-Vlad
On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote:
> Which branch do I check out to try it? HBASE-7912 is not it. I don't see an
> HBASE-16727...
> Thanks,
> M
>
>
> On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov <
> vladrodio...@gmail.com>
> wrote:
>
> > The last patch is on review board:
> > https://reviews.apache.org/r/52748
> >
> > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov <
> vladrodio...@gmail.com
> > >
> > wrote:
> >
> > > >> How hard to put in an hbase-backup module? hbase-server is fat
> enough
> > > >> already. Could be done as a follow-up.
> > >
> > > https://issues.apache.org/jira/browse/HBASE-16727?
> > > focusedCommentId=15531237&page=com.atlassian.jira.
> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237
> > >
> > > Can we do merge first? Then we can discuss separate module.
> > >
> > > -Vlad
> > >
> > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > >
> > >> Looks like the first quote was cut off.
> > >> The original sentence was:
> > >>
> > >> bq. no mapreduce job launched from master or region server.
> > >>
> > >> mapreduce job is launched from the node where command line tool is
> run.
> > >>
> > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote:
> > >>
> > >> > bq. launched from master or region server.
> > >> >
> > >> > What does this mean please? Has to be run from Master or
> RegionServer?
> > >> Can
> > >> > it be run from another node altogether?
> > >> >
> > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov <
> > >> vladrodio...@gmail.com
> > >> > >
> > >> > wrote:
> > >> >
> > >> > > >> mapreduce dependency has been moved to client side - no
> mapreduce
> > >> job
> > >> > >
> > >> > > 1. We have no code in the client module anymore, due to dependency
> > on
> > >> > > internal server API (HFile and WAL access).
> > >> > > 2. Backup/ restore are client - driven operations, but all the
> code
> > >> > resides
> > >> > > in the server module
> > >> > >
> > >> >
> > >> > How hard to put in an hbase-backup module? hbase-server is fat
> enough
> > >> > already. Could be done as a follow-up.
> > >> >
> > >> > Thanks,
> > >> > St.Ack
> > >> >
> > >> >
> > >> >
> > >> > > 3. No MR in Master, no procedure - driven execution.
> > >> > > 4. Old good MR from command-line.
> > >> > > 5. Security was simplified and now only super-user is allowed to
> run
> > >> > > backup/restores.
> > >> > > 6. HBase Backup API was gone due to 1. Now only command-line
> access
> > to
> > >> > > backup tools.
> > >> > >
> > >> > > These consequences of refactoring has been discussed in
> HBASE-16727.
> > >> > >
> > >> > > -Vlad
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yuzhih...@gmail.com>
> > wrote:
> > >> > >
> > >> > > > Reviving this thread.
> > >> > > >
> > >> > > > The following has taken place:
> > >> > > >
> > >> > > > mapreduce dependency has been moved to client side - no
> mapreduce
> > >> job
> > >> > > > launched from master or region server.
> > >> > > > document patch (HBASE-16574) has been integrated.
> > >> > > > Updated mega patch has been attached to HBASE-14123: this covers
> > the
> > >> > > > refactor in #1 above and the protobuf 3 merge.
> > >> > > >
> > >> > > > If community has more feedback on the merge proposal, I would
> love
> > >> to
> > >> > > hear
> > >> > > > it.
> > >> > > >
> > >> > > > Thanks
> > >> > > >
> > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey <
> > bus...@cloudera.com>
> > >> > > wrote:
> > >> > > >
> > >> > > > > I'd like to see the docs proposed on HBASE-16574 integrated
> into
> > >> our
> > >> > > > > project's documentation prior to merge.
> > >> > > > >
> > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yuzhih...@gmail.com>
> > >> wrote:
> > >> > > > > > This feature can be marked experimental due to some
> > limitations
> > >> > such
> > >> > > as
> > >> > > > > > security.
> > >> > > > > >
> > >> > > > > > Your previous round of comments have been addressed.
> > >> > > > > > Command line tool has gone through:
> > >> > > > > >
> > >> > > > > > HBASE-16620 Fix backup command-line tool usability issues
> > >> > > > > > HBASE-16655 hbase backup describe with incorrect backup id
> > >> results
> > >> > in
> > >> > > > NPE
> > >> > > > > >
> > >> > > > > > The updated doc has been attached to HBASE-16574.
> > >> > > > > >
> > >> > > > > > Cheers
> > >> > > > > >
> > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net>
> > >> wrote:
> > >> > > > > >
> > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu <
> yuzhih...@gmail.com
> > >
> > >> > > wrote:
> > >> > > > > >>
> > >> > > > > >> > Are there more (review) comments ?
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> Are outstanding comments addressed?
> > >> > > > > >>
> > >> > > > > >> I don't see answer to my 'is this experimental/will it be
> > >> marked
> > >> > > > > >> experimental' question.
> > >> > > > > >>
> > >> > > > > >> I ran into some issues trying to use the feature and
> > suggested
> > >> > that
> > >> > > a
> > >> > > > > >> feature likes this needs polish else it'll just rot,
> unused.
> > >> Has
> > >> > > > polish
> > >> > > > > >> been applied? All ready for another 'user' test? Suggest
> that
> > >> you
> > >> > > > update
> > >> > > > > >> here going forward for the benefit of those trying to
> follow
> > >> along
> > >> > > and
> > >> > > > > who
> > >> > > > > >> are not watching JIRA change fly-by.
> > >> > > > > >>
> > >> > > > > >> It looks like doc got a revision -- I have to check -- to
> > take
> > >> on
> > >> > > > > >> suggestion made above but again, suggest, that this thread
> > gets
> > >> > > > updated.
> > >> > > > > >>
> > >> > > > > >> Thanks,
> > >> > > > > >> St.Ack
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > > >> > Thanks
> > >> > > > > >> >
> > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das <
> > >> > > d...@hortonworks.com
> > >> > > > >
> > >> > > > > >> > wrote:
> > >> > > > > >> >
> > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima,
> and
> > >> > others
> > >> > > > for
> > >> > > > > the
> > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for
> > >> taking
> > >> > > care
> > >> > > > of
> > >> > > > > >> the
> > >> > > > > >> > > feedback. Are we all good to do the merge now? Rather
> do
> > >> > sooner
> > >> > > > than
> > >> > > > > >> > later.
> > >> > > > > >> > > ________________________________________
> > >> > > > > >> > > From: saint....@gmail.com <saint....@gmail.com> on
> > behalf
> > >> of
> > >> > > > Stack
> > >> > > > > <
> > >> > > > > >> > > st...@duboce.net>
> > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM
> > >> > > > > >> > > To: HBase Dev List
> > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore -
> Branch
> > >> > > > HBASE-7912
> > >> > > > > >> > >
> > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu <
> > >> yuzhih...@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > > >> > >
> > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123.
> > >> > > > > >> > > >
> > >> > > > > >> > > > Please comment on HBASE-14123 on how you want to
> > review.
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago.
> > >> Suggest
> > >> > > > > updating
> > >> > > > > >> > it.
> > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so
> > should
> > >> be
> > >> > > > fine.
> > >> > > > > >> > >
> > >> > > > > >> > > St.Ack
> > >> > > > > >> > >
> > >> > > > > >> > >
> > >> > > > > >> > > >
> > >> > > > > >> > > > Thanks
> > >> > > > > >> > > >
> > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack <
> > >> st...@duboce.net>
> > >> > > > wrote:
> > >> > > > > >> > > >
> > >> > > > > >> > > > > On review of the 'patch', do I just compare the
> > branch
> > >> to
> > >> > > > > master or
> > >> > > > > >> > is
> > >> > > > > >> > > > > there a megapatch posted somewhere (I think I saw
> one
> > >> but
> > >> > it
> > >> > > > > seemed
> > >> > > > > >> > > stale
> > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb
> question.
> > >> > > > > >> > > > > St.Ack
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack <
> > >> st...@duboce.net
> > >> > >
> > >> > > > > wrote:
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > > Late to the game. A few comments after rereading
> > this
> > >> > > thread
> > >> > > > > as a
> > >> > > > > >> > > > 'user'.
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > + Before merge, a user-facing feature like this
> > >> should
> > >> > > work
> > >> > > > > (If
> > >> > > > > >> > this
> > >> > > > > >> > > is
> > >> > > > > >> > > > > "higher-bar
> > >> > > > > >> > > > > > for new features", bring it on -- smile).
> > >> > > > > >> > > > > > + As a user, I tried the branch with tools after
> > >> > reviewing
> > >> > > > the
> > >> > > > > >> > > > > just-posted
> > >> > > > > >> > > > > > doc. I had an 'interesting' experience (left
> > >> comments up
> > >> > > on
> > >> > > > > >> > issue). I
> > >> > > > > >> > > > > think
> > >> > > > > >> > > > > > the tooling/doc. important to get right. If it
> > breaks
> > >> > > easily
> > >> > > > > or
> > >> > > > > >> is
> > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will
> > >> judge
> > >> > the
> > >> > > > > whole
> > >> > > > > >> > > > > > backup/restore tooling chain as not trustworthy
> and
> > >> > > abandon
> > >> > > > > it.
> > >> > > > > >> > Lets
> > >> > > > > >> > > > not
> > >> > > > > >> > > > > > have this happen to this feature.
> > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter
> list)
> > >> that
> > >> > > > there
> > >> > > > > >> > needs
> > >> > > > > >> > > to
> > >> > > > > >> > > > > be
> > >> > > > > >> > > > > > explicit qualification on what is actually being
> > >> > delivered
> > >> > > > --
> > >> > > > > >> > > > including a
> > >> > > > > >> > > > > > listing of limitations (some look serious such as
> > >> data
> > >> > > bleed
> > >> > > > > from
> > >> > > > > >> > > other
> > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for my
> use
> > >> > > case...)
> > >> > > > --
> > >> > > > > >> > needs
> > >> > > > > >> > > to
> > >> > > > > >> > > > > > accompany the merge. Lets fold them into the user
> > >> doc.
> > >> > in
> > >> > > > the
> > >> > > > > >> > > technical
> > >> > > > > >> > > > > > overview area as suggested so user expectations
> are
> > >> > > properly
> > >> > > > > >> > managed
> > >> > > > > >> > > > > > (otherwise, they expect the world and will just
> > give
> > >> up
> > >> > > when
> > >> > > > > we
> > >> > > > > >> > fall
> > >> > > > > >> > > > > > short). Vladimir did a list of what is in each of
> > the
> > >> > > phases
> > >> > > > > >> above
> > >> > > > > >> > > > which
> > >> > > > > >> > > > > > would serve as a good start.
> > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks
> > above).
> > >> > I'd
> > >> > > > > prefer
> > >> > > > > >> it
> > >> > > > > >> > > is
> > >> > > > > >> > > > > > not. If it is, it should be labelled all over
> that
> > >> it is
> > >> > > > so. I
> > >> > > > > >> see
> > >> > > > > >> > > > > current
> > >> > > > > >> > > > > > state called out as a '... technical preview
> > >> feature'.
> > >> > > Does
> > >> > > > > this
> > >> > > > > >> > mean
> > >> > > > > >> > > > > > not-for-users?
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > St.Ack
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu <
> > >> > > > yuzhih...@gmail.com>
> > >> > > > > >> > wrote:
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >> Sean:
> > >> > > > > >> > > > > >> Do you have more comments ?
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> Cheers
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir
> Rodionov
> > <
> > >> > > > > >> > > > > vladrodio...@gmail.com
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> wrote:
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >> > Sean,
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > Backup/Restore can fail due to various
> reasons:
> > >> > network
> > >> > > > > outage
> > >> > > > > >> > > > > (cluster
> > >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS
> > layer,
> > >> M/R
> > >> > > > > failure
> > >> > > > > >> > due
> > >> > > > > >> > > to
> > >> > > > > >> > > > > >> "HDFS
> > >> > > > > >> > > > > >> > exceeded quota", user error (manual deletion
> of
> > >> data)
> > >> > > and
> > >> > > > > so
> > >> > > > > >> on
> > >> > > > > >> > so
> > >> > > > > >> > > > on.
> > >> > > > > >> > > > > >> That
> > >> > > > > >> > > > > >> > is impossible to enumerate all possible types
> of
> > >> > > failures
> > >> > > > > in a
> > >> > > > > >> > > > > >> distributed
> > >> > > > > >> > > > > >> > system - that is not our goal/task.
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > We focus completely on backup system table
> > >> > consistency
> > >> > > > in a
> > >> > > > > >> > > presence
> > >> > > > > >> > > > > of
> > >> > > > > >> > > > > >> any
> > >> > > > > >> > > > > >> > type of failure. That is what I call
> "tolerance
> > to
> > >> > > > > failures".
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > On a failure:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > BACKUP. All backup system information (prior
> to
> > >> > backup)
> > >> > > > > will
> > >> > > > > >> be
> > >> > > > > >> > > > > restored
> > >> > > > > >> > > > > >> > and all temporary data, related to a failed
> > >> session,
> > >> > in
> > >> > > > > HDFS
> > >> > > > > >> > will
> > >> > > > > >> > > be
> > >> > > > > >> > > > > >> > deleted
> > >> > > > > >> > > > > >> > RESTORE. We do not care about system data,
> > because
> > >> > > > restore
> > >> > > > > >> does
> > >> > > > > >> > > not
> > >> > > > > >> > > > > >> change
> > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up
> > and
> > >> > table
> > >> > > > > will
> > >> > > > > >> be
> > >> > > > > >> > > in a
> > >> > > > > >> > > > > >> state
> > >> > > > > >> > > > > >> > back to where it was before operation started.
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > This is what user should expect in case of a
> > >> failure.
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > -Vlad
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > -Vlad
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey <
> > >> > > > > >> bus...@apache.org
> > >> > > > > >> > >
> > >> > > > > >> > > > > wrote:
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs that
> > >> explain
> > >> > > the
> > >> > > > > >> > various
> > >> > > > > >> > > > > >> > > expected failures would be sufficient.
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir
> > >> Rodionov
> > >> > > > > >> > > > > >> > > <vladrodio...@gmail.com> wrote:
> > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today as
> a
> > >> > preview
> > >> > > > and
> > >> > > > > >> our
> > >> > > > > >> > > > writer
> > >> > > > > >> > > > > >> > Frank
> > >> > > > > >> > > > > >> > > > will be working on a putting it into
> Apache
> > >> > repo.
> > >> > > > > >> Timeline
> > >> > > > > >> > > > > depends
> > >> > > > > >> > > > > >> on
> > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it
> > >> rather
> > >> > > > sooner
> > >> > > > > >> than
> > >> > > > > >> > > > > later.
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > > As for failure testing, we are focusing
> only
> > >> on a
> > >> > > > > >> consistent
> > >> > > > > >> > > > state
> > >> > > > > >> > > > > >> of
> > >> > > > > >> > > > > >> > > > backup system data in a presence of any
> type
> > >> of
> > >> > > > > failures,
> > >> > > > > >> We
> > >> > > > > >> > > are
> > >> > > > > >> > > > > not
> > >> > > > > >> > > > > >> > > going
> > >> > > > > >> > > > > >> > > > to implement anything more "fancy", than
> > >> that.
> > >> > We
> > >> > > > > allow
> > >> > > > > >> > both:
> > >> > > > > >> > > > > >> backup
> > >> > > > > >> > > > > >> > and
> > >> > > > > >> > > > > >> > > > restore to fail. What we do not allow is
> to
> > >> have
> > >> > > > system
> > >> > > > > >> data
> > >> > > > > >> > > > > >> corrupted.
> > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any
> > other
> > >> > > > > concerns,
> > >> > > > > >> you
> > >> > > > > >> > > > want
> > >> > > > > >> > > > > >> us to
> > >> > > > > >> > > > > >> > > > address?
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > > -Vlad
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean
> > Busbey <
> > >> > > > > >> > > bus...@apache.org
> > >> > > > > >> > > > >
> > >> > > > > >> > > > > >> > wrote:
> > >> > > > > >> > > > > >> > > >
> > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not
> > >> address
> > >> > > my
> > >> > > > > >> concern
> > >> > > > > >> > > > > around
> > >> > > > > >> > > > > >> > docs
> > >> > > > > >> > > > > >> > > at
> > >> > > > > >> > > > > >> > > >> all, unless said docs have already made
> it
> > >> into
> > >> > > the
> > >> > > > > >> project
> > >> > > > > >> > > > > repo. I
> > >> > > > > >> > > > > >> > > don't
> > >> > > > > >> > > > > >> > > >> want third party resources for using a
> > major
> > >> and
> > >> > > > > >> important
> > >> > > > > >> > > > > feature
> > >> > > > > >> > > > > >> of
> > >> > > > > >> > > > > >> > > the
> > >> > > > > >> > > > > >> > > >> project, I want us to provide end users
> > with
> > >> > what
> > >> > > > they
> > >> > > > > >> need
> > >> > > > > >> > > to
> > >> > > > > >> > > > > get
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > > job
> > >> > > > > >> > > > > >> > > >> done.
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > > >> I see some calls for patience on the
> > failure
> > >> > > > testing,
> > >> > > > > but
> > >> > > > > >> > the
> > >> > > > > >> > > > > >> appeal
> > >> > > > > >> > > > > >> > to
> > >> > > > > >> > > > > >> > > us
> > >> > > > > >> > > > > >> > > >> having done a bad job of requiring proper
> > >> tests
> > >> > of
> > >> > > > > >> previous
> > >> > > > > >> > > > > >> features
> > >> > > > > >> > > > > >> > > just
> > >> > > > > >> > > > > >> > > >> makes me more concerned about not getting
> > >> them
> > >> > > > here. I
> > >> > > > > >> > don't
> > >> > > > > >> > > > want
> > >> > > > > >> > > > > >> to
> > >> > > > > >> > > > > >> > set
> > >> > > > > >> > > > > >> > > >> yet another bad example that will then be
> > >> > pointed
> > >> > > to
> > >> > > > > in
> > >> > > > > >> the
> > >> > > > > >> > > > > future.
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" <
> > >> > > yuzhih...@gmail.com
> > >> > > > >
> > >> > > > > >> > wrote:
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > > >> > Is there any concern which is not
> > >> addressed ?
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ?
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > Thanks
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew
> > >> > Purtell <
> > >> > > > > >> > > > > >> apurt...@apache.org
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> > > >> > wrote:
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > > Vlad,
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > I apologize for using the term
> > >> 'half-baked'
> > >> > > in a
> > >> > > > > way
> > >> > > > > >> > that
> > >> > > > > >> > > > > could
> > >> > > > > >> > > > > >> > > seem a
> > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant
> that
> > >> as a
> > >> > > > > general
> > >> > > > > >> > > > > >> hypothetical.
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM,
> > Vladimir
> > >> > > > Rodionov
> > >> > > > > <
> > >> > > > > >> > > > > >> > > >> > vladrodio...@gmail.com>
> > >> > > > > >> > > > > >> > > >> > > wrote:
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is
> > already
> > >> > lots
> > >> > > of
> > >> > > > > >> > > half-baked
> > >> > > > > >> > > > > >> code
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > the
> > >> > > > > >> > > > > >> > > >> > > > branch,
> > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding more?"
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > I meant - not production - ready
> yet.
> > >> This
> > >> > > is
> > >> > > > > 2.0
> > >> > > > > >> > > > > development
> > >> > > > > >> > > > > >> > > branch
> > >> > > > > >> > > > > >> > > >> > and,
> > >> > > > > >> > > > > >> > > >> > > > hence many features are in works,
> > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do not
> > >> > consider
> > >> > > > > backup
> > >> > > > > >> > as
> > >> > > > > >> > > > half
> > >> > > > > >> > > > > >> > baked
> > >> > > > > >> > > > > >> > > >> > > feature -
> > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and
> has
> > >> very
> > >> > > > good
> > >> > > > > >> doc,
> > >> > > > > >> > > > which
> > >> > > > > >> > > > > we
> > >> > > > > >> > > > > >> > will
> > >> > > > > >> > > > > >> > > >> > > provide
> > >> > > > > >> > > > > >> > > >> > > > to Apache shortly.
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > -Vlad
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM,
> > Andrew
> > >> > > > Purtell <
> > >> > > > > >> > > > > >> > > apurt...@apache.org>
> > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked
> > changes
> > >> > that
> > >> > > > > won't
> > >> > > > > >> be
> > >> > > > > >> > > > > >> finished.
> > >> > > > > >> > > > > >> > > >> However
> > >> > > > > >> > > > > >> > > >> > > in
> > >> > > > > >> > > > > >> > > >> > > > > this case the crew working on
> this
> > >> > feature
> > >> > > > are
> > >> > > > > >> long
> > >> > > > > >> > > > > timers
> > >> > > > > >> > > > > >> and
> > >> > > > > >> > > > > >> > > less
> > >> > > > > >> > > > > >> > > >> > > > likely
> > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave
> > >> > something
> > >> > > > in a
> > >> > > > > >> half
> > >> > > > > >> > > > baked
> > >> > > > > >> > > > > >> > > state. Of
> > >> > > > > >> > > > > >> > > >> > > > course
> > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how
> anything
> > >> will
> > >> > > turn
> > >> > > > > out,
> > >> > > > > >> > > but I
> > >> > > > > >> > > > > am
> > >> > > > > >> > > > > >> > > willing
> > >> > > > > >> > > > > >> > > >> to
> > >> > > > > >> > > > > >> > > >> > > > take
> > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel
> > their
> > >> > best
> > >> > > > path
> > >> > > > > >> > > forward
> > >> > > > > >> > > > > now
> > >> > > > > >> > > > > >> is
> > >> > > > > >> > > > > >> > to
> > >> > > > > >> > > > > >> > > >> > merge
> > >> > > > > >> > > > > >> > > >> > > to
> > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had
> bandwidth
> > to
> > >> > have
> > >> > > > > done
> > >> > > > > >> > some
> > >> > > > > >> > > > real
> > >> > > > > >> > > > > >> > > kicking
> > >> > > > > >> > > > > >> > > >> of
> > >> > > > > >> > > > > >> > > >> > > the
> > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week.
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that time
> > for
> > >> > this
> > >> > > > > email
> > >> > > > > >> > :-)
> > >> > > > > >> > > > but
> > >> > > > > >> > > > > I
> > >> > > > > >> > > > > >> > type
> > >> > > > > >> > > > > >> > > >> > fast.)
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to
> agitate
> > >> for
> > >> > > > making
> > >> > > > > 2.0
> > >> > > > > >> > > more
> > >> > > > > >> > > > > real
> > >> > > > > >> > > > > >> > and
> > >> > > > > >> > > > > >> > > >> spend
> > >> > > > > >> > > > > >> > > >> > > > some
> > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding
> > down
> > >> > with
> > >> > > > > 0.98. I
> > >> > > > > >> > > think
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > > means
> > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now
> and
> > >> even
> > >> > > > > evicting
> > >> > > > > >> > > > things
> > >> > > > > >> > > > > >> from
> > >> > > > > >> > > > > >> > > 2.0
> > >> > > > > >> > > > > >> > > >> > > branch
> > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable,
> > >> leaving
> > >> > > them
> > >> > > > > only
> > >> > > > > >> > > once
> > >> > > > > >> > > > > >> again
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > the
> > >> > > > > >> > > > > >> > > >> > > > master
> > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting
> > them.
> > >> > > Let's
> > >> > > > > take
> > >> > > > > >> it
> > >> > > > > >> > > > case
> > >> > > > > >> > > > > by
> > >> > > > > >> > > > > >> > > case.
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > I think this feature can come in
> > >> > > relatively
> > >> > > > > >> safely.
> > >> > > > > >> > > As
> > >> > > > > >> > > > > >> added
> > >> > > > > >> > > > > >> > > >> > insurance,
> > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it
> > could
> > >> be
> > >> > > > > reverted
> > >> > > > > >> on
> > >> > > > > >> > > the
> > >> > > > > >> > > > > 2.0
> > >> > > > > >> > > > > >> > > branch
> > >> > > > > >> > > > > >> > > >> if
> > >> > > > > >> > > > > >> > > >> > > > folks
> > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 decide
> > to
> > >> > evict
> > >> > > > it
> > >> > > > > >> > because
> > >> > > > > >> > > > it
> > >> > > > > >> > > > > is
> > >> > > > > >> > > > > >> > > >> > unfinished
> > >> > > > > >> > > > > >> > > >> > > or
> > >> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly
> > can
> > >> > > > happen. I
> > >> > > > > >> > would
> > >> > > > > >> > > > > >> expect if
> > >> > > > > >> > > > > >> > > talk
> > >> > > > > >> > > > > >> > > >> > > like
> > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help
> > finishing
> > >> or
> > >> > > > > >> stabilizing
> > >> > > > > >> > > > > what's
> > >> > > > > >> > > > > >> > under
> > >> > > > > >> > > > > >> > > >> > > > discussion
> > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a
> revert.
> > >> > Either
> > >> > > > way
> > >> > > > > >> the
> > >> > > > > >> > > > > outcome
> > >> > > > > >> > > > > >> is
> > >> > > > > >> > > > > >> > > >> > > acceptable.
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM,
> > Dima
> > >> > > Spivak
> > >> > > > <
> > >> > > > > >> > > > > >> > > dimaspi...@apache.org
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is
> > already
> > >> > lots
> > >> > > > of
> > >> > > > > >> > > > half-baked
> > >> > > > > >> > > > > >> code
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > >> the
> > >> > > > > >> > > > > >> > > >> > > > > branch,
> > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding
> > more?"
> > >> > is a
> > >> > > > > good
> > >> > > > > >> > code
> > >> > > > > >> > > > > commit
> > >> > > > > >> > > > > >> > > >> > philosophy
> > >> > > > > >> > > > > >> > > >> > > > for
> > >> > > > > >> > > > > >> > > >> > > > > a
> > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed data
> > >> store.
> > >> > > ;)
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test
> > >> > coverage
> > >> > > > for
> > >> > > > > >> > > existing
> > >> > > > > >> > > > > >> > features
> > >> > > > > >> > > > > >> > > >> > > shouldn't
> > >> > > > > >> > > > > >> > > >> > > > > be
> > >> > > > > >> > > > > >> > > >> > > > > > used as justification for
> > >> introducing
> > >> > > new
> > >> > > > > >> > features
> > >> > > > > >> > > > with
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > > same
> > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's
> > the
> > >> end
> > >> > > > user
> > >> > > > > who
> > >> > > > > >> > > will
> > >> > > > > >> > > > > feel
> > >> > > > > >> > > > > >> > the
> > >> > > > > >> > > > > >> > > >> pain,
> > >> > > > > >> > > > > >> > > >> > > so
> > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we
> can
> > >> to
> > >> > > > > mitigate
> > >> > > > > >> > that?
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > -Dima
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 AM,
> > >> > Vladimir
> > >> > > > > >> > Rodionov <
> > >> > > > > >> > > > > >> > > >> > > > > vladrodio...@gmail.com>
> > >> > > > > >> > > > > >> > > >> > > > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Sean,
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * have docs
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and
> backup
> > >> is
> > >> > the
> > >> > > > > most
> > >> > > > > >> > > > > documented
> > >> > > > > >> > > > > >> > > feature
> > >> > > > > >> > > > > >> > > >> > :),
> > >> > > > > >> > > > > >> > > >> > > we
> > >> > > > > >> > > > > >> > > >> > > > > > will
> > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to Apache.
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness
> > >> tests
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Feature has close to 60 test
> > >> cases,
> > >> > > > which
> > >> > > > > >> run
> > >> > > > > >> > > for
> > >> > > > > >> > > > > >> approx
> > >> > > > > >> > > > > >> > 30
> > >> > > > > >> > > > > >> > > >> min.
> > >> > > > > >> > > > > >> > > >> > > We
> > >> > > > > >> > > > > >> > > >> > > > > can
> > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do not
> > >> mind
> > >> > :)
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * have
> > >> > correctness-in-face-of-failure
> > >> > > > > tests
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests
> in
> > >> > > existing
> > >> > > > > >> > features?
> > >> > > > > >> > > > In
> > >> > > > > >> > > > > >> > works,
> > >> > > > > >> > > > > >> > > we
> > >> > > > > >> > > > > >> > > >> > > have a
> > >> > > > > >> > > > > >> > > >> > > > > > clear
> > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what should
> be
> > >> done
> > >> > > by
> > >> > > > > the
> > >> > > > > >> > time
> > >> > > > > >> > > of
> > >> > > > > >> > > > > 2.0
> > >> > > > > >> > > > > >> > > >> release.
> > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for
> us,
> > >> to
> > >> > > > verify
> > >> > > > > IT
> > >> > > > > >> > > monkey
> > >> > > > > >> > > > > for
> > >> > > > > >> > > > > >> > > >> existing
> > >> > > > > >> > > > > >> > > >> > > > code.
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things
> outside
> > of
> > >> > > HBase
> > >> > > > > for
> > >> > > > > >> > > normal
> > >> > > > > >> > > > > >> > operation
> > >> > > > > >> > > > > >> > > >> > (okay
> > >> > > > > >> > > > > >> > > >> > > > for
> > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation)
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > We do not.
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent
> > >> already
> > >> > > on
> > >> > > > > the
> > >> > > > > >> > > > > development
> > >> > > > > >> > > > > >> > and
> > >> > > > > >> > > > > >> > > >> > testing
> > >> > > > > >> > > > > >> > > >> > > > the
> > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our
> > >> internal
> > >> > > > tests
> > >> > > > > and
> > >> > > > > >> > > many
> > >> > > > > >> > > > > >> rounds
> > >> > > > > >> > > > > >> > of
> > >> > > > > >> > > > > >> > > >> code
> > >> > > > > >> > > > > >> > > >> > > > > reviews
> > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do
> not
> > >> mind
> > >> > if
> > >> > > > > >> someone
> > >> > > > > >> > > from
> > >> > > > > >> > > > > >> HBase
> > >> > > > > >> > > > > >> > > >> > community
> > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review
> the
> > >> > code,
> > >> > > > but
> > >> > > > > it
> > >> > > > > >> > will
> > >> > > > > >> > > > > >> probably
> > >> > > > > >> > > > > >> > > >> takes
> > >> > > > > >> > > > > >> > > >> > > > > forever
> > >> > > > > >> > > > > >> > > >> > > > > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the
> > feature
> > >> is
> > >> > > > quite
> > >> > > > > >> large
> > >> > > > > >> > > > (1MB+
> > >> > > > > >> > > > > >> > > >> cumulative
> > >> > > > > >> > > > > >> > > >> > > > patch)
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half
> > baked
> > >> > > > features,
> > >> > > > > >> most
> > >> > > > > >> > > of
> > >> > > > > >> > > > > them
> > >> > > > > >> > > > > >> > are
> > >> > > > > >> > > > > >> > > in
> > >> > > > > >> > > > > >> > > >> > > active
> > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am
> not
> > >> > > > following
> > >> > > > > you
> > >> > > > > >> > > here,
> > >> > > > > >> > > > > >> Sean?
> > >> > > > > >> > > > > >> > > Why
> > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912
> > >> > > > > >> > > > > >> > > >> > > > > > is
> > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be
> > >> integrated
> > >> > > > into
> > >> > > > > 2.0
> > >> > > > > >> > > > branch?
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23
> AM,
> > >> Sean
> > >> > > > > Busbey <
> > >> > > > > >> > > > > >> > > bus...@apache.org
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >> > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at
> 10:36
> > >> PM,
> > >> > > Josh
> > >> > > > > >> Elser <
> > >> > > > > >> > > > > >> > > >> > > josh.el...@gmail.com>
> > >> > > > > >> > > > > >> > > >> > > > > > > wrote:
> > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's
> > >> > original
> > >> > > > > >> question
> > >> > > > > >> > is
> > >> > > > > >> > > > "as
> > >> > > > > >> > > > > >> > > robust as
> > >> > > > > >> > > > > >> > > >> > > > > snapshots
> > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"?
> > >> (independence of
> > >> > > > > >> > > backup/restore
> > >> > > > > >> > > > > >> > failure
> > >> > > > > >> > > > > >> > > >> > > tolerance
> > >> > > > > >> > > > > >> > > >> > > > > > from
> > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure
> tolerance)
> > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question
> WRT
> > >> > > context
> > >> > > > of
> > >> > > > > >> the
> > >> > > > > >> > > > > change,
> > >> > > > > >> > > > > >> or
> > >> > > > > >> > > > > >> > > is it
> > >> > > > > >> > > > > >> > > >> > > means
> > >> > > > > >> > > > > >> > > >> > > > > > for a
> > >> > > > > >> > > > > >> > > >> > > > > > > > veto
> > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just
> trying
> > >> to
> > >> > > make
> > >> > > > > sure
> > >> > > > > >> > I'm
> > >> > > > > >> > > > > >> following
> > >> > > > > >> > > > > >> > > >> along
> > >> > > > > >> > > > > >> > > >> > > > > > > adequately.
> > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0,
> bordering
> > >> on
> > >> > -1
> > >> > > > but
> > >> > > > > not
> > >> > > > > >> > for
> > >> > > > > >> > > > > >> reasons
> > >> > > > > >> > > > > >> > I
> > >> > > > > >> > > > > >> > > can
> > >> > > > > >> > > > > >> > > >> > > > > > articulate
> > >> > > > > >> > > > > >> > > >> > > > > > > > well.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move,
> > as a
> > >> > > > > community,
> > >> > > > > >> > > > towards
> > >> > > > > >> > > > > >> > > minimizing
> > >> > > > > >> > > > > >> > > >> > > risk
> > >> > > > > >> > > > > >> > > >> > > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by getting
> > >> > > "complete
> > >> > > > > >> enough
> > >> > > > > >> > > for
> > >> > > > > >> > > > > use"
> > >> > > > > >> > > > > >> > > gates
> > >> > > > > >> > > > > >> > > >> in
> > >> > > > > >> > > > > >> > > >> > > > place
> > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new
> > >> features.
> > >> > > This
> > >> > > > > was
> > >> > > > > >> > > > spurred
> > >> > > > > >> > > > > >> by a
> > >> > > > > >> > > > > >> > > some
> > >> > > > > >> > > > > >> > > >> > > > features
> > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and
> > never
> > >> > > making
> > >> > > > > it
> > >> > > > > >> to
> > >> > > > > >> > > "can
> > >> > > > > >> > > > > >> really
> > >> > > > > >> > > > > >> > > use"
> > >> > > > > >> > > > > >> > > >> > > > status
> > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of
> distributed
> > >> log
> > >> > > > replay
> > >> > > > > and
> > >> > > > > >> > the
> > >> > > > > >> > > > > >> zk-less
> > >> > > > > >> > > > > >> > > >> > > assignment
> > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if
> > there
> > >> was
> > >> > > > > more).
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally,
> > included
> > >> > > things
> > >> > > > > like:
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs
> > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day
> correctness
> > >> tests
> > >> > > > > >> > > > > >> > > >> > > > > > > > * have
> > >> > > correctness-in-face-of-failure
> > >> > > > > tests
> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things
> > outside
> > >> of
> > >> > > > HBase
> > >> > > > > for
> > >> > > > > >> > > > normal
> > >> > > > > >> > > > > >> > > operation
> > >> > > > > >> > > > > >> > > >> > > (okay
> > >> > > > > >> > > > > >> > > >> > > > > for
> > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation)
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the
> > MOB
> > >> > work
> > >> > > > off
> > >> > > > > in
> > >> > > > > >> a
> > >> > > > > >> > > > branch
> > >> > > > > >> > > > > >> and
> > >> > > > > >> > > > > >> > > out
> > >> > > > > >> > > > > >> > > >> of
> > >> > > > > >> > > > > >> > > >> > > > master
> > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these
> > >> > criteria.
> > >> > > > The
> > >> > > > > big
> > >> > > > > >> > > > > exemption
> > >> > > > > >> > > > > >> > > we've
> > >> > > > > >> > > > > >> > > >> had
> > >> > > > > >> > > > > >> > > >> > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark
> > >> > > integration,
> > >> > > > > where
> > >> > > > > >> > we
> > >> > > > > >> > > > all
> > >> > > > > >> > > > > >> > agreed
> > >> > > > > >> > > > > >> > > it
> > >> > > > > >> > > > > >> > > >> > > could
> > >> > > > > >> > > > > >> > > >> > > > > > > > land in master because it
> was
> > >> very
> > >> > > > well
> > >> > > > > >> > > isolated
> > >> > > > > >> > > > > (the
> > >> > > > > >> > > > > >> > > slide
> > >> > > > > >> > > > > >> > > >> > away
> > >> > > > > >> > > > > >> > > >> > > > from
> > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a
> > first-class
> > >> > part
> > >> > > > of
> > >> > > > > >> > > building
> > >> > > > > >> > > > up
> > >> > > > > >> > > > > >> that
> > >> > > > > >> > > > > >> > > >> > > > integration
> > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the
> > wisdom
> > >> of
> > >> > > this
> > >> > > > > >> > > decision).
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating
> > >> inclusion
> > >> > > in
> > >> > > > a
> > >> > > > > >> > > "probably
> > >> > > > > >> > > > > >> will
> > >> > > > > >> > > > > >> > be
> > >> > > > > >> > > > > >> > > >> > > released
> > >> > > > > >> > > > > >> > > >> > > > to
> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a
> > >> higher
> > >> > > bar,
> > >> > > > > >> > requiring
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact
> > >> > > performance
> > >> > > > > when
> > >> > > > > >> > the
> > >> > > > > >> > > > > >> feature
> > >> > > > > >> > > > > >> > > isn't
> > >> > > > > >> > > > > >> > > >> in
> > >> > > > > >> > > > > >> > > >> > > use
> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact
> > >> > performance
> > >> > > > when
> > >> > > > > >> the
> > >> > > > > >> > > > > feature
> > >> > > > > >> > > > > >> is
> > >> > > > > >> > > > > >> > in
> > >> > > > > >> > > > > >> > > >> use
> > >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or
> > show
> > >> > > enough
> > >> > > > > >> demand
> > >> > > > > >> > to
> > >> > > > > >> > > > > >> believe
> > >> > > > > >> > > > > >> > a
> > >> > > > > >> > > > > >> > > >> > > > non-trivial
> > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn
> the
> > >> > > feature
> > >> > > > on
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and
> > >> > > hbase-spark
> > >> > > > > >> > > > integration
> > >> > > > > >> > > > > >> out
> > >> > > > > >> > > > > >> > of
> > >> > > > > >> > > > > >> > > >> > > branch-1,
> > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've
> > >> "gotten
> > >> > > more
> > >> > > > > >> stable"
> > >> > > > > >> > > in
> > >> > > > > >> > > > > >> master
> > >> > > > > >> > > > > >> > > from
> > >> > > > > >> > > > > >> > > >> > the
> > >> > > > > >> > > > > >> > > >> > > > odd
> > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0
> > >> release
> > >> > > > > before
> > >> > > > > >> the
> > >> > > > > >> > > end
> > >> > > > > >> > > > > of
> > >> > > > > >> > > > > >> the
> > >> > > > > >> > > > > >> > > >> year?
> > >> > > > > >> > > > > >> > > >> > > > We're
> > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years
> since
> > >> the
> > >> > > > > release of
> > >> > > > > >> > > > version
> > >> > > > > >> > > > > >> 1.0;
> > >> > > > > >> > > > > >> > > >> seems
> > >> > > > > >> > > > > >> > > >> > > like
> > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I
> > >> haven't
> > >> > > seen
> > >> > > > > any
> > >> > > > > >> > > > concrete
> > >> > > > > >> > > > > >> > plans
> > >> > > > > >> > > > > >> > > >> this
> > >> > > > > >> > > > > >> > > >> > > > year.
> > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to
> > have
> > >> one
> > >> > > by
> > >> > > > > the
> > >> > > > > >> end
> > >> > > > > >> > > of
> > >> > > > > >> > > > > the
> > >> > > > > >> > > > > >> > > year, it
> > >> > > > > >> > > > > >> > > >> > > > seems a
> > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be
> adding
> > in
> > >> > > > > "features
> > >> > > > > >> > that
> > >> > > > > >> > > > need
> > >> > > > > >> > > > > >> > > maturing"
> > >> > > > > >> > > > > >> > > >> > on
> > >> > > > > >> > > > > >> > > >> > > > the
> > >> > > > > >> > > > > >> > > >> > > > > > > > branch.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete plan
> > for
> > >> > 2.0
> > >> > > > > keeps
> > >> > > > > >> me
> > >> > > > > >> > > from
> > >> > > > > >> > > > > >> > > considering
> > >> > > > > >> > > > > >> > > >> > > these
> > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the
> moment.
> > >> But
> > >> > I
> > >> > > > know
> > >> > > > > >> > first
> > >> > > > > >> > > > hand
> > >> > > > > >> > > > > >> how
> > >> > > > > >> > > > > >> > > much
> > >> > > > > >> > > > > >> > > >> > > > trouble
> > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other
> > >> features
> > >> > > > that
> > >> > > > > >> have
> > >> > > > > >> > > gone
> > >> > > > > >> > > > > >> into
> > >> > > > > >> > > > > >> > > >> > downstream
> > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without
> > >> robustness
> > >> > > > > checks
> > >> > > > > >> > (i.e.
> > >> > > > > >> > > > > >> > > replication),
> > >> > > > > >> > > > > >> > > >> > and
> > >> > > > > >> > > > > >> > > >> > > > I'm
> > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're
> > >> setting
> > >> > > up
> > >> > > > if
> > >> > > > > >> 2.0
> > >> > > > > >> > > goes
> > >> > > > > >> > > > > out
> > >> > > > > >> > > > > >> > with
> > >> > > > > >> > > > > >> > > >> this
> > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current
> state.
> > >> > > > > >> > > > > >> > > >> > > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > > >
> > >> > > > > >> > > > > >> > > >> > > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > --
> > >> > > > > >> > > > > >> > > >> > > > > Best regards,
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > - Andy
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove
> > their
> > >> > > worth
> > >> > > > by
> > >> > > > > >> > > hitting
> > >> > > > > >> > > > > >> back. -
> > >> > > > > >> > > > > >> > > Piet
> > >> > > > > >> > > > > >> > > >> > > Hein
> > >> > > > > >> > > > > >> > > >> > > > > (via Tom White)
> > >> > > > > >> > > > > >> > > >> > > > >
> > >> > > > > >> > > > > >> > > >> > > >
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > --
> > >> > > > > >> > > > > >> > > >> > > Best regards,
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > - Andy
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove their
> > >> worth
> > >> > by
> > >> > > > > >> hitting
> > >> > > > > >> > > > back.
> > >> > > > > >> > > > > -
> > >> > > > > >> > > > > >> > Piet
> > >> > > > > >> > > > > >> > > >> Hein
> > >> > > > > >> > > > > >> > > >> > > (via Tom White)
> > >> > > > > >> > > > > >> > > >> > >
> > >> > > > > >> > > > > >> > > >> >
> > >> > > > > >> > > > > >> > > >>
> > >> > > > > >> > > > > >> > >
> > >> > > > > >> > > > > >> >
> > >> > > > > >> > > > > >>
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > > >
> > >> > > > > >> > > > >
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > busbey
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
