Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

2017-03-16 Thread Stack 
On Thu, Mar 16, 2017 at 10:01 AM, Vladimir Rodionov 
wrote:

> v61 is against current master.
>
>
It was what I thought. I got confused by the hijacker.
St.Ack



> On Thu, Mar 16, 2017 at 7:18 AM, Stack  wrote:
>
> > On Wed, Mar 15, 2017 at 11:38 PM, Vladimir Rodionov <
> > vladrodio...@gmail.com>
> > wrote:
> >
> > > >> I thought I was reviewing a patch that was against master
> > >
> > > Yes, it was against master. We used to develop in HBASE-7912 branch,
> but
> > it
> > > was abandoned 6-8 mo ago.
> > > This is why the only way to merge backup is to apply v61 directly to
> > master
> > > branch.
> > >
> > >
> > If you are saying that v61 is against a master of 6-8 months ago, then
> I'd
> > think this VOTE is premature. Don't you have a mountain of rebasing to do
> > before you can put your patch up for a vote?
> >
> > St.Ack
> >
> >
> >
> > > -Vlad
> > >
> > >
> > > On Wed, Mar 15, 2017 at 9:17 PM, Stack  wrote:
> > >
> > > > On Wed, Mar 15, 2017 at 5:51 PM, Ted Yu  wrote:
> > > >
> > > > > Thanks for the vote, Josh.
> > > > >
> > > > > So far there have been 3 +1's (Enis', mine and Josh's)
> > > > >
> > > > > Should we start another thread on the procedure of merging or use
> > this
> > > > > thread ?
> > > > > Background:
> > > > > HBASE-7912 branch is way out of date.
> > > > > Latest rounds of mega patch were based on then-current master
> branch.
> > > > >
> > > > >
> > > > 1. Vlad is running this VOTE, not you.
> > > > 2. Don't you need 3 PMC votes to merge? I count 2.
> > > > 3. You almost derailed the merge with your earlier interjection
> rushing
> > > to
> > > > take us in the wrong direction; you'd think you'd learn from your
> past
> > > > experience but here you are again w/ haste and rash summary.
> > > > 4. This vote has been awkward. Josh's note is considered and careful.
> > > You'd
> > > > think it could hang out there a while to see if draws a response
> rather
> > > > than rush to close.
> > > >
> > > > I thought I was reviewing a patch that was against master, not some
> > > version
> > > > of master from eons ago. Thats a problem. When was the last rebase?
> > > >
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Thanks
> > > > >
> > > > > On Wed, Mar 15, 2017 at 4:22 PM, Josh Elser 
> > wrote:
> > > > >
> > > > > > Spent the day playing around with this on 6 nodes.
> > > > > >
> > > > > > I've found some rough edges: some known (the docs blocker Vlad
> > > pointed
> > > > > > out) and some that I think are unknown (tooling is rough --
> > partially
> > > > > from
> > > > > > wrong help messages and, I think, changes in design like
> > > > Master-submitted
> > > > > > MR jobs).
> > > > > >
> > > > > > But, Stack's assessment (and Andrew's reminder) that further
> > tweaking
> > > > > > would just throw us back into another review cycle is a real
> > concern.
> > > > > It's
> > > > > > unfortunate that this feature has lingered so long aside of
> master
> > to
> > > > get
> > > > > > to this point, but I don't see any realistic resolution to this
> > > problem
> > > > > > than a merge. In the future, this is something we'll have to try
> > > harder
> > > > > to
> > > > > > avoid letting happen (looking in the mirror with quota work...)
> > > > > >
> > > > > > Thankfully, I can help out with development/review on the
> > outstanding
> > > > > > blockers (notably, the two I pruned from Vlad's original of five
> --
> > > the
> > > > > > other three still seem to be improvements). In addition to these
> > > > > blockers,
> > > > > > I believe the documentation *must* be updated before a release to
> > > note
> > > > > that
> > > > > > this feature is still growing -- it does not feel like a quality
> > > > feature
> > > > > > that I've come to expect from this community. This isn't a knock
> on
> > > > Vlad
> > > > > > and company; this is a hard problem and one that I could not have
> > > done
> > > > > > better given time constraints, but it is also one that users will
> > > > demand
> > > > > > simplicity and the utmost correctness around. To this end, I will
> > > also
> > > > > try
> > > > > > to help out to smooth out these issues in the following 2.x
> > release.
> > > > > >
> > > > > > So, this leaves me to say: +1 to merge with the caveat that the
> > docs
> > > > are
> > > > > > updated to make sure that any known, user-pitfalls are clearly
> > > > > documented.
> > > > > > This vote also comes with as real of a promise that I can make to
> > > help
> > > > > > avoid any issues with this feature that would prevent a 2.0
> > release.
> > > > > >
> > > > > > Thanks to all the giants whose shoulders I'm standing on to be
> able
> > > to
> > > > > > make this vote.
> > > > > >
> > > > > > Andrew Purtell wrote:
> > > > > >
> > > > > >> Great, and I changed my vote to -0 because Stack made a good
> > > argument
> > > > > that
> > > > > >> making more changes would

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread Yu Li 
+1 on allow adding methods to interfaces in minor release, especially
before 2.0 is out, I don't think a minor release is that "minor" (smile).

Regarding refguide, shall we also remind user to check the "incompatible"
changes in release note before upgrading for minor version?

Best Regards,
Yu

On 17 March 2017 at 08:50, Andrew Purtell  wrote:

> This sounds like a good result to me.
> We've sometimes provided BaseXXX implementations of interfaces for Java <=
> 7 but haven't been consistent. Would be good to do better.
>
> On Thu, Mar 16, 2017 at 2:18 PM, Stack  wrote:
>
> > On Thu, Mar 16, 2017 at 11:02 AM, Josh Elser  wrote:
> >
> > > +1 to the JDK8 default implementation approach. I'd say this would be
> > good
> > > to push forward for the future.
> > >
> > > For 1.x, I see no reason why we couldn't provide concrete
> implementations
> > > as a stop-gap. Identifying and creating those classes is probably the
> > > biggest barrier :). I don't think anything really stops us from doing
> > this
> > > now...
> > >
> > >
> > I can change the wording to say we will make a best effort at providing a
> > default implementation but allow that it may not be always possible.
> >
> > St.Ack
> >
> >
> >
> >
> > >
> > > James Taylor wrote:
> > >
> > >> How about the idea of HBase having base implementations for common
> > >> interfaces? That would often help applications built on top of HBase
> to
> > >> maintain backward compatibility from release to release.
> > >>
> > >> On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:
> > >>
> > >> On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang
> > wrote:
> > >>>
> > >>> There are some comments in https://issues.apache.org/
> >  jira/browse/HBASE-17584
> >  .
> > 
> >  In HBASE-17584, we want to remove Scan.getScanMetrics and move the
> >  method
> >  to ResultScanner  which is an interface and part of our public API.
> > 
> >  In our refguide, it is clear that we can do this for a major
> release.
> >  And for patch release, it is also clear that we are not allowed to
> do
> > 
> > >>> this.
> > >>>
> >  New APIs introduced in a patch version will only be added in a
> source
> >  compatible way [1]:
> > i.e.
> >  code that implements public APIs will continue to compile.
> > 
> > 
> >  But for minor release, it is a liitle inexplicit.
> > 
> >  A minor upgrade requires no application/client code modification.
> >  Ideally
> >  it would be a drop-in replacement but client code, coprocessors,
> >  filters,
> >  etc might have to be recompiled if new jars are used.
> > 
> >  If no client code modification is required, then we are not allowed
> to
> > 
> > >>> add
> > >>>
> >  methods to interface as it will cause compliation error if user
> > extends
> > 
> > >>> the
> > >>>
> >  interface as on branch-1, we still need to support JDK7 which does
> not
> >  support default method. But I think this will make minor release
> > almost
> > 
> > >>> the
> > >>>
> >  same with patch release? And another point is that, for most users,
> > they
> >  only use the interface and do not implement it, so adding methods
> will
> > 
> > >>> not
> > >>>
> >  break their code.
> > 
> >  So here we want to see what the community think of whether we should
> > 
> > >>> allow
> > >>>
> >  adding methods to interface in minor release. Any comments are
> > welcomed.
> > 
> > 
> >  I'd suggest we add to the refguide a clarification that allows
> > addition
> > >>> of
> > >>> new methods to Interfaces in minor releases. Here is a suggested
> > >>> addition:
> > >>>
> > >>> "For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may
> > have
> > >>> to
> > >>> recompile. We reserve the right to add new methods to existing
> > Interfaces
> > >>> on minor updates (this will be less of an issue in hbase-2.0.0 which
> > >>> requires JDK8; JDK8 allows specification of 'default' implementations
> > of
> > >>> Interface methods)"
> > >>>
> > >>> I could add a further qualifier only applies to hbase-1.4.0 and
> beyond?
> > >>>
> > >>> St.Ack
> > >>>
> > >>>
> > >>>
> > >>> Thanks.
> > 
> > 
> > >>
> >
>
>
>
> --
> Best regards,
>
>- Andy
>
> If you are given a choice, you believe you have acted freely. - Raymond
> Teller (via Peter Watts)
>

[jira] [Reopened] (HBASE-17707) New More Accurate Table Skew cost function/generator

2017-03-16 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reopened HBASE-17707:


> New More Accurate Table Skew cost function/generator
> 
>
> Key: HBASE-17707
> URL: https://issues.apache.org/jira/browse/HBASE-17707
> Project: HBase
>  Issue Type: New Feature
>  Components: Balancer
>Affects Versions: 1.2.0
> Environment: CentOS Derivative with a derivative of the 3.18.43 
> kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>Reporter: Kahlil Oppenheimer
>Assignee: Kahlil Oppenheimer
>Priority: Minor
> Fix For: 2.0
>
> Attachments: HBASE-17707-00.patch, HBASE-17707-01.patch, 
> HBASE-17707-02.patch, HBASE-17707-03.patch, HBASE-17707-04.patch, 
> HBASE-17707-05.patch, HBASE-17707-06.patch, HBASE-17707-07.patch, 
> HBASE-17707-08.patch, HBASE-17707-09.patch, test-balancer2-13617.out
>
>
> This patch includes new version of the TableSkewCostFunction and a new 
> TableSkewCandidateGenerator.
> The new TableSkewCostFunction computes table skew by counting the minimal 
> number of region moves required for a given table to perfectly balance the 
> table across the cluster (i.e. as if the regions from that table had been 
> round-robin-ed across the cluster). This number of moves is computer for each 
> table, then normalized to a score between 0-1 by dividing by the number of 
> moves required in the absolute worst case (i.e. the entire table is stored on 
> one server), and stored in an array. The cost function then takes a weighted 
> average of the average and maximum value across all tables. The weights in 
> this average are configurable to allow for certain users to more strongly 
> penalize situations where one table is skewed versus where every table is a 
> little bit skewed. To better spread this value more evenly across the range 
> 0-1, we take the square root of the weighted average to get the final value.
> The new TableSkewCandidateGenerator generates region moves/swaps to optimize 
> the above TableSkewCostFunction. It first simply tries to move regions until 
> each server has the right number of regions, then it swaps regions around 
> such that each region swap improves table skew across the cluster.
> We tested the cost function and generator in our production clusters with 
> 100s of TBs of data and 100s of tables across dozens of servers and found 
> both to be very performant and accurate.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread Andrew Purtell 
This sounds like a good result to me.
We've sometimes provided BaseXXX implementations of interfaces for Java <=
7 but haven't been consistent. Would be good to do better.

On Thu, Mar 16, 2017 at 2:18 PM, Stack  wrote:

> On Thu, Mar 16, 2017 at 11:02 AM, Josh Elser  wrote:
>
> > +1 to the JDK8 default implementation approach. I'd say this would be
> good
> > to push forward for the future.
> >
> > For 1.x, I see no reason why we couldn't provide concrete implementations
> > as a stop-gap. Identifying and creating those classes is probably the
> > biggest barrier :). I don't think anything really stops us from doing
> this
> > now...
> >
> >
> I can change the wording to say we will make a best effort at providing a
> default implementation but allow that it may not be always possible.
>
> St.Ack
>
>
>
>
> >
> > James Taylor wrote:
> >
> >> How about the idea of HBase having base implementations for common
> >> interfaces? That would often help applications built on top of HBase to
> >> maintain backward compatibility from release to release.
> >>
> >> On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:
> >>
> >> On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang
> wrote:
> >>>
> >>> There are some comments in https://issues.apache.org/
>  jira/browse/HBASE-17584
>  .
> 
>  In HBASE-17584, we want to remove Scan.getScanMetrics and move the
>  method
>  to ResultScanner  which is an interface and part of our public API.
> 
>  In our refguide, it is clear that we can do this for a major release.
>  And for patch release, it is also clear that we are not allowed to do
> 
> >>> this.
> >>>
>  New APIs introduced in a patch version will only be added in a source
>  compatible way [1]:
> i.e.
>  code that implements public APIs will continue to compile.
> 
> 
>  But for minor release, it is a liitle inexplicit.
> 
>  A minor upgrade requires no application/client code modification.
>  Ideally
>  it would be a drop-in replacement but client code, coprocessors,
>  filters,
>  etc might have to be recompiled if new jars are used.
> 
>  If no client code modification is required, then we are not allowed to
> 
> >>> add
> >>>
>  methods to interface as it will cause compliation error if user
> extends
> 
> >>> the
> >>>
>  interface as on branch-1, we still need to support JDK7 which does not
>  support default method. But I think this will make minor release
> almost
> 
> >>> the
> >>>
>  same with patch release? And another point is that, for most users,
> they
>  only use the interface and do not implement it, so adding methods will
> 
> >>> not
> >>>
>  break their code.
> 
>  So here we want to see what the community think of whether we should
> 
> >>> allow
> >>>
>  adding methods to interface in minor release. Any comments are
> welcomed.
> 
> 
>  I'd suggest we add to the refguide a clarification that allows
> addition
> >>> of
> >>> new methods to Interfaces in minor releases. Here is a suggested
> >>> addition:
> >>>
> >>> "For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may
> have
> >>> to
> >>> recompile. We reserve the right to add new methods to existing
> Interfaces
> >>> on minor updates (this will be less of an issue in hbase-2.0.0 which
> >>> requires JDK8; JDK8 allows specification of 'default' implementations
> of
> >>> Interface methods)"
> >>>
> >>> I could add a further qualifier only applies to hbase-1.4.0 and beyond?
> >>>
> >>> St.Ack
> >>>
> >>>
> >>>
> >>> Thanks.
> 
> 
> >>
>



-- 
Best regards,

   - Andy

If you are given a choice, you believe you have acted freely. - Raymond
Teller (via Peter Watts)

[jira] [Created] (HBASE-17794) Remove lingering "violation" in favor of the accurate "snapshot"

2017-03-16 Thread Josh Elser (JIRA) 
Josh Elser created HBASE-17794:
--

 Summary: Remove lingering "violation" in favor of the accurate 
"snapshot"
 Key: HBASE-17794
 URL: https://issues.apache.org/jira/browse/HBASE-17794
 Project: HBase
  Issue Type: Sub-task
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Trivial
 Fix For: HBASE-16961


Previously, quota violations used to be stored in the quota table. Eventually, 
I realized that I needed to actually persist the utilization of each table/NS, 
not just the violation state.

I know there are cases in the code where "violation" is incorrectly describing 
things (it should be "snapshot"). Clean these up for clarity.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread Stack 
On Thu, Mar 16, 2017 at 11:02 AM, Josh Elser  wrote:

> +1 to the JDK8 default implementation approach. I'd say this would be good
> to push forward for the future.
>
> For 1.x, I see no reason why we couldn't provide concrete implementations
> as a stop-gap. Identifying and creating those classes is probably the
> biggest barrier :). I don't think anything really stops us from doing this
> now...
>
>
I can change the wording to say we will make a best effort at providing a
default implementation but allow that it may not be always possible.

St.Ack




>
> James Taylor wrote:
>
>> How about the idea of HBase having base implementations for common
>> interfaces? That would often help applications built on top of HBase to
>> maintain backward compatibility from release to release.
>>
>> On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:
>>
>> On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang  wrote:
>>>
>>> There are some comments in https://issues.apache.org/
 jira/browse/HBASE-17584
 .

 In HBASE-17584, we want to remove Scan.getScanMetrics and move the
 method
 to ResultScanner  which is an interface and part of our public API.

 In our refguide, it is clear that we can do this for a major release.
 And for patch release, it is also clear that we are not allowed to do

>>> this.
>>>
 New APIs introduced in a patch version will only be added in a source
 compatible way [1]: i.e.
 code that implements public APIs will continue to compile.


 But for minor release, it is a liitle inexplicit.

 A minor upgrade requires no application/client code modification.
 Ideally
 it would be a drop-in replacement but client code, coprocessors,
 filters,
 etc might have to be recompiled if new jars are used.

 If no client code modification is required, then we are not allowed to

>>> add
>>>
 methods to interface as it will cause compliation error if user extends

>>> the
>>>
 interface as on branch-1, we still need to support JDK7 which does not
 support default method. But I think this will make minor release almost

>>> the
>>>
 same with patch release? And another point is that, for most users, they
 only use the interface and do not implement it, so adding methods will

>>> not
>>>
 break their code.

 So here we want to see what the community think of whether we should

>>> allow
>>>
 adding methods to interface in minor release. Any comments are welcomed.


 I'd suggest we add to the refguide a clarification that allows addition
>>> of
>>> new methods to Interfaces in minor releases. Here is a suggested
>>> addition:
>>>
>>> "For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may have
>>> to
>>> recompile. We reserve the right to add new methods to existing Interfaces
>>> on minor updates (this will be less of an issue in hbase-2.0.0 which
>>> requires JDK8; JDK8 allows specification of 'default' implementations of
>>> Interface methods)"
>>>
>>> I could add a further qualifier only applies to hbase-1.4.0 and beyond?
>>>
>>> St.Ack
>>>
>>>
>>>
>>> Thanks.


>>

[jira] [Resolved] (HBASE-17525) SecureBulkLoadEndpoint not implemented

2017-03-16 Thread Josh Elser (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser resolved HBASE-17525.

Resolution: Won't Fix

Making the call that we'll not include this -- the "standard" path forward 
should work.

> SecureBulkLoadEndpoint not implemented
> --
>
> Key: HBASE-17525
> URL: https://issues.apache.org/jira/browse/HBASE-17525
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Josh Elser
>Assignee: Josh Elser
> Attachments: HBASE-17525.001.patch
>
>
> Looks like I only implemented the RSRpcServices method for loading hfiles, 
> not the SecureBulkLoadEndpoint. Need to do that one too and add a test.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread Ted Yu 
+1 to what Josh and James said.

On Thu, Mar 16, 2017 at 11:02 AM, Josh Elser  wrote:

> +1 to the JDK8 default implementation approach. I'd say this would be good
> to push forward for the future.
>
> For 1.x, I see no reason why we couldn't provide concrete implementations
> as a stop-gap. Identifying and creating those classes is probably the
> biggest barrier :). I don't think anything really stops us from doing this
> now...
>
> James Taylor wrote:
>
>> How about the idea of HBase having base implementations for common
>> interfaces? That would often help applications built on top of HBase to
>> maintain backward compatibility from release to release.
>>
>> On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:
>>
>> On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang  wrote:
>>>
>>> There are some comments in https://issues.apache.org/
 jira/browse/HBASE-17584
 .

 In HBASE-17584, we want to remove Scan.getScanMetrics and move the
 method
 to ResultScanner  which is an interface and part of our public API.

 In our refguide, it is clear that we can do this for a major release.
 And for patch release, it is also clear that we are not allowed to do

>>> this.
>>>
 New APIs introduced in a patch version will only be added in a source
 compatible way [1]: i.e.
 code that implements public APIs will continue to compile.


 But for minor release, it is a liitle inexplicit.

 A minor upgrade requires no application/client code modification.
 Ideally
 it would be a drop-in replacement but client code, coprocessors,
 filters,
 etc might have to be recompiled if new jars are used.

 If no client code modification is required, then we are not allowed to

>>> add
>>>
 methods to interface as it will cause compliation error if user extends

>>> the
>>>
 interface as on branch-1, we still need to support JDK7 which does not
 support default method. But I think this will make minor release almost

>>> the
>>>
 same with patch release? And another point is that, for most users, they
 only use the interface and do not implement it, so adding methods will

>>> not
>>>
 break their code.

 So here we want to see what the community think of whether we should

>>> allow
>>>
 adding methods to interface in minor release. Any comments are welcomed.


 I'd suggest we add to the refguide a clarification that allows addition
>>> of
>>> new methods to Interfaces in minor releases. Here is a suggested
>>> addition:
>>>
>>> "For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may have
>>> to
>>> recompile. We reserve the right to add new methods to existing Interfaces
>>> on minor updates (this will be less of an issue in hbase-2.0.0 which
>>> requires JDK8; JDK8 allows specification of 'default' implementations of
>>> Interface methods)"
>>>
>>> I could add a further qualifier only applies to hbase-1.4.0 and beyond?
>>>
>>> St.Ack
>>>
>>>
>>>
>>> Thanks.


>>

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread Enis Söztutar 
+1 for the wording in the book. Otherwise, we cannot add methods to Table /
Admin in a minor release, which is very restricting.

> How about the idea of HBase having base implementations for common
interfaces
This is only needed in JDK-7, so basically in 1.x releases. With Java-8, we
are doing default methods in interfaces instead.

Enis

On Thu, Mar 16, 2017 at 11:02 AM, Josh Elser  wrote:

> +1 to the JDK8 default implementation approach. I'd say this would be good
> to push forward for the future.
>
> For 1.x, I see no reason why we couldn't provide concrete implementations
> as a stop-gap. Identifying and creating those classes is probably the
> biggest barrier :). I don't think anything really stops us from doing this
> now...
>
>
> James Taylor wrote:
>
>> How about the idea of HBase having base implementations for common
>> interfaces? That would often help applications built on top of HBase to
>> maintain backward compatibility from release to release.
>>
>> On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:
>>
>> On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang  wrote:
>>>
>>> There are some comments in https://issues.apache.org/
 jira/browse/HBASE-17584
 .

 In HBASE-17584, we want to remove Scan.getScanMetrics and move the
 method
 to ResultScanner  which is an interface and part of our public API.

 In our refguide, it is clear that we can do this for a major release.
 And for patch release, it is also clear that we are not allowed to do

>>> this.
>>>
 New APIs introduced in a patch version will only be added in a source
 compatible way [1]: i.e.
 code that implements public APIs will continue to compile.


 But for minor release, it is a liitle inexplicit.

 A minor upgrade requires no application/client code modification.
 Ideally
 it would be a drop-in replacement but client code, coprocessors,
 filters,
 etc might have to be recompiled if new jars are used.

 If no client code modification is required, then we are not allowed to

>>> add
>>>
 methods to interface as it will cause compliation error if user extends

>>> the
>>>
 interface as on branch-1, we still need to support JDK7 which does not
 support default method. But I think this will make minor release almost

>>> the
>>>
 same with patch release? And another point is that, for most users, they
 only use the interface and do not implement it, so adding methods will

>>> not
>>>
 break their code.

 So here we want to see what the community think of whether we should

>>> allow
>>>
 adding methods to interface in minor release. Any comments are welcomed.


 I'd suggest we add to the refguide a clarification that allows addition
>>> of
>>> new methods to Interfaces in minor releases. Here is a suggested
>>> addition:
>>>
>>> "For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may have
>>> to
>>> recompile. We reserve the right to add new methods to existing Interfaces
>>> on minor updates (this will be less of an issue in hbase-2.0.0 which
>>> requires JDK8; JDK8 allows specification of 'default' implementations of
>>> Interface methods)"
>>>
>>> I could add a further qualifier only applies to hbase-1.4.0 and beyond?
>>>
>>> St.Ack
>>>
>>>
>>>
>>> Thanks.


>>

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread Josh Elser 
+1 to the JDK8 default implementation approach. I'd say this would be 
good to push forward for the future.


For 1.x, I see no reason why we couldn't provide concrete 
implementations as a stop-gap. Identifying and creating those classes is 
probably the biggest barrier :). I don't think anything really stops us 
from doing this now...


James Taylor wrote:

How about the idea of HBase having base implementations for common
interfaces? That would often help applications built on top of HBase to
maintain backward compatibility from release to release.

On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:


On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang  wrote:


There are some comments in https://issues.apache.org/
jira/browse/HBASE-17584
.

In HBASE-17584, we want to remove Scan.getScanMetrics and move the method
to ResultScanner  which is an interface and part of our public API.

In our refguide, it is clear that we can do this for a major release.
And for patch release, it is also clear that we are not allowed to do

this.

New APIs introduced in a patch version will only be added in a source
compatible way [1]: i.e.
code that implements public APIs will continue to compile.


But for minor release, it is a liitle inexplicit.

A minor upgrade requires no application/client code modification. Ideally
it would be a drop-in replacement but client code, coprocessors, filters,
etc might have to be recompiled if new jars are used.

If no client code modification is required, then we are not allowed to

add

methods to interface as it will cause compliation error if user extends

the

interface as on branch-1, we still need to support JDK7 which does not
support default method. But I think this will make minor release almost

the

same with patch release? And another point is that, for most users, they
only use the interface and do not implement it, so adding methods will

not

break their code.

So here we want to see what the community think of whether we should

allow

adding methods to interface in minor release. Any comments are welcomed.



I'd suggest we add to the refguide a clarification that allows addition of
new methods to Interfaces in minor releases. Here is a suggested addition:

"For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may have to
recompile. We reserve the right to add new methods to existing Interfaces
on minor updates (this will be less of an issue in hbase-2.0.0 which
requires JDK8; JDK8 allows specification of 'default' implementations of
Interface methods)"

I could add a further qualifier only applies to hbase-1.4.0 and beyond?

St.Ack




Thanks.

[jira] [Resolved] (HBASE-17789) ZKConfig.getZKQuorumServersStringFromHbaseConfig creates a incorrect zkquorum string

2017-03-16 Thread Josh Elser (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser resolved HBASE-17789.

Resolution: Invalid

> ZKConfig.getZKQuorumServersStringFromHbaseConfig creates a incorrect zkquorum 
> string
> 
>
> Key: HBASE-17789
> URL: https://issues.apache.org/jira/browse/HBASE-17789
> Project: HBase
>  Issue Type: Bug
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>
> , if we try to a hbase using "HBaseAdmin" by reading a value of 
> ""hbase.zookeeper.quorum" from list peers with value as 
> "a:2181,b:2181c:2181:/hbase". 
> org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersStringFromHbaseConfig
>  returns a incorrect queryString which causes following exception Because 
> last host would see  :/hbase also in the string. This method was existing 
> earlier but never being called before above change



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

2017-03-16 Thread Vladimir Rodionov 
v61 is against current master.

On Thu, Mar 16, 2017 at 7:18 AM, Stack  wrote:

> On Wed, Mar 15, 2017 at 11:38 PM, Vladimir Rodionov <
> vladrodio...@gmail.com>
> wrote:
>
> > >> I thought I was reviewing a patch that was against master
> >
> > Yes, it was against master. We used to develop in HBASE-7912 branch, but
> it
> > was abandoned 6-8 mo ago.
> > This is why the only way to merge backup is to apply v61 directly to
> master
> > branch.
> >
> >
> If you are saying that v61 is against a master of 6-8 months ago, then I'd
> think this VOTE is premature. Don't you have a mountain of rebasing to do
> before you can put your patch up for a vote?
>
> St.Ack
>
>
>
> > -Vlad
> >
> >
> > On Wed, Mar 15, 2017 at 9:17 PM, Stack  wrote:
> >
> > > On Wed, Mar 15, 2017 at 5:51 PM, Ted Yu  wrote:
> > >
> > > > Thanks for the vote, Josh.
> > > >
> > > > So far there have been 3 +1's (Enis', mine and Josh's)
> > > >
> > > > Should we start another thread on the procedure of merging or use
> this
> > > > thread ?
> > > > Background:
> > > > HBASE-7912 branch is way out of date.
> > > > Latest rounds of mega patch were based on then-current master branch.
> > > >
> > > >
> > > 1. Vlad is running this VOTE, not you.
> > > 2. Don't you need 3 PMC votes to merge? I count 2.
> > > 3. You almost derailed the merge with your earlier interjection rushing
> > to
> > > take us in the wrong direction; you'd think you'd learn from your past
> > > experience but here you are again w/ haste and rash summary.
> > > 4. This vote has been awkward. Josh's note is considered and careful.
> > You'd
> > > think it could hang out there a while to see if draws a response rather
> > > than rush to close.
> > >
> > > I thought I was reviewing a patch that was against master, not some
> > version
> > > of master from eons ago. Thats a problem. When was the last rebase?
> > >
> > > St.Ack
> > >
> > >
> > >
> > >
> > >
> > > > Thanks
> > > >
> > > > On Wed, Mar 15, 2017 at 4:22 PM, Josh Elser 
> wrote:
> > > >
> > > > > Spent the day playing around with this on 6 nodes.
> > > > >
> > > > > I've found some rough edges: some known (the docs blocker Vlad
> > pointed
> > > > > out) and some that I think are unknown (tooling is rough --
> partially
> > > > from
> > > > > wrong help messages and, I think, changes in design like
> > > Master-submitted
> > > > > MR jobs).
> > > > >
> > > > > But, Stack's assessment (and Andrew's reminder) that further
> tweaking
> > > > > would just throw us back into another review cycle is a real
> concern.
> > > > It's
> > > > > unfortunate that this feature has lingered so long aside of master
> to
> > > get
> > > > > to this point, but I don't see any realistic resolution to this
> > problem
> > > > > than a merge. In the future, this is something we'll have to try
> > harder
> > > > to
> > > > > avoid letting happen (looking in the mirror with quota work...)
> > > > >
> > > > > Thankfully, I can help out with development/review on the
> outstanding
> > > > > blockers (notably, the two I pruned from Vlad's original of five --
> > the
> > > > > other three still seem to be improvements). In addition to these
> > > > blockers,
> > > > > I believe the documentation *must* be updated before a release to
> > note
> > > > that
> > > > > this feature is still growing -- it does not feel like a quality
> > > feature
> > > > > that I've come to expect from this community. This isn't a knock on
> > > Vlad
> > > > > and company; this is a hard problem and one that I could not have
> > done
> > > > > better given time constraints, but it is also one that users will
> > > demand
> > > > > simplicity and the utmost correctness around. To this end, I will
> > also
> > > > try
> > > > > to help out to smooth out these issues in the following 2.x
> release.
> > > > >
> > > > > So, this leaves me to say: +1 to merge with the caveat that the
> docs
> > > are
> > > > > updated to make sure that any known, user-pitfalls are clearly
> > > > documented.
> > > > > This vote also comes with as real of a promise that I can make to
> > help
> > > > > avoid any issues with this feature that would prevent a 2.0
> release.
> > > > >
> > > > > Thanks to all the giants whose shoulders I'm standing on to be able
> > to
> > > > > make this vote.
> > > > >
> > > > > Andrew Purtell wrote:
> > > > >
> > > > >> Great, and I changed my vote to -0 because Stack made a good
> > argument
> > > > that
> > > > >> making more changes would invalidate review up to this point, and
> I
> > > > trust
> > > > >> this will be resolved before release.
> > > > >>
> > > > >> On Tue, Mar 14, 2017 at 4:29 PM, Josh Elser
> > > wrote:
> > > > >>
> > > > >> Sorry Andrew, let me clarify as that didn't come out right.
> > > > >>>
> > > > >>> I didn't mean that isn't a conversation worth having _now_, just
> > > that I
> > > > >>> was intentionally avoiding it in my previous email

Successful: HBase Generate Website

2017-03-16 Thread Apache Jenkins Server 
Build status: Successful

If successful, the website and docs have been generated. To update the live 
site, follow the instructions below. If failed, skip to the bottom of this 
email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git

  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/517/artifact/website.patch.zip
 | funzip > e67eb6c424d76ee259f5076c277454a73e3a2bf4.patch
  git fetch
  git checkout -b asf-site-e67eb6c424d76ee259f5076c277454a73e3a2bf4 
origin/asf-site
  git am --whitespace=fix e67eb6c424d76ee259f5076c277454a73e3a2bf4.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-e67eb6c424d76ee259f5076c277454a73e3a2bf4 branch.

There are lots of spurious changes, such as timestamps and CSS styles in 
tables, so a generic git diff is not very useful. To see a list of files that 
have been added, deleted, renamed, changed type, or are otherwise interesting, 
use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 100 or more lines changed:

  git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'

When you are satisfied, publish your changes to origin/asf-site using these 
commands:

  git commit --allow-empty -m "Empty commit" # to work around a current ASF 
INFRA bug
  git push origin asf-site-e67eb6c424d76ee259f5076c277454a73e3a2bf4:asf-site
  git checkout asf-site
  git branch -D asf-site-e67eb6c424d76ee259f5076c277454a73e3a2bf4

Changes take a couple of minutes to be propagated. You can verify whether they 
have been propagated by looking at the Last Published date at the bottom of 
http://hbase.apache.org/. It should match the date in the index.html on the 
asf-site branch in Git.

As a courtesy- reply-all to this email to let other committers know you pushed 
the site.



If failed, see https://builds.apache.org/job/hbase_generate_website/517/console

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

2017-03-16 Thread Stack 
On Wed, Mar 15, 2017 at 11:38 PM, Vladimir Rodionov 
wrote:

> >> I thought I was reviewing a patch that was against master
>
> Yes, it was against master. We used to develop in HBASE-7912 branch, but it
> was abandoned 6-8 mo ago.
> This is why the only way to merge backup is to apply v61 directly to master
> branch.
>
>
If you are saying that v61 is against a master of 6-8 months ago, then I'd
think this VOTE is premature. Don't you have a mountain of rebasing to do
before you can put your patch up for a vote?

St.Ack



> -Vlad
>
>
> On Wed, Mar 15, 2017 at 9:17 PM, Stack  wrote:
>
> > On Wed, Mar 15, 2017 at 5:51 PM, Ted Yu  wrote:
> >
> > > Thanks for the vote, Josh.
> > >
> > > So far there have been 3 +1's (Enis', mine and Josh's)
> > >
> > > Should we start another thread on the procedure of merging or use this
> > > thread ?
> > > Background:
> > > HBASE-7912 branch is way out of date.
> > > Latest rounds of mega patch were based on then-current master branch.
> > >
> > >
> > 1. Vlad is running this VOTE, not you.
> > 2. Don't you need 3 PMC votes to merge? I count 2.
> > 3. You almost derailed the merge with your earlier interjection rushing
> to
> > take us in the wrong direction; you'd think you'd learn from your past
> > experience but here you are again w/ haste and rash summary.
> > 4. This vote has been awkward. Josh's note is considered and careful.
> You'd
> > think it could hang out there a while to see if draws a response rather
> > than rush to close.
> >
> > I thought I was reviewing a patch that was against master, not some
> version
> > of master from eons ago. Thats a problem. When was the last rebase?
> >
> > St.Ack
> >
> >
> >
> >
> >
> > > Thanks
> > >
> > > On Wed, Mar 15, 2017 at 4:22 PM, Josh Elser  wrote:
> > >
> > > > Spent the day playing around with this on 6 nodes.
> > > >
> > > > I've found some rough edges: some known (the docs blocker Vlad
> pointed
> > > > out) and some that I think are unknown (tooling is rough -- partially
> > > from
> > > > wrong help messages and, I think, changes in design like
> > Master-submitted
> > > > MR jobs).
> > > >
> > > > But, Stack's assessment (and Andrew's reminder) that further tweaking
> > > > would just throw us back into another review cycle is a real concern.
> > > It's
> > > > unfortunate that this feature has lingered so long aside of master to
> > get
> > > > to this point, but I don't see any realistic resolution to this
> problem
> > > > than a merge. In the future, this is something we'll have to try
> harder
> > > to
> > > > avoid letting happen (looking in the mirror with quota work...)
> > > >
> > > > Thankfully, I can help out with development/review on the outstanding
> > > > blockers (notably, the two I pruned from Vlad's original of five --
> the
> > > > other three still seem to be improvements). In addition to these
> > > blockers,
> > > > I believe the documentation *must* be updated before a release to
> note
> > > that
> > > > this feature is still growing -- it does not feel like a quality
> > feature
> > > > that I've come to expect from this community. This isn't a knock on
> > Vlad
> > > > and company; this is a hard problem and one that I could not have
> done
> > > > better given time constraints, but it is also one that users will
> > demand
> > > > simplicity and the utmost correctness around. To this end, I will
> also
> > > try
> > > > to help out to smooth out these issues in the following 2.x release.
> > > >
> > > > So, this leaves me to say: +1 to merge with the caveat that the docs
> > are
> > > > updated to make sure that any known, user-pitfalls are clearly
> > > documented.
> > > > This vote also comes with as real of a promise that I can make to
> help
> > > > avoid any issues with this feature that would prevent a 2.0 release.
> > > >
> > > > Thanks to all the giants whose shoulders I'm standing on to be able
> to
> > > > make this vote.
> > > >
> > > > Andrew Purtell wrote:
> > > >
> > > >> Great, and I changed my vote to -0 because Stack made a good
> argument
> > > that
> > > >> making more changes would invalidate review up to this point, and I
> > > trust
> > > >> this will be resolved before release.
> > > >>
> > > >> On Tue, Mar 14, 2017 at 4:29 PM, Josh Elser
> > wrote:
> > > >>
> > > >> Sorry Andrew, let me clarify as that didn't come out right.
> > > >>>
> > > >>> I didn't mean that isn't a conversation worth having _now_, just
> > that I
> > > >>> was intentionally avoiding it in my previous email because I didn't
> > > >>> understand the scope of those issues that Vlad had identified. I
> > wanted
> > > >>> to
> > > >>> better understand what they were really meant for a user before
> > coming
> > > to
> > > >>> my own decision about whether or not I think they are blockers.
> > > >>>
> > > >>> That aside, I would agree with you that HBASE-15227 sounds like a
> > real
> > > >>>

Re: Assigning regions after restart

2017-03-16 Thread Lars George 
JIRA creation is done: https://issues.apache.org/jira/browse/HBASE-17791

I will have a look into it over the next few days, maybe I can come up
with a patch.

On Wed, Mar 15, 2017 at 6:14 AM, Stack  wrote:
> File a blocker please Lars. I'm pretty sure the boolean on whether we are
> doing a recovery or not has been there a long time so yeah, a single server
> recovery could throw us off, but you make a practical point, that one
> server should not destroy locality over the cluster.
>
> St.Ack
>
> On Tue, Mar 14, 2017 at 7:09 AM, Lars George  wrote:
>
>> Wait, HBASE-15251 is not enough methinks. The checks added help, but
>> are not covering all the possible edge cases. In particular, say a
>> node really fails, why not just reassign the few regions it did hold
>> and leave all the others where they are? Seems insane as it is.
>>
>> On Tue, Mar 14, 2017 at 2:24 PM, Lars George 
>> wrote:
>> > Doh, https://issues.apache.org/jira/browse/HBASE-15251 addresses this
>> > (though I am not sure exactly how, see below). This should be
>> > backported to all 1.x branches!
>> >
>> > As for the patch, I see this
>> >
>> >  if (!failover) {
>> >// Fresh cluster startup.
>> > -  LOG.info("Clean cluster startup. Assigning user regions");
>> > +  LOG.info("Clean cluster startup. Don't reassign user regions");
>> >assignAllUserRegions(allRegions);
>> > +} else {
>> > +  LOG.info("Failover! Reassign user regions");
>> >  }
>> >
>> > Would be interesting to see how that log message is actually
>> > reassigning the user regions. I would have assumed the
>> > "assignAllUserRegions()" would have to be called.
>> >
>> >
>> > On Tue, Mar 14, 2017 at 2:21 PM, Lars George 
>> wrote:
>> >> Looking at the code more... it seems the issue is here
>> >>
>> >> In AssignmentManager.processDeadServersAndRegionsInTransition():
>> >>
>> >> ...
>> >> failoverCleanupDone();
>> >> if (!failover) {
>> >>   // Fresh cluster startup.
>> >>   LOG.info("Clean cluster startup. Assigning user regions");
>> >>   assignAllUserRegions(allRegions);
>> >> }
>> >> ...
>> >>
>> >> As soon as a single node failed, failover is set to true. So no
>> >> assignment is done. Later on the servers are recovered from the
>> >> "crash" (which a clean restart is as well), and then all unassigned
>> >> regions (all of them now, since they are unassigned earlier) are
>> >> assigned round-robin.
>> >>
>> >>
>> >>
>> >> On Tue, Mar 14, 2017 at 1:54 PM, Lars George 
>> wrote:
>> >>> Hi,
>> >>>
>> >>> I had this happened at multiple clusters recently where after the
>> >>> restart the locality dropped from close to or exactly 100% down to
>> >>> single digits. The reason is that all regions were completely shuffled
>> >>> and reassigned to random servers. Upon reading the (yet again
>> >>> non-trivial) assignment code, I found that a single server missing
>> >>> will trigger a full "recovery" of all servers, which includes a drop
>> >>> of all previous assignments and random new assignment.
>> >>>
>> >>> This is just terrible! In addition, I also assumed that - at least the
>> >>> StochasticLoadBalancer - is checking which node holds most of the data
>> >>> of a region locality wise and picks that server. But that is _not_ the
>> >>> case! It just spreads everything seemingly randomly across the
>> >>> servers.
>> >>>
>> >>> To me this is a big regression (or straight bug) given that a single
>> >>> server out of, for example, hundreds could trigger that and destroy
>> >>> the locality completely. Running a major compaction is not an approach
>> >>> for many reasons.
>> >>>
>> >>> This used to work better, why that regression?
>> >>>
>> >>> Lars
>>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

2017-03-16 Thread Vladimir Rodionov 
>> I thought I was reviewing a patch that was against master

Yes, it was against master. We used to develop in HBASE-7912 branch, but it
was abandoned 6-8 mo ago.
This is why the only way to merge backup is to apply v61 directly to master
branch.

-Vlad


On Wed, Mar 15, 2017 at 9:17 PM, Stack  wrote:

> On Wed, Mar 15, 2017 at 5:51 PM, Ted Yu  wrote:
>
> > Thanks for the vote, Josh.
> >
> > So far there have been 3 +1's (Enis', mine and Josh's)
> >
> > Should we start another thread on the procedure of merging or use this
> > thread ?
> > Background:
> > HBASE-7912 branch is way out of date.
> > Latest rounds of mega patch were based on then-current master branch.
> >
> >
> 1. Vlad is running this VOTE, not you.
> 2. Don't you need 3 PMC votes to merge? I count 2.
> 3. You almost derailed the merge with your earlier interjection rushing to
> take us in the wrong direction; you'd think you'd learn from your past
> experience but here you are again w/ haste and rash summary.
> 4. This vote has been awkward. Josh's note is considered and careful. You'd
> think it could hang out there a while to see if draws a response rather
> than rush to close.
>
> I thought I was reviewing a patch that was against master, not some version
> of master from eons ago. Thats a problem. When was the last rebase?
>
> St.Ack
>
>
>
>
>
> > Thanks
> >
> > On Wed, Mar 15, 2017 at 4:22 PM, Josh Elser  wrote:
> >
> > > Spent the day playing around with this on 6 nodes.
> > >
> > > I've found some rough edges: some known (the docs blocker Vlad pointed
> > > out) and some that I think are unknown (tooling is rough -- partially
> > from
> > > wrong help messages and, I think, changes in design like
> Master-submitted
> > > MR jobs).
> > >
> > > But, Stack's assessment (and Andrew's reminder) that further tweaking
> > > would just throw us back into another review cycle is a real concern.
> > It's
> > > unfortunate that this feature has lingered so long aside of master to
> get
> > > to this point, but I don't see any realistic resolution to this problem
> > > than a merge. In the future, this is something we'll have to try harder
> > to
> > > avoid letting happen (looking in the mirror with quota work...)
> > >
> > > Thankfully, I can help out with development/review on the outstanding
> > > blockers (notably, the two I pruned from Vlad's original of five -- the
> > > other three still seem to be improvements). In addition to these
> > blockers,
> > > I believe the documentation *must* be updated before a release to note
> > that
> > > this feature is still growing -- it does not feel like a quality
> feature
> > > that I've come to expect from this community. This isn't a knock on
> Vlad
> > > and company; this is a hard problem and one that I could not have done
> > > better given time constraints, but it is also one that users will
> demand
> > > simplicity and the utmost correctness around. To this end, I will also
> > try
> > > to help out to smooth out these issues in the following 2.x release.
> > >
> > > So, this leaves me to say: +1 to merge with the caveat that the docs
> are
> > > updated to make sure that any known, user-pitfalls are clearly
> > documented.
> > > This vote also comes with as real of a promise that I can make to help
> > > avoid any issues with this feature that would prevent a 2.0 release.
> > >
> > > Thanks to all the giants whose shoulders I'm standing on to be able to
> > > make this vote.
> > >
> > > Andrew Purtell wrote:
> > >
> > >> Great, and I changed my vote to -0 because Stack made a good argument
> > that
> > >> making more changes would invalidate review up to this point, and I
> > trust
> > >> this will be resolved before release.
> > >>
> > >> On Tue, Mar 14, 2017 at 4:29 PM, Josh Elser
> wrote:
> > >>
> > >> Sorry Andrew, let me clarify as that didn't come out right.
> > >>>
> > >>> I didn't mean that isn't a conversation worth having _now_, just
> that I
> > >>> was intentionally avoiding it in my previous email because I didn't
> > >>> understand the scope of those issues that Vlad had identified. I
> wanted
> > >>> to
> > >>> better understand what they were really meant for a user before
> coming
> > to
> > >>> my own decision about whether or not I think they are blockers.
> > >>>
> > >>> That aside, I would agree with you that HBASE-15227 sounds like a
> real
> > >>> blocker. Forcing a new full backup for every table sounds really bad
> --
> > >>> that's not the kind of experience we'd want a user to have.
> > >>>
> > >>> Andrew Purtell wrote:
> > >>>
> > >>> I'm going to intentional avoid addressing the discussion of shipping
> >  partial features (related, but not relevant at the moment).
> > 
> >  Then we are not having the same conversation, because it is
> precisely
> >  because this is a vote for this feature to go into 2.0, which is
> > already
> >  overdue, so should be released

Re: About adding methods to an interface which is part of our public API in minor release

2017-03-16 Thread James Taylor 
How about the idea of HBase having base implementations for common
interfaces? That would often help applications built on top of HBase to
maintain backward compatibility from release to release.

On Wed, Mar 15, 2017 at 10:38 PM Stack  wrote:

> On Wed, Mar 15, 2017 at 10:28 PM, Duo Zhang  wrote:
>
> > There are some comments in https://issues.apache.org/
> > jira/browse/HBASE-17584
> > .
> >
> > In HBASE-17584, we want to remove Scan.getScanMetrics and move the method
> > to ResultScanner  which is an interface and part of our public API.
> >
> > In our refguide, it is clear that we can do this for a major release.
> > And for patch release, it is also clear that we are not allowed to do
> this.
> >
> > New APIs introduced in a patch version will only be added in a source
> > compatible way [1 ]: i.e.
> > code that implements public APIs will continue to compile.
> >
> >
> > But for minor release, it is a liitle inexplicit.
> >
> > A minor upgrade requires no application/client code modification. Ideally
> > it would be a drop-in replacement but client code, coprocessors, filters,
> > etc might have to be recompiled if new jars are used.
> >
> > If no client code modification is required, then we are not allowed to
> add
> > methods to interface as it will cause compliation error if user extends
> the
> > interface as on branch-1, we still need to support JDK7 which does not
> > support default method. But I think this will make minor release almost
> the
> > same with patch release? And another point is that, for most users, they
> > only use the interface and do not implement it, so adding methods will
> not
> > break their code.
> >
> > So here we want to see what the community think of whether we should
> allow
> > adding methods to interface in minor release. Any comments are welcomed.
> >
> >
> I'd suggest we add to the refguide a clarification that allows addition of
> new methods to Interfaces in minor releases. Here is a suggested addition:
>
> "For a minor release, IF YOU HAVE IMPLEMENTED AN INTERFACE, you may have to
> recompile. We reserve the right to add new methods to existing Interfaces
> on minor updates (this will be less of an issue in hbase-2.0.0 which
> requires JDK8; JDK8 allows specification of 'default' implementations of
> Interface methods)"
>
> I could add a further qualifier only applies to hbase-1.4.0 and beyond?
>
> St.Ack
>
>
>
> > Thanks.
> >
>