Re: [ANNOUNCE] New HBase committer Zheng Hu

[Misty Stanley-Jones]

Welcome Zheng Hu and thanks for your contributions to the project!

On Sun, Oct 22, 2017 at 11:18 PM Duo Zhang  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Zheng's generous contributions thus far and look forward
> to his continued involvement.
>
> Congratulations and welcome, Zheng!
>

[jira] [Created] (HBASE-19077) Have Region*CoprocessorEnvironment provide an ImmutableOnlineRegions

[stack (JIRA)]

stack created HBASE-19077:
-

 Summary: Have Region*CoprocessorEnvironment provide an 
ImmutableOnlineRegions
 Key: HBASE-19077
 URL: https://issues.apache.org/jira/browse/HBASE-19077
 Project: HBase
  Issue Type: Bug
  Components: Coprocessors
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0-alpha-4


This is about restoring some of the hbase1 parity... You used to be able to get 
list of Region on the local server and talk to them directly.

Suggested by [~anoop.hbase] up in review. Makes sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Moving 2.0 forward

[Stack]

That'd be a good one to get in Ram.
S

On Mon, Oct 23, 2017 at 9:42 PM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Hi Stack
>
> Do you want HBASE-18995 before the alpha-4 (REmoving exposed internal APIs
> from CellUtil)? Because you had mentioned no more API changes. If so I will
> start making changes and put up a patch ASAP.
>
> Regards
> Ram
>
> On Tue, Oct 24, 2017 at 3:22 AM, Stack  wrote:
>
> > On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser  wrote:
> >
> > > +1
> > >
> > > I was trying to work on helping out on the outstanding alpha-4 stuff
> last
> > > week -- will be continuing to try to do the same this week.
> > >
> > > If you need any help, Stack, or if others need reviews where I haven't
> > > noticed on my own: feel free to @mention me.
> > >
> > >
> > Thanks for the offer Josh. All items seem assigned and are being actively
> > worked on. If you get a moment, reviews by you (or anyone else) helps
> move
> > the process along.
> >
> > We need to merge in HBASE-18410 branch to pick up Filter improvements.
> Then
> > HBASE-13346 can go in.
> >
> > You are already helping out on HBASE-18906, thanks. Looks like that will
> be
> > addressed by other alpha-4s about to land.
> >
> > St.Ack
> > TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > > On 10/23/17 12:53 PM, Stack wrote:
> > >
> > >> (Reviving this thread)
> > >>
> > >> Lets push out alpha-4 this week. Alpha-4 is the release that has the
> > >> refactor of the Coprocessor API shutting down access to internals
> marked
> > >> InterfaceAudience.Private.
> > >>
> > >> The outstanding list is here:
> > >> https://issues.apache.org/jira/projects/HBASE/versions/12341594
> > >>
> > >> Please push in anything marked alpha-4 that belongs to you.
> > >>
> > >> If issue, talk out loud on this thread. If you need a review to land
> an
> > >> item, shout on the issue and here; we'll help you out.
> > >>
> > >> As is, items are coming along nicely I'd say. We need to merge the
> > filter
> > >> branch -- HBASE-18410 -- so APIs are finished for hbase2.
> > >>
> > >> Post alpha-4, we'll have to hunt down our downstreamers and help them
> > test
> > >> on top of alpha-4 so rolling into beta-1, we have confidence our
> > >> downstreamers know what to expect (or we discover what we missed
> BEFORE
> > we
> > >> beta-1).
> > >>
> > >> Thanks for time,
> > >> S
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Fri, Sep 8, 2017 at 2:04 PM, Stack  wrote:
> > >>
> > >> I'll put up an alpha3 RC Monday, probably Monday night. That should be
> > >>> time, if we all sprint, for the public-facing API fixes to be done.
> > >>>
> > >>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3
> > but
> > >>> it is plain that more time is needed (in spite of valiant effort so
> far
> > >>> by
> > >>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
> > theme
> > >>> is
> > >>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
> > following
> > >>> week.
> > >>>
> > >>> We should then be ready for beta (beta == no new features, no API
> > >>> changes,
> > >>> just fixes).
> > >>>
> > >>> Thanks,
> > >>> St.Ack
> > >>>
> > >>>
> > >>> On Thu, Aug 17, 2017 at 12:35 PM, Stack  wrote:
> > >>>
> > >>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.
> > 
> >  For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get
> a
> >  release out in the next week or so.
> > 
> >  I did a weeding of 2.0.0 issues over the last day. If folks are
> >  interested in helping out, below are the items I think we need done
> > for
> >  alpha3 (below are at least 'Critical' status, are API possibly
> > altering
> >  items, and are absent those JIRAs that are making active progress,
> > i.e.
> >  the
> >  HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
> >  doing is
> >  what Andrew did comparing 1.3. and 1.4 APIs
> > 
> >  * HBASE-18622 Mitigate compatibility concerns between branch-1 and
> >  branch-2
> >  This is to do what Andrew did between 1.3 and 1.4 branches only do
> it
> >  between branch-1 and branch-2.
> > 
> >  * HBASE-10462 Recategorize some of the client facing Public /
> Private
> >  interfaces
> >  This one is almost done. It could do with a finish, attention to the
> >  items in last comment, and then our codebase could do with another
> > sweep
> >  after the spirit of this issue since a bunch has gone in since the
> > pass
> >  that was the basis of this issue.
> > 
> >  * HBASE-10504 Define Replication Interface
> >  I was going to take a crack at this as part of the revamp forced by
> >  'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service'
> > but
> >  if
> >  anyone else is interested, be my guest.
> > 
> >  * HBASE-14996 Some more API cleanup for 2.0
> >  Has a bunc

Re: Moving 2.0 forward

[ramkrishna vasudevan]

Hi Stack

Do you want HBASE-18995 before the alpha-4 (REmoving exposed internal APIs
from CellUtil)? Because you had mentioned no more API changes. If so I will
start making changes and put up a patch ASAP.

Regards
Ram

On Tue, Oct 24, 2017 at 3:22 AM, Stack  wrote:

> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser  wrote:
>
> > +1
> >
> > I was trying to work on helping out on the outstanding alpha-4 stuff last
> > week -- will be continuing to try to do the same this week.
> >
> > If you need any help, Stack, or if others need reviews where I haven't
> > noticed on my own: feel free to @mention me.
> >
> >
> Thanks for the offer Josh. All items seem assigned and are being actively
> worked on. If you get a moment, reviews by you (or anyone else) helps move
> the process along.
>
> We need to merge in HBASE-18410 branch to pick up Filter improvements. Then
> HBASE-13346 can go in.
>
> You are already helping out on HBASE-18906, thanks. Looks like that will be
> addressed by other alpha-4s about to land.
>
> St.Ack
> TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
>
>
>
>
>
>
>
>
>
> > On 10/23/17 12:53 PM, Stack wrote:
> >
> >> (Reviving this thread)
> >>
> >> Lets push out alpha-4 this week. Alpha-4 is the release that has the
> >> refactor of the Coprocessor API shutting down access to internals marked
> >> InterfaceAudience.Private.
> >>
> >> The outstanding list is here:
> >> https://issues.apache.org/jira/projects/HBASE/versions/12341594
> >>
> >> Please push in anything marked alpha-4 that belongs to you.
> >>
> >> If issue, talk out loud on this thread. If you need a review to land an
> >> item, shout on the issue and here; we'll help you out.
> >>
> >> As is, items are coming along nicely I'd say. We need to merge the
> filter
> >> branch -- HBASE-18410 -- so APIs are finished for hbase2.
> >>
> >> Post alpha-4, we'll have to hunt down our downstreamers and help them
> test
> >> on top of alpha-4 so rolling into beta-1, we have confidence our
> >> downstreamers know what to expect (or we discover what we missed BEFORE
> we
> >> beta-1).
> >>
> >> Thanks for time,
> >> S
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Sep 8, 2017 at 2:04 PM, Stack  wrote:
> >>
> >> I'll put up an alpha3 RC Monday, probably Monday night. That should be
> >>> time, if we all sprint, for the public-facing API fixes to be done.
> >>>
> >>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3
> but
> >>> it is plain that more time is needed (in spite of valiant effort so far
> >>> by
> >>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
> theme
> >>> is
> >>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
> following
> >>> week.
> >>>
> >>> We should then be ready for beta (beta == no new features, no API
> >>> changes,
> >>> just fixes).
> >>>
> >>> Thanks,
> >>> St.Ack
> >>>
> >>>
> >>> On Thu, Aug 17, 2017 at 12:35 PM, Stack  wrote:
> >>>
> >>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.
> 
>  For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get a
>  release out in the next week or so.
> 
>  I did a weeding of 2.0.0 issues over the last day. If folks are
>  interested in helping out, below are the items I think we need done
> for
>  alpha3 (below are at least 'Critical' status, are API possibly
> altering
>  items, and are absent those JIRAs that are making active progress,
> i.e.
>  the
>  HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
>  doing is
>  what Andrew did comparing 1.3. and 1.4 APIs
> 
>  * HBASE-18622 Mitigate compatibility concerns between branch-1 and
>  branch-2
>  This is to do what Andrew did between 1.3 and 1.4 branches only do it
>  between branch-1 and branch-2.
> 
>  * HBASE-10462 Recategorize some of the client facing Public / Private
>  interfaces
>  This one is almost done. It could do with a finish, attention to the
>  items in last comment, and then our codebase could do with another
> sweep
>  after the spirit of this issue since a bunch has gone in since the
> pass
>  that was the basis of this issue.
> 
>  * HBASE-10504 Define Replication Interface
>  I was going to take a crack at this as part of the revamp forced by
>  'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service'
> but
>  if
>  anyone else is interested, be my guest.
> 
>  * HBASE-14996 Some more API cleanup for 2.0
>  Has a bunch of subtasks, some of which are being worked on. Needs
>  finishing.
> 
>  * HBASE-14998 Unify synchronous and asynchronous methods in Admin and
>  cleanup
>  Needs a pass. Small issue I think. Could also look at new AsyncClient
>  and
>  make sure symmetry.
> 
>  * HBASE-15607 Remove PB references from Admin for 2.0
>  Predicated on result of an ongoing DISCUSSION thread but needs to be
>  do

[jira] [Created] (HBASE-19076) Ensure findbugs jsr305 jar isn't present in hbase-error-prone module

[Qilin Cao (JIRA)]

Qilin Cao created HBASE-19076:
-

 Summary: Ensure findbugs jsr305 jar isn't present in 
hbase-error-prone module
 Key: HBASE-19076
 URL: https://issues.apache.org/jira/browse/HBASE-19076
 Project: HBase
  Issue Type: Bug
  Components: dependencies
Affects Versions: 3.0.0
Reporter: Qilin Cao
Assignee: Qilin Cao
Priority: Blocker


After HBASE-16321 ensure findbugs jsr305 jar isn't present, we have failures 
with the hbase-error-prone module.

{panel:title=My title}
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hbase-error-prone ---
[INFO] Deleting 
/home/caoqilin/Codebase/github/hbase/hbase-build-support/hbase-error-prone/target
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce 
(min-maven-min-java-banned-xerces) @ hbase-error-prone ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (banned-jsr305) @ 
hbase-error-prone ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
with message:
We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321.
Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
{panel}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19075) Task tabs on master UI cause page scroll

[Mike Drob (JIRA)]

Mike Drob created HBASE-19075:
-

 Summary: Task tabs on master UI cause page scroll
 Key: HBASE-19075
 URL: https://issues.apache.org/jira/browse/HBASE-19075
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Mike Drob


On the master info page, the clicking the tabs under Tasks causes the page to 
scroll back to the top of the page.

{noformat}
Tasks
Show All Monitored Tasks Show non-RPC Tasks Show All RPC Handler Tasks Show 
Active RPC Calls Show Client Operations View as JSON
{noformat}

^^ Any of those



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Allan Yang]

Congratulations!!!

Best Regards
Allan Yang

2017-10-24 3:45 GMT+08:00 Umesh Agashe :

> Congratulations, Zheng!
>
> On Mon, Oct 23, 2017 at 11:58 AM, Josh Elser  wrote:
>
> > Congrats!
> >
> >
> > On 10/23/17 2:18 AM, Duo Zhang wrote:
> >
> >> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng
> Hu
> >> has accepted the PMC's invitation to become a committer on the project.
> We
> >> appreciate all of Zheng's generous contributions thus far and look
> forward
> >> to his continued involvement.
> >>
> >> Congratulations and welcome, Zheng!
> >>
> >>
>

[jira] [Created] (HBASE-19074) Miscellaneous Observer cleanups

[stack (JIRA)]

stack created HBASE-19074:
-

 Summary: Miscellaneous Observer cleanups
 Key: HBASE-19074
 URL: https://issues.apache.org/jira/browse/HBASE-19074
 Project: HBase
  Issue Type: Sub-task
  Components: Coprocessors
Reporter: stack
Assignee: stack
 Fix For: 2.0.0-alpha-4


Going through Observers after fixing up MasterObserver, i see a few violations 
such as Store returning a MemStoreSize instance (which would let coprocessors 
inc/dec MemStore size which would mess us up). This issue is about cleaning 
these remainders up.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Fate of CoordinatedStateManager and hbase.coordinated.state.manager.class config

[Apekshit Sharma]

https://issues.apache.org/jira/browse/HBASE-19073
Has attached patch and link to review board too.
Thanks!

Cheers,
Appy

On Mon, Oct 23, 2017 at 2:56 PM, Mikhail Antonov 
wrote:

> Sounds good to me.
>
> This effort [to make coordination engine that HBase uses pluggable] was
> started and work was done towards making dependency of ZK separated into
> the set of defined interfaces, but subsequent phases never were completed.
>
> The proposed cleanup sounds good to me. Thanks for taking that up!
>
> -Mikhail
>
> On Mon, Oct 23, 2017 at 1:15 PM, Andrew Purtell 
> wrote:
>
> > Sounds good to me.
> >
> >
> > On Mon, Oct 23, 2017 at 1:00 PM, Apekshit Sharma 
> > wrote:
> >
> > > Thanks Andrew!
> > >
> > > How does the following cleanup sounds?
> > >
> > > - Remove the configuration hbase.coordinated.state.manager.class
> > > - Keep following interface since they nicely separate ZK based
> > > implementation: SplitLogWorkerCoordination,
> SplitLogManagerCoordination,
> > > ProcedureCoordinatorRpcs, ProcedureMemberRpcs
> > > - Replace CSM (interface) + BCSM (unnecessary middle hierarchy) with
> > single
> > > CSM interface.
> > >
> > > @InterfaceAudience.Private
> > > public interface CoordinatedStateManager {
> > >   void initialize(Server server)
> > >   SplitLogWorkerCoordination getSplitLogWorkerCoordination();
> > >   SplitLogManagerCoordination getSplitLogManagerCoordination();
> > >   ProcedureCoordinatorRpcs getProcedureCoordinatorRpcs(String
> procType,
> > > String coordNode) throws IOException;
> > >   ProcedureMemberRpcs getProcedureMemberRpcs(String procType) throws
> > > KeeperException;
> > > }
> > >
> > > On Mon, Oct 23, 2017 at 12:37 PM, Andrew Purtell 
> > > wrote:
> > >
> > > > This work was done by a small development group at WANDisco. The
> intent
> > > was
> > > > to allow substitution of a proprietary consensus and state
> replication
> > > > service for ZooKeeper. There hasn't been progress on this for a long
> > > time.
> > > > I think it is safe for us to clean it up. No need necessarily to
> > abandon
> > > > the idea of abstraction between our coordination needs and the
> > > > implementation.
> > > >
> > > >
> > > > On Mon, Oct 23, 2017 at 12:16 PM, Apekshit Sharma  >
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > Am coming from limited knowledge here, so pardon me if it seems
> > > > outrageous.
> > > > > I guess this effort (HBASE-10909
> > > > > ) was to
> separate
> > > out
> > > > > state into an interface which was then made pluggable via the
> config
> > > > > hbase.coordinated.state.manager.class.
> > > > >
> > > > > - Is this effort complete? Can someone use it to completely switch
> > out
> > > ZK
> > > > > based state with something else? I see all tasks in HBASE-10909
> > > > >  are complete,
> > but
> > > > it's
> > > > > named 'phase1' and i don't see a phase2.
> > > > >
> > > > > - Is anyone aware of any use cases where it's actually being used
> to
> > > > > replace zk?
> > > > >
> > > > > ** If yes, I think that at the very least, we should clean it up
> > (more
> > > on
> > > > > it further down) and made these relevant interfaced IA.Public.
> > > > >
> > > > > ** If not, can we get rid of the (incomplete??) 'feature' and do
> more
> > > > > rigorous cleanup? I'll sign up for it.
> > > > > -
> > > > >
> > > > > Cleanup:
> > > > > Our internal class hierarchy is:
> > > > > CoordinatedStateManager -> BaseCoordinatedStateManager ->
> > > > > ZkCoordinatedStateManager.
> > > > >
> > > > > - We carry around CSM objects but cast them to BCSM in so many
> > places!
> > > If
> > > > > anyone implements CSM and plugs it in, it won't work. Better to
> just
> > > > unify
> > > > > them and make it easier to understand.
> > > > >
> > > > >
> > > > > -- Appy
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Words like orphans lost among the crosstalk, meaning torn from
> truth's
> > > > decrepit hands
> > > >- A23, Crosstalk
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > -- Appy
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>



-- 

-- Appy

[jira] [Created] (HBASE-19073) Cleanup CoordinatedStateManager

[Appy (JIRA)]

Appy created HBASE-19073:


 Summary: Cleanup CoordinatedStateManager
 Key: HBASE-19073
 URL: https://issues.apache.org/jira/browse/HBASE-19073
 Project: HBase
  Issue Type: Bug
Reporter: Appy
Assignee: Appy


Discussion thread on dev@ mailing list.
http://mail-archives.apache.org/mod_mbox/hbase-dev/201710.mbox/%3CCAAjhxrqjOg90Fdi73kZZe_Gxtrqq8ff%2B%3DAj_epptO_XO812Abg%40mail.gmail.com%3E



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Fate of CoordinatedStateManager and hbase.coordinated.state.manager.class config

[Mikhail Antonov]

Sounds good to me.

This effort [to make coordination engine that HBase uses pluggable] was
started and work was done towards making dependency of ZK separated into
the set of defined interfaces, but subsequent phases never were completed.

The proposed cleanup sounds good to me. Thanks for taking that up!

-Mikhail

On Mon, Oct 23, 2017 at 1:15 PM, Andrew Purtell  wrote:

> Sounds good to me.
>
>
> On Mon, Oct 23, 2017 at 1:00 PM, Apekshit Sharma 
> wrote:
>
> > Thanks Andrew!
> >
> > How does the following cleanup sounds?
> >
> > - Remove the configuration hbase.coordinated.state.manager.class
> > - Keep following interface since they nicely separate ZK based
> > implementation: SplitLogWorkerCoordination, SplitLogManagerCoordination,
> > ProcedureCoordinatorRpcs, ProcedureMemberRpcs
> > - Replace CSM (interface) + BCSM (unnecessary middle hierarchy) with
> single
> > CSM interface.
> >
> > @InterfaceAudience.Private
> > public interface CoordinatedStateManager {
> >   void initialize(Server server)
> >   SplitLogWorkerCoordination getSplitLogWorkerCoordination();
> >   SplitLogManagerCoordination getSplitLogManagerCoordination();
> >   ProcedureCoordinatorRpcs getProcedureCoordinatorRpcs(String procType,
> > String coordNode) throws IOException;
> >   ProcedureMemberRpcs getProcedureMemberRpcs(String procType) throws
> > KeeperException;
> > }
> >
> > On Mon, Oct 23, 2017 at 12:37 PM, Andrew Purtell 
> > wrote:
> >
> > > This work was done by a small development group at WANDisco. The intent
> > was
> > > to allow substitution of a proprietary consensus and state replication
> > > service for ZooKeeper. There hasn't been progress on this for a long
> > time.
> > > I think it is safe for us to clean it up. No need necessarily to
> abandon
> > > the idea of abstraction between our coordination needs and the
> > > implementation.
> > >
> > >
> > > On Mon, Oct 23, 2017 at 12:16 PM, Apekshit Sharma 
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > Am coming from limited knowledge here, so pardon me if it seems
> > > outrageous.
> > > > I guess this effort (HBASE-10909
> > > > ) was to separate
> > out
> > > > state into an interface which was then made pluggable via the config
> > > > hbase.coordinated.state.manager.class.
> > > >
> > > > - Is this effort complete? Can someone use it to completely switch
> out
> > ZK
> > > > based state with something else? I see all tasks in HBASE-10909
> > > >  are complete,
> but
> > > it's
> > > > named 'phase1' and i don't see a phase2.
> > > >
> > > > - Is anyone aware of any use cases where it's actually being used to
> > > > replace zk?
> > > >
> > > > ** If yes, I think that at the very least, we should clean it up
> (more
> > on
> > > > it further down) and made these relevant interfaced IA.Public.
> > > >
> > > > ** If not, can we get rid of the (incomplete??) 'feature' and do more
> > > > rigorous cleanup? I'll sign up for it.
> > > > -
> > > >
> > > > Cleanup:
> > > > Our internal class hierarchy is:
> > > > CoordinatedStateManager -> BaseCoordinatedStateManager ->
> > > > ZkCoordinatedStateManager.
> > > >
> > > > - We carry around CSM objects but cast them to BCSM in so many
> places!
> > If
> > > > anyone implements CSM and plugs it in, it won't work. Better to just
> > > unify
> > > > them and make it easier to understand.
> > > >
> > > >
> > > > -- Appy
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >- A23, Crosstalk
> > >
> >
> >
> >
> > --
> >
> > -- Appy
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>



-- 
Thanks,
Michael Antonov

Re: Moving 2.0 forward

[Stack]

On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser  wrote:

> +1
>
> I was trying to work on helping out on the outstanding alpha-4 stuff last
> week -- will be continuing to try to do the same this week.
>
> If you need any help, Stack, or if others need reviews where I haven't
> noticed on my own: feel free to @mention me.
>
>
Thanks for the offer Josh. All items seem assigned and are being actively
worked on. If you get a moment, reviews by you (or anyone else) helps move
the process along.

We need to merge in HBASE-18410 branch to pick up Filter improvements. Then
HBASE-13346 can go in.

You are already helping out on HBASE-18906, thanks. Looks like that will be
addressed by other alpha-4s about to land.

St.Ack
TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594









> On 10/23/17 12:53 PM, Stack wrote:
>
>> (Reviving this thread)
>>
>> Lets push out alpha-4 this week. Alpha-4 is the release that has the
>> refactor of the Coprocessor API shutting down access to internals marked
>> InterfaceAudience.Private.
>>
>> The outstanding list is here:
>> https://issues.apache.org/jira/projects/HBASE/versions/12341594
>>
>> Please push in anything marked alpha-4 that belongs to you.
>>
>> If issue, talk out loud on this thread. If you need a review to land an
>> item, shout on the issue and here; we'll help you out.
>>
>> As is, items are coming along nicely I'd say. We need to merge the filter
>> branch -- HBASE-18410 -- so APIs are finished for hbase2.
>>
>> Post alpha-4, we'll have to hunt down our downstreamers and help them test
>> on top of alpha-4 so rolling into beta-1, we have confidence our
>> downstreamers know what to expect (or we discover what we missed BEFORE we
>> beta-1).
>>
>> Thanks for time,
>> S
>>
>>
>>
>>
>>
>> On Fri, Sep 8, 2017 at 2:04 PM, Stack  wrote:
>>
>> I'll put up an alpha3 RC Monday, probably Monday night. That should be
>>> time, if we all sprint, for the public-facing API fixes to be done.
>>>
>>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 but
>>> it is plain that more time is needed (in spite of valiant effort so far
>>> by
>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose theme
>>> is
>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the following
>>> week.
>>>
>>> We should then be ready for beta (beta == no new features, no API
>>> changes,
>>> just fixes).
>>>
>>> Thanks,
>>> St.Ack
>>>
>>>
>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack  wrote:
>>>
>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.

 For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get a
 release out in the next week or so.

 I did a weeding of 2.0.0 issues over the last day. If folks are
 interested in helping out, below are the items I think we need done for
 alpha3 (below are at least 'Critical' status, are API possibly altering
 items, and are absent those JIRAs that are making active progress, i.e.
 the
 HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
 doing is
 what Andrew did comparing 1.3. and 1.4 APIs

 * HBASE-18622 Mitigate compatibility concerns between branch-1 and
 branch-2
 This is to do what Andrew did between 1.3 and 1.4 branches only do it
 between branch-1 and branch-2.

 * HBASE-10462 Recategorize some of the client facing Public / Private
 interfaces
 This one is almost done. It could do with a finish, attention to the
 items in last comment, and then our codebase could do with another sweep
 after the spirit of this issue since a bunch has gone in since the pass
 that was the basis of this issue.

 * HBASE-10504 Define Replication Interface
 I was going to take a crack at this as part of the revamp forced by
 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service' but
 if
 anyone else is interested, be my guest.

 * HBASE-14996 Some more API cleanup for 2.0
 Has a bunch of subtasks, some of which are being worked on. Needs
 finishing.

 * HBASE-14998 Unify synchronous and asynchronous methods in Admin and
 cleanup
 Needs a pass. Small issue I think. Could also look at new AsyncClient
 and
 make sure symmetry.

 * HBASE-15607 Remove PB references from Admin for 2.0
 Predicated on result of an ongoing DISCUSSION thread but needs to be
 done.

 Rolling upgrade will have implications for our API. Would be good to try
 it and figure what needs fixup (as said above, according to trial by
 Sean,
 we might not be too bad here):
 * HBASE-16060 1.x clients cannot access table state talking to 2.0
 cluster
 * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master and 1.x
 RSs; i.e. support Rolling Upgrade from hbase-1 to -2.

 * HBASE-17442 Move most of the replication related classes to
 hbase-server package
 The above

Re: [DISCUSS] Fate of CoordinatedStateManager and hbase.coordinated.state.manager.class config

[Andrew Purtell]

Sounds good to me.


On Mon, Oct 23, 2017 at 1:00 PM, Apekshit Sharma  wrote:

> Thanks Andrew!
>
> How does the following cleanup sounds?
>
> - Remove the configuration hbase.coordinated.state.manager.class
> - Keep following interface since they nicely separate ZK based
> implementation: SplitLogWorkerCoordination, SplitLogManagerCoordination,
> ProcedureCoordinatorRpcs, ProcedureMemberRpcs
> - Replace CSM (interface) + BCSM (unnecessary middle hierarchy) with single
> CSM interface.
>
> @InterfaceAudience.Private
> public interface CoordinatedStateManager {
>   void initialize(Server server)
>   SplitLogWorkerCoordination getSplitLogWorkerCoordination();
>   SplitLogManagerCoordination getSplitLogManagerCoordination();
>   ProcedureCoordinatorRpcs getProcedureCoordinatorRpcs(String procType,
> String coordNode) throws IOException;
>   ProcedureMemberRpcs getProcedureMemberRpcs(String procType) throws
> KeeperException;
> }
>
> On Mon, Oct 23, 2017 at 12:37 PM, Andrew Purtell 
> wrote:
>
> > This work was done by a small development group at WANDisco. The intent
> was
> > to allow substitution of a proprietary consensus and state replication
> > service for ZooKeeper. There hasn't been progress on this for a long
> time.
> > I think it is safe for us to clean it up. No need necessarily to abandon
> > the idea of abstraction between our coordination needs and the
> > implementation.
> >
> >
> > On Mon, Oct 23, 2017 at 12:16 PM, Apekshit Sharma 
> > wrote:
> >
> > > Hi everyone,
> > >
> > > Am coming from limited knowledge here, so pardon me if it seems
> > outrageous.
> > > I guess this effort (HBASE-10909
> > > ) was to separate
> out
> > > state into an interface which was then made pluggable via the config
> > > hbase.coordinated.state.manager.class.
> > >
> > > - Is this effort complete? Can someone use it to completely switch out
> ZK
> > > based state with something else? I see all tasks in HBASE-10909
> > >  are complete, but
> > it's
> > > named 'phase1' and i don't see a phase2.
> > >
> > > - Is anyone aware of any use cases where it's actually being used to
> > > replace zk?
> > >
> > > ** If yes, I think that at the very least, we should clean it up (more
> on
> > > it further down) and made these relevant interfaced IA.Public.
> > >
> > > ** If not, can we get rid of the (incomplete??) 'feature' and do more
> > > rigorous cleanup? I'll sign up for it.
> > > -
> > >
> > > Cleanup:
> > > Our internal class hierarchy is:
> > > CoordinatedStateManager -> BaseCoordinatedStateManager ->
> > > ZkCoordinatedStateManager.
> > >
> > > - We carry around CSM objects but cast them to BCSM in so many places!
> If
> > > anyone implements CSM and plugs it in, it won't work. Better to just
> > unify
> > > them and make it easier to understand.
> > >
> > >
> > > -- Appy
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>
>
>
> --
>
> -- Appy
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Fate of CoordinatedStateManager and hbase.coordinated.state.manager.class config

[Apekshit Sharma]

Thanks Andrew!

How does the following cleanup sounds?

- Remove the configuration hbase.coordinated.state.manager.class
- Keep following interface since they nicely separate ZK based
implementation: SplitLogWorkerCoordination, SplitLogManagerCoordination,
ProcedureCoordinatorRpcs, ProcedureMemberRpcs
- Replace CSM (interface) + BCSM (unnecessary middle hierarchy) with single
CSM interface.

@InterfaceAudience.Private
public interface CoordinatedStateManager {
  void initialize(Server server)
  SplitLogWorkerCoordination getSplitLogWorkerCoordination();
  SplitLogManagerCoordination getSplitLogManagerCoordination();
  ProcedureCoordinatorRpcs getProcedureCoordinatorRpcs(String procType,
String coordNode) throws IOException;
  ProcedureMemberRpcs getProcedureMemberRpcs(String procType) throws
KeeperException;
}

On Mon, Oct 23, 2017 at 12:37 PM, Andrew Purtell 
wrote:

> This work was done by a small development group at WANDisco. The intent was
> to allow substitution of a proprietary consensus and state replication
> service for ZooKeeper. There hasn't been progress on this for a long time.
> I think it is safe for us to clean it up. No need necessarily to abandon
> the idea of abstraction between our coordination needs and the
> implementation.
>
>
> On Mon, Oct 23, 2017 at 12:16 PM, Apekshit Sharma 
> wrote:
>
> > Hi everyone,
> >
> > Am coming from limited knowledge here, so pardon me if it seems
> outrageous.
> > I guess this effort (HBASE-10909
> > ) was to separate out
> > state into an interface which was then made pluggable via the config
> > hbase.coordinated.state.manager.class.
> >
> > - Is this effort complete? Can someone use it to completely switch out ZK
> > based state with something else? I see all tasks in HBASE-10909
> >  are complete, but
> it's
> > named 'phase1' and i don't see a phase2.
> >
> > - Is anyone aware of any use cases where it's actually being used to
> > replace zk?
> >
> > ** If yes, I think that at the very least, we should clean it up (more on
> > it further down) and made these relevant interfaced IA.Public.
> >
> > ** If not, can we get rid of the (incomplete??) 'feature' and do more
> > rigorous cleanup? I'll sign up for it.
> > -
> >
> > Cleanup:
> > Our internal class hierarchy is:
> > CoordinatedStateManager -> BaseCoordinatedStateManager ->
> > ZkCoordinatedStateManager.
> >
> > - We carry around CSM objects but cast them to BCSM in so many places! If
> > anyone implements CSM and plugs it in, it won't work. Better to just
> unify
> > them and make it easier to understand.
> >
> >
> > -- Appy
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>



-- 

-- Appy

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Umesh Agashe]

Congratulations, Zheng!

On Mon, Oct 23, 2017 at 11:58 AM, Josh Elser  wrote:

> Congrats!
>
>
> On 10/23/17 2:18 AM, Duo Zhang wrote:
>
>> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
>> has accepted the PMC's invitation to become a committer on the project. We
>> appreciate all of Zheng's generous contributions thus far and look forward
>> to his continued involvement.
>>
>> Congratulations and welcome, Zheng!
>>
>>

[jira] [Created] (HBASE-19072) Missing beak in catch block of InterruptedException in HRegion#waitForFlushes()

[Ted Yu (JIRA)]

Ted Yu created HBASE-19072:
--

 Summary: Missing beak in catch block of InterruptedException in 
HRegion#waitForFlushes() 
 Key: HBASE-19072
 URL: https://issues.apache.org/jira/browse/HBASE-19072
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu


During review of HBASE-18358, [~elserj] found that there was missing break in 
the catch block:
{code}
  } catch (InterruptedException iex) {
// essentially ignore and propagate the interrupt back up
LOG.warn("Interrupted while waiting");
interrupted = true;
  }
{code}
When thread is interrupted, we shouldn't retry flushing anymore.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Fate of CoordinatedStateManager and hbase.coordinated.state.manager.class config

[Andrew Purtell]

This work was done by a small development group at WANDisco. The intent was
to allow substitution of a proprietary consensus and state replication
service for ZooKeeper. There hasn't been progress on this for a long time.
I think it is safe for us to clean it up. No need necessarily to abandon
the idea of abstraction between our coordination needs and the
implementation.


On Mon, Oct 23, 2017 at 12:16 PM, Apekshit Sharma  wrote:

> Hi everyone,
>
> Am coming from limited knowledge here, so pardon me if it seems outrageous.
> I guess this effort (HBASE-10909
> ) was to separate out
> state into an interface which was then made pluggable via the config
> hbase.coordinated.state.manager.class.
>
> - Is this effort complete? Can someone use it to completely switch out ZK
> based state with something else? I see all tasks in HBASE-10909
>  are complete, but it's
> named 'phase1' and i don't see a phase2.
>
> - Is anyone aware of any use cases where it's actually being used to
> replace zk?
>
> ** If yes, I think that at the very least, we should clean it up (more on
> it further down) and made these relevant interfaced IA.Public.
>
> ** If not, can we get rid of the (incomplete??) 'feature' and do more
> rigorous cleanup? I'll sign up for it.
> -
>
> Cleanup:
> Our internal class hierarchy is:
> CoordinatedStateManager -> BaseCoordinatedStateManager ->
> ZkCoordinatedStateManager.
>
> - We carry around CSM objects but cast them to BCSM in so many places! If
> anyone implements CSM and plugs it in, it won't work. Better to just unify
> them and make it easier to understand.
>
>
> -- Appy
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

[DISCUSS] Fate of CoordinatedStateManager and hbase.coordinated.state.manager.class config

[Apekshit Sharma]

Hi everyone,

Am coming from limited knowledge here, so pardon me if it seems outrageous.
I guess this effort (HBASE-10909
) was to separate out
state into an interface which was then made pluggable via the config
hbase.coordinated.state.manager.class.

- Is this effort complete? Can someone use it to completely switch out ZK
based state with something else? I see all tasks in HBASE-10909
 are complete, but it's
named 'phase1' and i don't see a phase2.

- Is anyone aware of any use cases where it's actually being used to
replace zk?

** If yes, I think that at the very least, we should clean it up (more on
it further down) and made these relevant interfaced IA.Public.

** If not, can we get rid of the (incomplete??) 'feature' and do more
rigorous cleanup? I'll sign up for it.
-

Cleanup:
Our internal class hierarchy is:
CoordinatedStateManager -> BaseCoordinatedStateManager ->
ZkCoordinatedStateManager.

- We carry around CSM objects but cast them to BCSM in so many places! If
anyone implements CSM and plugs it in, it won't work. Better to just unify
them and make it easier to understand.


-- Appy

Re: PreCommit whitespace failures on JVM crash dump files

[Josh Elser]

Thanks, Sean & Mike! Glad I'm not the only one who has seen this :)

Re .gitignore: not that I can tell (touch'ing an 
hbase-server/hs_err_pid30492.log shows it up as untracked in my local 
checkout).


On 10/23/17 12:12 PM, Sean Busbey wrote:

we use git clean I think to do the post-checkout cleanup. are the dump
files in our git ignore?

On Mon, Oct 23, 2017 at 10:48 AM, Mike Drob  wrote:

I've seen this too, but I don't understand how the dumps are present from
what should notionally be a clean checkout.

On Mon, Oct 23, 2017 at 10:45 AM, Josh Elser  wrote:


Hiya,

Noticed a PreCommit failure from over the weekend on the whitespace (EOL,
tabs) checks [1][2]:

e.g.

```
./hbase-server/hs_err_pid30492.log:637:73100-734f0 rw-p 
00:00 0
./hbase-server/hs_err_pid30492.log:638:734f0-75100 rw-p 
00:00 0
```

Any thoughts on the right way to fix this one?

The whitespace check ran before the tests did, so I can only assume these
were laying around on the filesystem before the PreCommit job itself ran.
Do we need to be a bit more aggressive in the clean steps?

I do see in the Jenkinsfile, we are taking advantage of the options in
Yetus to ignore certain files for whitespace. That would be another option
to "fix" this.

- Josh

[1] https://builds.apache.org/job/PreCommit-HBASE-Build/9309/art
ifact/patchprocess/whitespace-eol.txt
[2] https://builds.apache.org/job/PreCommit-HBASE-Build/9309/art
ifact/patchprocess/whitespace-tabs.txt

Re: Moving 2.0 forward

[Josh Elser]

+1

I was trying to work on helping out on the outstanding alpha-4 stuff 
last week -- will be continuing to try to do the same this week.


If you need any help, Stack, or if others need reviews where I haven't 
noticed on my own: feel free to @mention me.


On 10/23/17 12:53 PM, Stack wrote:

(Reviving this thread)

Lets push out alpha-4 this week. Alpha-4 is the release that has the
refactor of the Coprocessor API shutting down access to internals marked
InterfaceAudience.Private.

The outstanding list is here:
https://issues.apache.org/jira/projects/HBASE/versions/12341594

Please push in anything marked alpha-4 that belongs to you.

If issue, talk out loud on this thread. If you need a review to land an
item, shout on the issue and here; we'll help you out.

As is, items are coming along nicely I'd say. We need to merge the filter
branch -- HBASE-18410 -- so APIs are finished for hbase2.

Post alpha-4, we'll have to hunt down our downstreamers and help them test
on top of alpha-4 so rolling into beta-1, we have confidence our
downstreamers know what to expect (or we discover what we missed BEFORE we
beta-1).

Thanks for time,
S





On Fri, Sep 8, 2017 at 2:04 PM, Stack  wrote:


I'll put up an alpha3 RC Monday, probably Monday night. That should be
time, if we all sprint, for the public-facing API fixes to be done.

I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 but
it is plain that more time is needed (in spite of valiant effort so far by
Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose theme is
"Coprocessor Fixup". Hopefully we can put an alpha-4 up by the following
week.

We should then be ready for beta (beta == no new features, no API changes,
just fixes).

Thanks,
St.Ack


On Thu, Aug 17, 2017 at 12:35 PM, Stack  wrote:


I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.

For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get a
release out in the next week or so.

I did a weeding of 2.0.0 issues over the last day. If folks are
interested in helping out, below are the items I think we need done for
alpha3 (below are at least 'Critical' status, are API possibly altering
items, and are absent those JIRAs that are making active progress, i.e. the
HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs doing is
what Andrew did comparing 1.3. and 1.4 APIs

* HBASE-18622 Mitigate compatibility concerns between branch-1 and
branch-2
This is to do what Andrew did between 1.3 and 1.4 branches only do it
between branch-1 and branch-2.

* HBASE-10462 Recategorize some of the client facing Public / Private
interfaces
This one is almost done. It could do with a finish, attention to the
items in last comment, and then our codebase could do with another sweep
after the spirit of this issue since a bunch has gone in since the pass
that was the basis of this issue.

* HBASE-10504 Define Replication Interface
I was going to take a crack at this as part of the revamp forced by
'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service' but if
anyone else is interested, be my guest.

* HBASE-14996 Some more API cleanup for 2.0
Has a bunch of subtasks, some of which are being worked on. Needs
finishing.

* HBASE-14998 Unify synchronous and asynchronous methods in Admin and
cleanup
Needs a pass. Small issue I think. Could also look at new AsyncClient and
make sure symmetry.

* HBASE-15607 Remove PB references from Admin for 2.0
Predicated on result of an ongoing DISCUSSION thread but needs to be done.

Rolling upgrade will have implications for our API. Would be good to try
it and figure what needs fixup (as said above, according to trial by Sean,
we might not be too bad here):
* HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
* HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master and 1.x
RSs; i.e. support Rolling Upgrade from hbase-1 to -2.

* HBASE-17442 Move most of the replication related classes to
hbase-server package
The above would be good to do generally but it may make for ripples in
API so would be good to do now.

* HBASE-18106 Redo ProcedureInfo and LockInfo
Balazs is working on this. The idea is that we avoid adding two new types
to our API, two types that are nought but curtailed, read-only views on
internals. Input if you have time appreciated.

* HBASE-18596 A hbase1 cluster should be able to replicate to a hbase2
cluster; verify
Esteban is looking at this one

* HBASE-9417 SecureBulkLoadEndpoint should be folded in core
* HBASE-17143 Scan improvement

Our Coprocessor Interface needs a tough edit. It exposes implementations
marked audience Private and returns implementations rather than Interfaces.
In a few locations, we allow returning an alternate implementation
altogether which is probably something we don't want a CP doing. To that
end, the following issues started by Duo and Anoop need to be taken to the
finish line; ideally they'd have an owner:

* HBASE-18169 Coproces

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Josh Elser]

Congrats!

On 10/23/17 2:18 AM, Duo Zhang wrote:

On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
has accepted the PMC's invitation to become a committer on the project. We
appreciate all of Zheng's generous contributions thus far and look forward
to his continued involvement.

Congratulations and welcome, Zheng!

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Jerry He]

Congrats and welcome!


On Mon, Oct 23, 2017 at 10:29 AM, Huaxiang Sun  wrote:
> Congratulations, Zheng!
>

[jira] [Created] (HBASE-19071) Import from Hbase version 0.94.27 to higher version 1.2.1 not working

[Manjeet Singh (JIRA)]

Manjeet Singh created HBASE-19071:
-

 Summary: Import from Hbase version 0.94.27 to higher version 1.2.1 
not working 
 Key: HBASE-19071
 URL: https://issues.apache.org/jira/browse/HBASE-19071
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 1.2.1
Reporter: Manjeet Singh


Data migration from one cluster to another cluster in same N/W, but with a 
different version of hbase one is 0.94.27 (source cluster hbase) and another is 
destination cluster hbase version is 1.2.1. is not working.

I have used below command to take backup of hbase table on source cluster is:
 ./hbase org.apache.hadoop.hbase.mapreduce.Export SPDBRebuild /data/backupData/
and as a result below files were genrated by using above command:-
drwxr-xr-x 3 root root4096 Dec  9  2016 _logs
-rw-r--r-- 1 root root   788227695 Dec 16  2016 part-m-0
-rw-r--r-- 1 root root  1098757026 Dec 16  2016 part-m-1
-rw-r--r-- 1 root root   906973626 Dec 16  2016 part-m-2
-rw-r--r-- 1 root root  1981769314 Dec 16  2016 part-m-3
-rw-r--r-- 1 root root  2099785782 Dec 16  2016 part-m-4
-rw-r--r-- 1 root root  4118835540 Dec 16  2016 part-m-5
-rw-r--r-- 1 root root 14217981341 Dec 16  2016 part-m-6
-rw-r--r-- 1 root root   0 Dec 16  2016 _SUCCESS
 I import these file in another Hbase version (1.2.1)

in order to restore these files I am assuming I have to move these files in 
destination cluster and have to run below command 

hbase org.apache.hadoop.hbase.mapreduce.Import   


sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.Import test_table  
hdfs://:8020/data/ExportedFiles

I am getting below error

17/10/23 16:13:50 INFO mapreduce.Job: Task Id : 
attempt_1505781444745_0070_m_03_0, Status : FAILED
Error: java.io.IOException: keyvalues=NONE read 2 bytes, should read 121347
at 
org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2306)
at 
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:78)
at 
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
at 
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at 
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Huaxiang Sun]

Congratulations, Zheng!

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Mike Drob]

Welcome, Zheng!

On Mon, Oct 23, 2017 at 11:50 AM, Pankaj kr  wrote:

> Congratulations Zheng ..!!
>
> Regards,
> Pankaj
>
> -Original Message-
> From: Duo Zhang [mailto:zhang...@apache.org]
> Sent: Monday, October 23, 2017 2:19 PM
> To: dev@hbase.apache.org; hbase-user
> Subject: [ANNOUNCE] New HBase committer Zheng Hu
>
> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Zheng's generous contributions thus far and look forward
> to his continued involvement.
>
> Congratulations and welcome, Zheng!
>

Re: Moving 2.0 forward

[Stack]

(Reviving this thread)

Lets push out alpha-4 this week. Alpha-4 is the release that has the
refactor of the Coprocessor API shutting down access to internals marked
InterfaceAudience.Private.

The outstanding list is here:
https://issues.apache.org/jira/projects/HBASE/versions/12341594

Please push in anything marked alpha-4 that belongs to you.

If issue, talk out loud on this thread. If you need a review to land an
item, shout on the issue and here; we'll help you out.

As is, items are coming along nicely I'd say. We need to merge the filter
branch -- HBASE-18410 -- so APIs are finished for hbase2.

Post alpha-4, we'll have to hunt down our downstreamers and help them test
on top of alpha-4 so rolling into beta-1, we have confidence our
downstreamers know what to expect (or we discover what we missed BEFORE we
beta-1).

Thanks for time,
S





On Fri, Sep 8, 2017 at 2:04 PM, Stack  wrote:

> I'll put up an alpha3 RC Monday, probably Monday night. That should be
> time, if we all sprint, for the public-facing API fixes to be done.
>
> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 but
> it is plain that more time is needed (in spite of valiant effort so far by
> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose theme is
> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the following
> week.
>
> We should then be ready for beta (beta == no new features, no API changes,
> just fixes).
>
> Thanks,
> St.Ack
>
>
> On Thu, Aug 17, 2017 at 12:35 PM, Stack  wrote:
>
>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it.
>>
>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get a
>> release out in the next week or so.
>>
>> I did a weeding of 2.0.0 issues over the last day. If folks are
>> interested in helping out, below are the items I think we need done for
>> alpha3 (below are at least 'Critical' status, are API possibly altering
>> items, and are absent those JIRAs that are making active progress, i.e. the
>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs doing is
>> what Andrew did comparing 1.3. and 1.4 APIs
>>
>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and
>> branch-2
>> This is to do what Andrew did between 1.3 and 1.4 branches only do it
>> between branch-1 and branch-2.
>>
>> * HBASE-10462 Recategorize some of the client facing Public / Private
>> interfaces
>> This one is almost done. It could do with a finish, attention to the
>> items in last comment, and then our codebase could do with another sweep
>> after the spirit of this issue since a bunch has gone in since the pass
>> that was the basis of this issue.
>>
>> * HBASE-10504 Define Replication Interface
>> I was going to take a crack at this as part of the revamp forced by
>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service' but if
>> anyone else is interested, be my guest.
>>
>> * HBASE-14996 Some more API cleanup for 2.0
>> Has a bunch of subtasks, some of which are being worked on. Needs
>> finishing.
>>
>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin and
>> cleanup
>> Needs a pass. Small issue I think. Could also look at new AsyncClient and
>> make sure symmetry.
>>
>> * HBASE-15607 Remove PB references from Admin for 2.0
>> Predicated on result of an ongoing DISCUSSION thread but needs to be done.
>>
>> Rolling upgrade will have implications for our API. Would be good to try
>> it and figure what needs fixup (as said above, according to trial by Sean,
>> we might not be too bad here):
>> * HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master and 1.x
>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
>>
>> * HBASE-17442 Move most of the replication related classes to
>> hbase-server package
>> The above would be good to do generally but it may make for ripples in
>> API so would be good to do now.
>>
>> * HBASE-18106 Redo ProcedureInfo and LockInfo
>> Balazs is working on this. The idea is that we avoid adding two new types
>> to our API, two types that are nought but curtailed, read-only views on
>> internals. Input if you have time appreciated.
>>
>> * HBASE-18596 A hbase1 cluster should be able to replicate to a hbase2
>> cluster; verify
>> Esteban is looking at this one
>>
>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core
>> * HBASE-17143 Scan improvement
>>
>> Our Coprocessor Interface needs a tough edit. It exposes implementations
>> marked audience Private and returns implementations rather than Interfaces.
>> In a few locations, we allow returning an alternate implementation
>> altogether which is probably something we don't want a CP doing. To that
>> end, the following issues started by Duo and Anoop need to be taken to the
>> finish line; ideally they'd have an owner:
>>
>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <= The
>> umbrella issue

RE: [ANNOUNCE] New HBase committer Zheng Hu

[Pankaj kr]

Congratulations Zheng ..!! 

Regards,
Pankaj

-Original Message-
From: Duo Zhang [mailto:zhang...@apache.org] 
Sent: Monday, October 23, 2017 2:19 PM
To: dev@hbase.apache.org; hbase-user
Subject: [ANNOUNCE] New HBase committer Zheng Hu

On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu has 
accepted the PMC's invitation to become a committer on the project. We 
appreciate all of Zheng's generous contributions thus far and look forward to 
his continued involvement.

Congratulations and welcome, Zheng!

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Ashish Singhi]

Congratulations.

On Mon, Oct 23, 2017 at 11:48 AM, Duo Zhang  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Zheng's generous contributions thus far and look forward
> to his continued involvement.
>
> Congratulations and welcome, Zheng!
>

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Stack]

Welcome Zheng Hu!
S

On Sun, Oct 22, 2017 at 11:18 PM, Duo Zhang  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Zheng's generous contributions thus far and look forward
> to his continued involvement.
>
> Congratulations and welcome, Zheng!
>

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Andrew Purtell]

Congratulations and welcome!


> On Oct 22, 2017, at 11:18 PM, Duo Zhang  wrote:
> 
> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Zheng's generous contributions thus far and look forward
> to his continued involvement.
> 
> Congratulations and welcome, Zheng!

Re: PreCommit whitespace failures on JVM crash dump files

[Sean Busbey]

we use git clean I think to do the post-checkout cleanup. are the dump
files in our git ignore?

On Mon, Oct 23, 2017 at 10:48 AM, Mike Drob  wrote:
> I've seen this too, but I don't understand how the dumps are present from
> what should notionally be a clean checkout.
>
> On Mon, Oct 23, 2017 at 10:45 AM, Josh Elser  wrote:
>
>> Hiya,
>>
>> Noticed a PreCommit failure from over the weekend on the whitespace (EOL,
>> tabs) checks [1][2]:
>>
>> e.g.
>>
>> ```
>> ./hbase-server/hs_err_pid30492.log:637:73100-734f0 rw-p 
>> 00:00 0
>> ./hbase-server/hs_err_pid30492.log:638:734f0-75100 rw-p 
>> 00:00 0
>> ```
>>
>> Any thoughts on the right way to fix this one?
>>
>> The whitespace check ran before the tests did, so I can only assume these
>> were laying around on the filesystem before the PreCommit job itself ran.
>> Do we need to be a bit more aggressive in the clean steps?
>>
>> I do see in the Jenkinsfile, we are taking advantage of the options in
>> Yetus to ignore certain files for whitespace. That would be another option
>> to "fix" this.
>>
>> - Josh
>>
>> [1] https://builds.apache.org/job/PreCommit-HBASE-Build/9309/art
>> ifact/patchprocess/whitespace-eol.txt
>> [2] https://builds.apache.org/job/PreCommit-HBASE-Build/9309/art
>> ifact/patchprocess/whitespace-tabs.txt
>>

Re: PreCommit whitespace failures on JVM crash dump files

[Mike Drob]

I've seen this too, but I don't understand how the dumps are present from
what should notionally be a clean checkout.

On Mon, Oct 23, 2017 at 10:45 AM, Josh Elser  wrote:

> Hiya,
>
> Noticed a PreCommit failure from over the weekend on the whitespace (EOL,
> tabs) checks [1][2]:
>
> e.g.
>
> ```
> ./hbase-server/hs_err_pid30492.log:637:73100-734f0 rw-p 
> 00:00 0
> ./hbase-server/hs_err_pid30492.log:638:734f0-75100 rw-p 
> 00:00 0
> ```
>
> Any thoughts on the right way to fix this one?
>
> The whitespace check ran before the tests did, so I can only assume these
> were laying around on the filesystem before the PreCommit job itself ran.
> Do we need to be a bit more aggressive in the clean steps?
>
> I do see in the Jenkinsfile, we are taking advantage of the options in
> Yetus to ignore certain files for whitespace. That would be another option
> to "fix" this.
>
> - Josh
>
> [1] https://builds.apache.org/job/PreCommit-HBASE-Build/9309/art
> ifact/patchprocess/whitespace-eol.txt
> [2] https://builds.apache.org/job/PreCommit-HBASE-Build/9309/art
> ifact/patchprocess/whitespace-tabs.txt
>

PreCommit whitespace failures on JVM crash dump files

[Josh Elser]

Hiya,

Noticed a PreCommit failure from over the weekend on the whitespace 
(EOL, tabs) checks [1][2]:


e.g.

```
./hbase-server/hs_err_pid30492.log:637:73100-734f0 rw-p  
00:00 0
./hbase-server/hs_err_pid30492.log:638:734f0-75100 rw-p  
00:00 0

```

Any thoughts on the right way to fix this one?

The whitespace check ran before the tests did, so I can only assume 
these were laying around on the filesystem before the PreCommit job 
itself ran. Do we need to be a bit more aggressive in the clean steps?


I do see in the Jenkinsfile, we are taking advantage of the options in 
Yetus to ignore certain files for whitespace. That would be another 
option to "fix" this.


- Josh

[1] 
https://builds.apache.org/job/PreCommit-HBASE-Build/9309/artifact/patchprocess/whitespace-eol.txt
[2] 
https://builds.apache.org/job/PreCommit-HBASE-Build/9309/artifact/patchprocess/whitespace-tabs.txt

[jira] [Created] (HBASE-19070) temporarily make the mvnsite nightly test non-voting.

[Sean Busbey (JIRA)]

Sean Busbey created HBASE-19070:
---

 Summary: temporarily make the mvnsite nightly test non-voting.
 Key: HBASE-19070
 URL: https://issues.apache.org/jira/browse/HBASE-19070
 Project: HBase
  Issue Type: Sub-task
  Components: build
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Guanghao Zhang]

Congratulations!

2017-10-23 18:11 GMT+08:00 Jingcheng Du :

> Congratulations!
>
> 2017-10-23 16:00 GMT+08:00 OpenInx :
>
> > Thanks the HBase Community  &  the HBase Committee.  Will do my best to
> > grow with the community.
> >
> > On Mon, Oct 23, 2017 at 3:10 PM, Ted Yu  wrote:
> >
> > > Congratulations,  Cheng.
> > >  Original message From: libis 
> > > Date: 10/22/17  11:50 PM  (GMT-08:00) To: dev@hbase.apache.org
> Subject:
> > > Re: [ANNOUNCE] New HBase committer Zheng Hu
> > > Congratulations!!!
> > >
> > > 2017-10-23 14:21 GMT+08:00 ramkrishna vasudevan <
> > > ramkrishna.s.vasude...@gmail.com>:
> > >
> > > > Welcome Zheng and congratulations !!!
> > > >
> > > > On Mon, Oct 23, 2017 at 11:48 AM, Duo Zhang 
> > wrote:
> > > >
> > > > > On behalf of the Apache HBase PMC, I am pleased to announce that
> > Zheng
> > > Hu
> > > > > has accepted the PMC's invitation to become a committer on the
> > project.
> > > > We
> > > > > appreciate all of Zheng's generous contributions thus far and look
> > > > forward
> > > > > to his continued involvement.
> > > > >
> > > > > Congratulations and welcome, Zheng!
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > ==
> > Openinx  blog : http://openinx.github.io
> >
> > TO BE A GREAT HACKER !
> > ==
> >
>

[jira] [Created] (HBASE-19069) Do not wrap the original CompactionLifeCycleTracker when calling CP hooks

[Duo Zhang (JIRA)]

Duo Zhang created HBASE-19069:
-

 Summary: Do not wrap the original CompactionLifeCycleTracker when 
calling CP hooks
 Key: HBASE-19069
 URL: https://issues.apache.org/jira/browse/HBASE-19069
 Project: HBase
  Issue Type: Sub-task
  Components: Compaction, Coprocessors
Reporter: Duo Zhang
Priority: Blocker


My fault...

The UT added in HBASE-18989 does not follow the new rule so the CP is not 
loaded actually... When fixed the UT fails...

The reason is that I use a wrapped CompactionLifeCycleTracker for implementing 
CompactionLifeCycleTracker. Obviously this is not the correct approach as we 
need to pass the original one to CP hooks...

Let me fix.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Jingcheng Du]

Congratulations!

2017-10-23 16:00 GMT+08:00 OpenInx :

> Thanks the HBase Community  &  the HBase Committee.  Will do my best to
> grow with the community.
>
> On Mon, Oct 23, 2017 at 3:10 PM, Ted Yu  wrote:
>
> > Congratulations,  Cheng.
> >  Original message From: libis 
> > Date: 10/22/17  11:50 PM  (GMT-08:00) To: dev@hbase.apache.org Subject:
> > Re: [ANNOUNCE] New HBase committer Zheng Hu
> > Congratulations!!!
> >
> > 2017-10-23 14:21 GMT+08:00 ramkrishna vasudevan <
> > ramkrishna.s.vasude...@gmail.com>:
> >
> > > Welcome Zheng and congratulations !!!
> > >
> > > On Mon, Oct 23, 2017 at 11:48 AM, Duo Zhang 
> wrote:
> > >
> > > > On behalf of the Apache HBase PMC, I am pleased to announce that
> Zheng
> > Hu
> > > > has accepted the PMC's invitation to become a committer on the
> project.
> > > We
> > > > appreciate all of Zheng's generous contributions thus far and look
> > > forward
> > > > to his continued involvement.
> > > >
> > > > Congratulations and welcome, Zheng!
> > > >
> > >
> >
>
>
>
> --
> ==
> Openinx  blog : http://openinx.github.io
>
> TO BE A GREAT HACKER !
> ==
>

Re: [ANNOUNCE] New HBase committer Zheng Hu

[OpenInx]

Thanks the HBase Community  &  the HBase Committee.  Will do my best to
grow with the community.

On Mon, Oct 23, 2017 at 3:10 PM, Ted Yu  wrote:

> Congratulations,  Cheng.
>  Original message From: libis 
> Date: 10/22/17  11:50 PM  (GMT-08:00) To: dev@hbase.apache.org Subject:
> Re: [ANNOUNCE] New HBase committer Zheng Hu
> Congratulations!!!
>
> 2017-10-23 14:21 GMT+08:00 ramkrishna vasudevan <
> ramkrishna.s.vasude...@gmail.com>:
>
> > Welcome Zheng and congratulations !!!
> >
> > On Mon, Oct 23, 2017 at 11:48 AM, Duo Zhang  wrote:
> >
> > > On behalf of the Apache HBase PMC, I am pleased to announce that Zheng
> Hu
> > > has accepted the PMC's invitation to become a committer on the project.
> > We
> > > appreciate all of Zheng's generous contributions thus far and look
> > forward
> > > to his continued involvement.
> > >
> > > Congratulations and welcome, Zheng!
> > >
> >
>



-- 
==
Openinx  blog : http://openinx.github.io

TO BE A GREAT HACKER !
==

[jira] [Created] (HBASE-19068) Change all url of apache.org from HTTP to HTTPS in HBase book

[Yung-An He (JIRA)]

Yung-An He created HBASE-19068:
--

 Summary: Change all url of apache.org from HTTP to HTTPS in HBase 
book
 Key: HBASE-19068
 URL: https://issues.apache.org/jira/browse/HBASE-19068
 Project: HBase
  Issue Type: Improvement
  Components: documentation
Reporter: Yung-An He
Assignee: Yung-An He
 Fix For: 2.0.0


We should change all the links of apache.org from HTTP to HTTPS in our hbase 
book due to the security issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Ted Yu]

Congratulations,  Cheng. 
 Original message From: libis  Date: 
10/22/17  11:50 PM  (GMT-08:00) To: dev@hbase.apache.org Subject: Re: 
[ANNOUNCE] New HBase committer Zheng Hu 
Congratulations!!!

2017-10-23 14:21 GMT+08:00 ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com>:

> Welcome Zheng and congratulations !!!
>
> On Mon, Oct 23, 2017 at 11:48 AM, Duo Zhang  wrote:
>
> > On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> > has accepted the PMC's invitation to become a committer on the project.
> We
> > appreciate all of Zheng's generous contributions thus far and look
> forward
> > to his continued involvement.
> >
> > Congratulations and welcome, Zheng!
> >
>