Re: [VOTE] First release candidate for HBase 2.0.3 is available

2018-11-30 Thread Stack
Thanks Sean. Vote noted!

On Fri, Nov 30, 2018 at 8:04 PM Sean Busbey  wrote:

> +1 (binding)
>
> sorry for the lag!
> On Sun, Nov 25, 2018 at 6:29 PM Stack  wrote:
> >
> > The first release candidate for HBase 2.0.3 is available for download:
> >
> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/
> >
> > Maven artifacts are also available in a staging repository at:
> >
> >  https://repository.apache.org/content/repositories/orgapachehbase-1237
> >
> > Artifacts are signed with my key (DB9D313DA7874F29) published in our
> > KEYS file at http://www.apache.org/dist/hbase/KEYS
> >
> > The RC corresponds to the signed tag 2.0.3RC0, which currently points
> > to commit
> >
> >   87a3aea8ee2d284807f7d4fbdac1f6d9dfedbc17
> >
> > HBase 2.0.3 is the third maintenance release in the HBase 2.0 line,
> > continuing on the theme of bringing a stable, reliable database to
> > the Hadoop and NoSQL communities. This release includes ~120 bug
> > and improvements fixes done since the 2.0.2 release almost 3 months
> > ago.
> >
> > The detailed source and binary compatibility report vs 2.0.2 has been
> > published for your review, at:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/compat-check-report-2.0.2-vs-2.0.3.html
> >
> > The report shows no incompatibilities.
> >
> > The full list of fixes included in this release is available in
> > the CHANGES.md that ships as part of the release also available
> > here:
> >
> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/CHANGES.md
> >
> > The RELEASENOTES.md are here:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/RELEASENOTES.md
> >
> > Please try out this candidate and vote +1/-1 on whether we should
> > release these artifacts as HBase 2.0.3.
> >
> > The VOTE will remain open for at least 72 hours. Given sufficient votes
> > I would like to close it on Thursday, November 29th, 2018.
> >
> > Thanks,
> > S
>


Re: The child procedure is scheduled in the reversed order in Procedure V2 Framework

2018-11-30 Thread Misty Linville
I like your solution in principle, but I think it would be better to loop
until the queue reports that it's empty.

On Wed, Nov 28, 2018, 8:18 PM OpenInx  Thanks Allan's reply
>
> So if we can ensure that all children are independent, then it won't be a
> problem.
> But the reversed shcedule order is some counterintuitive.   I think a minor
> change can
> help fix this:
>
> diff --git
>
> a/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
>
> b/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> index cbdb9b8..b688ec3 100644
> ---
>
> a/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> +++
>
> b/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
> @@ -1794,7 +1794,7 @@ public class ProcedureExecutor {
>}
>
>private void submitChildrenProcedures(Procedure[]
> subprocs) {
> -for (int i = 0; i < subprocs.length; ++i) {
> +for (int i = subprocs.length - 1; i >= 0; i--) {
>Procedure subproc = subprocs[i];
>subproc.updateMetricsOnSubmit(getEnvironment());
>assert !procedures.containsKey(subproc.getProcId());
>
> Thanks.
>
> On Wed, Nov 28, 2018 at 11:13 PM Allan Yang  wrote:
>
> > Yes, you are right, every procedure will be add to front, so the final
> > execution order may be reversed, But actually, there will be more than
> one
> > worker threads, so likely, they will be executed at the same time. I
> think
> > the design is unintentionally, since all the child procedure should be
> > independent and won't depend on each other, so they can be executed at
> any
> > order. And more, after HBASE-21375
> >  checked in all 2.x
> > branch, the worker thread will execute every possible procedure in the
> > queue, so the front ones won't block, so this won't be a problem.
> > Best Regards
> > Allan Yang
> >
> >
> > OpenInx  于2018年11月28日周三 下午10:42写道:
> >
> > > Hi :
> > >
> > >  I read parts of the procedure v2 framework, and found that  if a
> > procedure
> > > has  3 added child procedure,
> > >  then it's children will be schedued in the reversed order.
> > >
> > > Let me give an example. if  a procedure A added 3 child procedure: B,
> C ,
> > > D.
> > >
> > > a.addChildProcedure(B, C, D)
> > >
> > > The procedure framework will add the B,C,D child produre in a dequeue
> to
> > > schedule
> > >
> > > ( Code Path  --- ProcedureExecutor#execProcedure
> > > -- submitChildrenProcedures  -- dequeue#addFront )
> > >
> > > So the dequeue will be :(front)   D, C, B  (back)
> > >
> > > Then we will poll each procedure from the dequeue, and dispatch to the
> > > executor to run ..
> > >
> > > In the final,   the procedure executing order will be :  D, C, B,
> which
> > is
> > > the revered order  compared to the addChildProcedure order.
> > >
> > > My question is :  is it designed intentionally ?  Or unintentionally
> > doing
> > > the wrong implementation ?
> > >
> > > Seems most the child procedure are region assign or unassign, looks
> like
> > no
> > > problem now..
> > >
> > > Please correct me if I am wrong or missing something.
> > >
> > > Thanks.
> > >
> >
>


[jira] [Created] (HBASE-21538) Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection

2018-11-30 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-21538:
-

 Summary: Rewrite RegionReplicaFlushHandler to use 
AsyncClusterConnection
 Key: HBASE-21538
 URL: https://issues.apache.org/jira/browse/HBASE-21538
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-21537) Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection

2018-11-30 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-21537:
-

 Summary: Rewrite ServerManager.closeRegionSilentlyAndWait to use 
AsyncClusterConnection
 Key: HBASE-21537
 URL: https://issues.apache.org/jira/browse/HBASE-21537
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [VOTE] First release candidate for HBase 2.0.3 is available

2018-11-30 Thread Sean Busbey
+1 (binding)

sorry for the lag!
On Sun, Nov 25, 2018 at 6:29 PM Stack  wrote:
>
> The first release candidate for HBase 2.0.3 is available for download:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/
>
> Maven artifacts are also available in a staging repository at:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1237
>
> Artifacts are signed with my key (DB9D313DA7874F29) published in our
> KEYS file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 2.0.3RC0, which currently points
> to commit
>
>   87a3aea8ee2d284807f7d4fbdac1f6d9dfedbc17
>
> HBase 2.0.3 is the third maintenance release in the HBase 2.0 line,
> continuing on the theme of bringing a stable, reliable database to
> the Hadoop and NoSQL communities. This release includes ~120 bug
> and improvements fixes done since the 2.0.2 release almost 3 months
> ago.
>
> The detailed source and binary compatibility report vs 2.0.2 has been
> published for your review, at:
>
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/compat-check-report-2.0.2-vs-2.0.3.html
>
> The report shows no incompatibilities.
>
> The full list of fixes included in this release is available in
> the CHANGES.md that ships as part of the release also available
> here:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/CHANGES.md
>
> The RELEASENOTES.md are here:
>
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/RELEASENOTES.md
>
> Please try out this candidate and vote +1/-1 on whether we should
> release these artifacts as HBase 2.0.3.
>
> The VOTE will remain open for at least 72 hours. Given sufficient votes
> I would like to close it on Thursday, November 29th, 2018.
>
> Thanks,
> S


Re: [[RESULT][VOTE] First release candidate for HBase 2.0.3 is available

2018-11-30 Thread Stack
(Thanks Zach)

With 6+1s the vote passes (three binding). Thanks to all who voted.  Let me
push out the release.

S

On Sun, Nov 25, 2018 at 4:22 PM Stack  wrote:

> The first release candidate for HBase 2.0.3 is available for download:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/
>
> Maven artifacts are also available in a staging repository at:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1237
>
> Artifacts are signed with my key (DB9D313DA7874F29) published in our
> KEYS file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 2.0.3RC0, which currently points
> to commit
>
>   87a3aea8ee2d284807f7d4fbdac1f6d9dfedbc17
>
> HBase 2.0.3 is the third maintenance release in the HBase 2.0 line,
> continuing on the theme of bringing a stable, reliable database to
> the Hadoop and NoSQL communities. This release includes ~120 bug
> and improvements fixes done since the 2.0.2 release almost 3 months
> ago.
>
> The detailed source and binary compatibility report vs 2.0.2 has been
> published for your review, at:
>
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/compat-check-report-2.0.2-vs-2.0.3.html
>
> The report shows no incompatibilities.
>
> The full list of fixes included in this release is available in
> the CHANGES.md that ships as part of the release also available
> here:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/CHANGES.md
>
> The RELEASENOTES.md are here:
>
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/RELEASENOTES.md
>
> Please try out this candidate and vote +1/-1 on whether we should
> release these artifacts as HBase 2.0.3.
>
> The VOTE will remain open for at least 72 hours. Given sufficient votes
> I would like to close it on Thursday, November 29th, 2018.
>
> Thanks,
> S
>


Re: [VOTE] First release candidate for HBase 2.0.3 is available

2018-11-30 Thread Zach York
+1

- checked sums and sigs: Ok
- Rat check: Ok
- built from source: Ok (8u92)
- Ran unit tests: Ok
- Messed around with some basic shell commands: Ok
- Loaded and read 1M rows with LTT: Ok
- Sanity checked Web UI: Ok

On Fri, Nov 30, 2018 at 4:16 PM Zach York 
wrote:

> I'm in the process of testing it. I'll be back in an hour or less
> (hopefully :))
>
> On Fri, Nov 30, 2018 at 3:27 PM Stack  wrote:
>
>> I love these non-binding votes. Thanks to you all for trying the RC. Need
>> one more binding vote to release after my +1 here (Allan Yang just made
>> PMC! Yah! for Allan!).
>> Thanks,
>> S
>>
>> On Sun, Nov 25, 2018 at 4:22 PM Stack  wrote:
>>
>> > The first release candidate for HBase 2.0.3 is available for download:
>> >
>> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/
>> >
>> > Maven artifacts are also available in a staging repository at:
>> >
>> >  https://repository.apache.org/content/repositories/orgapachehbase-1237
>> >
>> > Artifacts are signed with my key (DB9D313DA7874F29) published in our
>> > KEYS file at http://www.apache.org/dist/hbase/KEYS
>> >
>> > The RC corresponds to the signed tag 2.0.3RC0, which currently points
>> > to commit
>> >
>> >   87a3aea8ee2d284807f7d4fbdac1f6d9dfedbc17
>> >
>> > HBase 2.0.3 is the third maintenance release in the HBase 2.0 line,
>> > continuing on the theme of bringing a stable, reliable database to
>> > the Hadoop and NoSQL communities. This release includes ~120 bug
>> > and improvements fixes done since the 2.0.2 release almost 3 months
>> > ago.
>> >
>> > The detailed source and binary compatibility report vs 2.0.2 has been
>> > published for your review, at:
>> >
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/compat-check-report-2.0.2-vs-2.0.3.html
>> >
>> > The report shows no incompatibilities.
>> >
>> > The full list of fixes included in this release is available in
>> > the CHANGES.md that ships as part of the release also available
>> > here:
>> >
>> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/CHANGES.md
>> >
>> > The RELEASENOTES.md are here:
>> >
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/RELEASENOTES.md
>> >
>> > Please try out this candidate and vote +1/-1 on whether we should
>> > release these artifacts as HBase 2.0.3.
>> >
>> > The VOTE will remain open for at least 72 hours. Given sufficient votes
>> > I would like to close it on Thursday, November 29th, 2018.
>> >
>> > Thanks,
>> > S
>> >
>>
>


Re: [VOTE] First release candidate for HBase 2.0.3 is available

2018-11-30 Thread Zach York
I'm in the process of testing it. I'll be back in an hour or less
(hopefully :))

On Fri, Nov 30, 2018 at 3:27 PM Stack  wrote:

> I love these non-binding votes. Thanks to you all for trying the RC. Need
> one more binding vote to release after my +1 here (Allan Yang just made
> PMC! Yah! for Allan!).
> Thanks,
> S
>
> On Sun, Nov 25, 2018 at 4:22 PM Stack  wrote:
>
> > The first release candidate for HBase 2.0.3 is available for download:
> >
> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/
> >
> > Maven artifacts are also available in a staging repository at:
> >
> >  https://repository.apache.org/content/repositories/orgapachehbase-1237
> >
> > Artifacts are signed with my key (DB9D313DA7874F29) published in our
> > KEYS file at http://www.apache.org/dist/hbase/KEYS
> >
> > The RC corresponds to the signed tag 2.0.3RC0, which currently points
> > to commit
> >
> >   87a3aea8ee2d284807f7d4fbdac1f6d9dfedbc17
> >
> > HBase 2.0.3 is the third maintenance release in the HBase 2.0 line,
> > continuing on the theme of bringing a stable, reliable database to
> > the Hadoop and NoSQL communities. This release includes ~120 bug
> > and improvements fixes done since the 2.0.2 release almost 3 months
> > ago.
> >
> > The detailed source and binary compatibility report vs 2.0.2 has been
> > published for your review, at:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/compat-check-report-2.0.2-vs-2.0.3.html
> >
> > The report shows no incompatibilities.
> >
> > The full list of fixes included in this release is available in
> > the CHANGES.md that ships as part of the release also available
> > here:
> >
> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/CHANGES.md
> >
> > The RELEASENOTES.md are here:
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/RELEASENOTES.md
> >
> > Please try out this candidate and vote +1/-1 on whether we should
> > release these artifacts as HBase 2.0.3.
> >
> > The VOTE will remain open for at least 72 hours. Given sufficient votes
> > I would like to close it on Thursday, November 29th, 2018.
> >
> > Thanks,
> > S
> >
>


Re: [VOTE] First release candidate for HBase 2.0.3 is available

2018-11-30 Thread Stack
I love these non-binding votes. Thanks to you all for trying the RC. Need
one more binding vote to release after my +1 here (Allan Yang just made
PMC! Yah! for Allan!).
Thanks,
S

On Sun, Nov 25, 2018 at 4:22 PM Stack  wrote:

> The first release candidate for HBase 2.0.3 is available for download:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/
>
> Maven artifacts are also available in a staging repository at:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1237
>
> Artifacts are signed with my key (DB9D313DA7874F29) published in our
> KEYS file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 2.0.3RC0, which currently points
> to commit
>
>   87a3aea8ee2d284807f7d4fbdac1f6d9dfedbc17
>
> HBase 2.0.3 is the third maintenance release in the HBase 2.0 line,
> continuing on the theme of bringing a stable, reliable database to
> the Hadoop and NoSQL communities. This release includes ~120 bug
> and improvements fixes done since the 2.0.2 release almost 3 months
> ago.
>
> The detailed source and binary compatibility report vs 2.0.2 has been
> published for your review, at:
>
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/compat-check-report-2.0.2-vs-2.0.3.html
>
> The report shows no incompatibilities.
>
> The full list of fixes included in this release is available in
> the CHANGES.md that ships as part of the release also available
> here:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/CHANGES.md
>
> The RELEASENOTES.md are here:
>
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.3RC0/RELEASENOTES.md
>
> Please try out this candidate and vote +1/-1 on whether we should
> release these artifacts as HBase 2.0.3.
>
> The VOTE will remain open for at least 72 hours. Given sufficient votes
> I would like to close it on Thursday, November 29th, 2018.
>
> Thanks,
> S
>


[jira] [Reopened] (HBASE-21464) Splitting blocked with meta NSRE during split transaction

2018-11-30 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell reopened HBASE-21464:


I reverted my commit because I was able to reproduce the problem again. 

That part of the change we removed where the cache lookup for meta region was 
modified was what was necessary. There is something wrong with the cache 
lookup. I don't see a change between 1.4.2 and 1.4.3 that is any kind of 
smoking gun so am a bit stumped here.

> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to 
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO  
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
>  client.RpcRetryingCaller: Call exception, tries=13, retries=350, 
> started=88590 ms ago, cancelled=false, 
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
>      at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
>         at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
>         at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
>         at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
>         at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
>         at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row 
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table 
> 'hbase:meta' at region=hbase:meta,1.1588230740, 
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586, 
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165] 
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta 
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying 
> after sleep of 20158 because: No server address listed in hbase:meta for 
> region 
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081. 
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG 
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster: 
> Not running balancer because 3 region(s) in transition: 
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606, 
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417}, 
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606, 
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [DISCUSS] JDK8 for >=1.5.x?

2018-11-30 Thread Sean Busbey
I'm opposed if only because I don't like weakening our compatibility
promises.

What does Phoenix (or other downstream folks) lose out on because we work
with jdk7 on our branch-1 releases?

On Fri, Nov 30, 2018, 15:20 Josh Elser  We came back across this in Phoenix recently and Andrew suggested we
> talk about it here in HBase.
>
> Can/should we move to requiring Java8 for HBase 1.5 and beyond?
>
> It would have my vote.
>
> - Josh
>


[DISCUSS] JDK8 for >=1.5.x?

2018-11-30 Thread Josh Elser
We came back across this in Phoenix recently and Andrew suggested we 
talk about it here in HBase.


Can/should we move to requiring Java8 for HBase 1.5 and beyond?

It would have my vote.

- Josh


[jira] [Created] (HBASE-21536) Fix completebulkload usage instructions

2018-11-30 Thread Artem Ervits (JIRA)
Artem Ervits created HBASE-21536:


 Summary: Fix completebulkload usage instructions
 Key: HBASE-21536
 URL: https://issues.apache.org/jira/browse/HBASE-21536
 Project: HBase
  Issue Type: Task
  Components: documentation, mapreduce
Reporter: Artem Ervits
Assignee: Artem Ervits


Usage information upon invoking LoadIncrementalHFiles is misleading and 
error-prone.
{code:java}
usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
-Dcreate.table=no - can be used to avoid creation of table by this tool
Note: if you set this to 'no', then the target table must already exist in HBase
-loadTable implies your baseDirectory to store file has a depth of 3 ,you must 
have an existing table
-Dignore.unmatched.families=yes - can be used to ignore unmatched column 
families{code}
in case of invoking the class via hbase command, completebulkload argument is 
unnecessary and only required via hadoop jar invocation. This is also an 
attempt to clarify where <-loadTable> and <-Dargs> arguments go on the command 
line.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-21535) Zombie Master detector is not working

2018-11-30 Thread Pankaj Kumar (JIRA)
Pankaj Kumar created HBASE-21535:


 Summary: Zombie Master detector is not working
 Key: HBASE-21535
 URL: https://issues.apache.org/jira/browse/HBASE-21535
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Pankaj Kumar
Assignee: Pankaj Kumar


We have InitializationMonitor thread in HMaster which detects Zombie Hmaster 
based on _hbase.master.initializationmonitor.timeout _and halts if 
_hbase.master.initializationmonitor.haltontimeout_ set _true_.

After HBASE-19694, HMaster initialization order was correted. Hmaster is set 
active after Initializing ZK system trackers as follows,
{noformat}
 status.setStatus("Initializing ZK system trackers");
 initializeZKBasedSystemTrackers();
 status.setStatus("Loading last flushed sequence id of regions");
 try {
 this.serverManager.loadLastFlushedSequenceIds();
 } catch (IOException e) {
 LOG.debug("Failed to load last flushed sequence id of regions"
 + " from file system", e);
 }
 // Set ourselves as active Master now our claim has succeeded up in zk.
 this.activeMaster = true;
{noformat}

But Zombie detector thread is started at the begining phase of 
finishActiveMasterInitialization(),
{noformat}
 private void finishActiveMasterInitialization(MonitoredTask status) throws 
IOException,
 InterruptedException, KeeperException, ReplicationException {
 Thread zombieDetector = new Thread(new InitializationMonitor(this),
 "ActiveMasterInitializationMonitor-" + System.currentTimeMillis());
 zombieDetector.setDaemon(true);
 zombieDetector.start();
{noformat}

During zombieDetector execution "master.isActiveMaster()" will be false, so it 
won't wait and cant detect zombie master.
{noformat}
 @Override
 public void run() {
 try {
 while (!master.isStopped() && master.isActiveMaster()) {
 Thread.sleep(timeout);
 if (master.isInitialized()) {
 LOG.debug("Initialization completed within allotted tolerance. Monitor 
exiting.");
 } else {
 LOG.error("Master failed to complete initialization after " + timeout + "ms. 
Please"
 + " consider submitting a bug report including a thread dump of this 
process.");
 if (haltOnTimeout) {
 LOG.error("Zombie Master exiting. Thread dump to stdout");
 Threads.printThreadInfo(System.out, "Zombie HMaster");
 System.exit(-1);
 }
 }
 }
 } catch (InterruptedException ie) {
 LOG.trace("InitMonitor thread interrupted. Existing.");
 }
 }
 }
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)