[jira] [Updated] (HBASE-21084) The split parent region becomes online after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Toshihiro Suzuki updated HBASE-21084:
-
Attachment: HBASE-21084.master.001.patch

> The split parent region becomes online after cloning a snapshot
> ---
>
> Key: HBASE-21084
> URL: https://issues.apache.org/jira/browse/HBASE-21084
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21084.master.001.patch
>
>
> Investigating HBASE-21015, I found another issue. It seems like after 
> HBASE-20881, the split parent region becomes online after cloning a snapshot.
> Steps to reproduce are as follows, which is the same as the steps in 
> HBASE-21015:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some data into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After following the above steps, the split parent region becomes online.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21084) The split parent region becomes online after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Toshihiro Suzuki updated HBASE-21084:
-
Description: 
Investigating HBASE-21015, I found another issue. It seems like after 
HBASE-20881, the split parent region becomes online after cloning a snapshot.

Steps to reproduce are as follows, which is the same as the steps in 
HBASE-21015:

1. Create a table
{code}
create "test", "cf"
{code}

2. Put some data into the table
{code}
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
{code}

3. Split the table
{code}
split "test"
{code}

4. Take a snapshot of the table before CatalogJanitor cleans up the split 
parent region
{code}
snapshot "test", "snap"
{code}

5. Clone the snapshot
{code}
clone_snapshot "snap", "cloned_table"
{code}

After following the above steps, the split parent region becomes online.


  was:
Investigating HBASE-21015, I found another issue. It seems like after 
HBASE-20881, the split parent region becomes online after cloning a snapshot.

Steps to reproduce are as follows, which is the same as the steps in 
HBASE-21015:

1. Create a table
{code}
create "test", "cf"
{code}

2. Put some date into the table
{code}
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
{code}

3. Split the table
{code}
split "test"
{code}

4. Take a snapshot of the table before CatalogJanitor cleans up the split 
parent region
{code}
snapshot "test", "snap"
{code}

5. Clone the snapshot
{code}
clone_snapshot "snap", "cloned_table"
{code}

After following the above steps, the split parent region becomes online.



> The split parent region becomes online after cloning a snapshot
> ---
>
> Key: HBASE-21084
> URL: https://issues.apache.org/jira/browse/HBASE-21084
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
>
> Investigating HBASE-21015, I found another issue. It seems like after 
> HBASE-20881, the split parent region becomes online after cloning a snapshot.
> Steps to reproduce are as follows, which is the same as the steps in 
> HBASE-21015:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some data into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After following the above steps, the split parent region becomes online.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Toshihiro Suzuki updated HBASE-21015:
-
Description: 
Steps to reproduce are as follows:

1. Create a table
{code}
create "test", "cf"
{code}

2. Put some data into the table
{code}
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
{code}

3. Split the table
{code}
split "test"
{code}

4. Take a snapshot of the table before CatalogJanitor cleans up the split 
parent region
{code}
snapshot "test", "snap"
{code}

5. Clone the snapshot
{code}
clone_snapshot "snap", "cloned_table"
{code}

After cloning the snapshot, the state of the split parent region is OFFLINE, 
but it should be SPLIT.
 !Screen Shot 2018-08-07 at 00.40.40.png! 


  was:
Steps to reproduce are as follows:

1. Create a table
{code}
create "test", "cf"
{code}

2. Put some date into the table
{code}
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
{code}

3. Split the table
{code}
split "test"
{code}

4. Take a snapshot of the table before CatalogJanitor cleans up the split 
parent region
{code}
snapshot "test", "snap"
{code}

5. Clone the snapshot
{code}
clone_snapshot "snap", "cloned_table"
{code}

After cloning the snapshot, the state of the split parent region is OFFLINE, 
but it should be SPLIT.
 !Screen Shot 2018-08-07 at 00.40.40.png! 



> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some data into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-21084) The split parent region becomes online after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)
Toshihiro Suzuki created HBASE-21084:


 Summary: The split parent region becomes online after cloning a 
snapshot
 Key: HBASE-21084
 URL: https://issues.apache.org/jira/browse/HBASE-21084
 Project: HBase
  Issue Type: Bug
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki


Investigating HBASE-21015, I found another issue. It seems like after 
HBASE-20881, the split parent region becomes online after cloning a snapshot.

Steps to reproduce are as follows, which is the same as the steps in 
HBASE-21015:

1. Create a table
{code}
create "test", "cf"
{code}

2. Put some date into the table
{code}
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
{code}

3. Split the table
{code}
split "test"
{code}

4. Take a snapshot of the table before CatalogJanitor cleans up the split 
parent region
{code}
snapshot "test", "snap"
{code}

5. Clone the snapshot
{code}
clone_snapshot "snap", "cloned_table"
{code}

After following the above steps, the split parent region becomes online.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21076) TestTableResource fails with NPE

2018-08-21 Thread Sean Busbey (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587355#comment-16587355
 ] 

Sean Busbey commented on HBASE-21076:
-

note that due to HBASE-19265, precommit isn't going to run TestTableResource 
(it's in the untrusted list because it fails 100% of the time before this 
patch). reviewers should manually run the test locally to confirm it works. 
Doing this while doing a minimum of other things: {{mvn 
-Dtest=TestTableResource -pl hbase-rest -am package}}

> TestTableResource fails with NPE
> 
>
> Key: HBASE-21076
> URL: https://issues.apache.org/jira/browse/HBASE-21076
> Project: HBase
>  Issue Type: Test
>  Components: REST, test
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.1.1, 1.4.7
>Reporter: Ted Yu
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21076.0.patch
>
>
> The following can be observed in master branch:
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.rest.TestTableResource.setUpBeforeClass(TestTableResource.java:134)
> {code}
> The NPE comes from the following in TestEndToEndSplitTransaction :
> {code}
> compactAndBlockUntilDone(TEST_UTIL.getAdmin(),
>   TEST_UTIL.getMiniHBaseCluster().getRegionServer(0), 
> daughterA.getRegionName());
> {code}
> Initial check of the code shows that TestEndToEndSplitTransaction uses 
> TEST_UTIL instance which is created within TestEndToEndSplitTransaction. 
> However, TestTableResource creates its own instance of HBaseTestingUtility.
> Meaning TEST_UTIL.getMiniHBaseCluster() would return null, since the instance 
> created by TestEndToEndSplitTransaction has hbaseCluster as null.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21031) Memory leak if replay edits failed during region opening

2018-08-21 Thread Allan Yang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587350#comment-16587350
 ] 

Allan Yang commented on HBASE-21031:


{quote}
+ LOG.error("replay failed, that's expected", t);
I think the test is better without this since we already fail the test if the 
replay doesn't fail, so this is unneeded lines in the log. 
{quote}
Done!
{quote}
please try to put line breaks in between words instead of in the middle of 
words.
{quote}
Done!
{quote}
Also, I responded to your other question about my comments for the 
try/catch/finally directly on RB. Let me know if that doesn't make sense or if 
you think it's better to leave them as it is.
{quote}
I think we'd better leave them there, since we only catch IOException here, but 
we need to abort the status whatever exception is thrown.
Thanks for reviewing,[~mdrob]. Can you take a look at HBASE-21041 too, it is 
another bug you mentioned in the review board.

> Memory leak if replay edits failed during region opening
> 
>
> Key: HBASE-21031
> URL: https://issues.apache.org/jira/browse/HBASE-21031
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21031.branch-2.0.001.patch, 
> HBASE-21031.branch-2.0.002.patch, HBASE-21031.branch-2.0.003.patch, 
> HBASE-21031.branch-2.0.004.patch, HBASE-21031.branch-2.0.005.patch, 
> HBASE-21031.branch-2.0.006.patch, memoryleak.png
>
>
> Due to HBASE-21029, when replaying edits with a lot of same cells, the 
> memstore won't flush,  a exception will throw when all heap space was used:
> {code}
> 2018-08-06 15:52:27,590 ERROR 
> [RS_OPEN_REGION-regionserver/hb-bp10cw4ejoy0a2f3f-009:16020-2] 
> handler.OpenRegionHandler(302): Failed open of 
> region=hbase_test,dffa78,1531227033378.cbf9a2daf3aaa0c7e931e9c9a7b53f41., 
> starting to roll back the global memstore size.
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at 
> org.apache.hadoop.hbase.regionserver.OnheapChunk.allocateDataBuffer(OnheapChunk.java:41)
> at org.apache.hadoop.hbase.regionserver.Chunk.init(Chunk.java:104)
> at 
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:226)
> at 
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:180)
> at 
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:163)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.getOrMakeChunk(MemStoreLABImpl.java:273)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:148)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:111)
> at 
> org.apache.hadoop.hbase.regionserver.Segment.maybeCloneWithAllocator(Segment.java:178)
> at 
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.maybeCloneWithAllocator(AbstractMemStore.java:287)
> at 
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:107)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:706)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.restoreEdit(HRegion.java:5494)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4608)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:4404)
> {code}
> After this exception, the memstore did not roll back, and since MSLAB is 
> used, all the chunk allocated won't release for ever. Those memory is leak 
> forever...
> We need to rollback the memory if open region fails(For now, only global 
> memstore size is decreased after failure).
> Another problem is that we use replayEditsPerRegion in RegionServerAccounting 
> to record how many memory used during replaying. And decrease the global 
> memstore size if replay fails. This is not right, since during replaying, we 
> may also flush the memstore, the size in the map of replayEditsPerRegion is 
> not accurate at all! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21031) Memory leak if replay edits failed during region opening

2018-08-21 Thread Allan Yang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allan Yang updated HBASE-21031:
---
Attachment: HBASE-21031.branch-2.0.006.patch

> Memory leak if replay edits failed during region opening
> 
>
> Key: HBASE-21031
> URL: https://issues.apache.org/jira/browse/HBASE-21031
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21031.branch-2.0.001.patch, 
> HBASE-21031.branch-2.0.002.patch, HBASE-21031.branch-2.0.003.patch, 
> HBASE-21031.branch-2.0.004.patch, HBASE-21031.branch-2.0.005.patch, 
> HBASE-21031.branch-2.0.006.patch, memoryleak.png
>
>
> Due to HBASE-21029, when replaying edits with a lot of same cells, the 
> memstore won't flush,  a exception will throw when all heap space was used:
> {code}
> 2018-08-06 15:52:27,590 ERROR 
> [RS_OPEN_REGION-regionserver/hb-bp10cw4ejoy0a2f3f-009:16020-2] 
> handler.OpenRegionHandler(302): Failed open of 
> region=hbase_test,dffa78,1531227033378.cbf9a2daf3aaa0c7e931e9c9a7b53f41., 
> starting to roll back the global memstore size.
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at 
> org.apache.hadoop.hbase.regionserver.OnheapChunk.allocateDataBuffer(OnheapChunk.java:41)
> at org.apache.hadoop.hbase.regionserver.Chunk.init(Chunk.java:104)
> at 
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:226)
> at 
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:180)
> at 
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:163)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.getOrMakeChunk(MemStoreLABImpl.java:273)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:148)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:111)
> at 
> org.apache.hadoop.hbase.regionserver.Segment.maybeCloneWithAllocator(Segment.java:178)
> at 
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.maybeCloneWithAllocator(AbstractMemStore.java:287)
> at 
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:107)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:706)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.restoreEdit(HRegion.java:5494)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4608)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:4404)
> {code}
> After this exception, the memstore did not roll back, and since MSLAB is 
> used, all the chunk allocated won't release for ever. Those memory is leak 
> forever...
> We need to rollback the memory if open region fails(For now, only global 
> memstore size is decreased after failure).
> Another problem is that we use replayEditsPerRegion in RegionServerAccounting 
> to record how many memory used during replaying. And decrease the global 
> memstore size if replay fails. This is not right, since during replaying, we 
> may also flush the memstore, the size in the map of replayEditsPerRegion is 
> not accurate at all! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations

2018-08-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587329#comment-16587329
 ] 

Hadoop QA commented on HBASE-19008:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 5 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
 7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
23s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  3m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} hbase-client: The patch generated 0 new + 376 
unchanged - 13 fixed = 376 total (was 389) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
18s{color} | {color:green} hbase-server: The patch generated 0 new + 404 
unchanged - 2 fixed = 404 total (was 406) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch hbase-spark passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
27s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
8m  0s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m  
8s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}136m 
34s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  5m  
0s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
59s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}194m 20s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19008 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12936401/HBA

[jira] [Updated] (HBASE-21076) TestTableResource fails with NPE

2018-08-21 Thread Sean Busbey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-21076:

Status: Patch Available  (was: In Progress)

-v0
  - avoid calling split at all by requesting the number of regions we want at 
creation
  - fix a bug in how the test turns non-printable characters into a string

> TestTableResource fails with NPE
> 
>
> Key: HBASE-21076
> URL: https://issues.apache.org/jira/browse/HBASE-21076
> Project: HBase
>  Issue Type: Test
>  Components: REST, test
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.1.1, 1.4.7
>Reporter: Ted Yu
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21076.0.patch
>
>
> The following can be observed in master branch:
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.rest.TestTableResource.setUpBeforeClass(TestTableResource.java:134)
> {code}
> The NPE comes from the following in TestEndToEndSplitTransaction :
> {code}
> compactAndBlockUntilDone(TEST_UTIL.getAdmin(),
>   TEST_UTIL.getMiniHBaseCluster().getRegionServer(0), 
> daughterA.getRegionName());
> {code}
> Initial check of the code shows that TestEndToEndSplitTransaction uses 
> TEST_UTIL instance which is created within TestEndToEndSplitTransaction. 
> However, TestTableResource creates its own instance of HBaseTestingUtility.
> Meaning TEST_UTIL.getMiniHBaseCluster() would return null, since the instance 
> created by TestEndToEndSplitTransaction has hbaseCluster as null.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21076) TestTableResource fails with NPE

2018-08-21 Thread Sean Busbey (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-21076:

Attachment: HBASE-21076.0.patch

> TestTableResource fails with NPE
> 
>
> Key: HBASE-21076
> URL: https://issues.apache.org/jira/browse/HBASE-21076
> Project: HBase
>  Issue Type: Test
>  Components: REST, test
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.1.1, 1.4.7
>Reporter: Ted Yu
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21076.0.patch
>
>
> The following can be observed in master branch:
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.rest.TestTableResource.setUpBeforeClass(TestTableResource.java:134)
> {code}
> The NPE comes from the following in TestEndToEndSplitTransaction :
> {code}
> compactAndBlockUntilDone(TEST_UTIL.getAdmin(),
>   TEST_UTIL.getMiniHBaseCluster().getRegionServer(0), 
> daughterA.getRegionName());
> {code}
> Initial check of the code shows that TestEndToEndSplitTransaction uses 
> TEST_UTIL instance which is created within TestEndToEndSplitTransaction. 
> However, TestTableResource creates its own instance of HBaseTestingUtility.
> Meaning TEST_UTIL.getMiniHBaseCluster() would return null, since the instance 
> created by TestEndToEndSplitTransaction has hbaseCluster as null.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21042) processor.getRowsToLock() always assumes there is some row being locked in HRegion#processRowsWithLocks

2018-08-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587265#comment-16587265
 ] 

Hudson commented on HBASE-21042:


Results for branch branch-1.2
[build #439 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/439/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/439//General_Nightly_Build_Report/]


(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/439//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/439//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> processor.getRowsToLock() always assumes there is some row being locked in 
> HRegion#processRowsWithLocks
> ---
>
> Key: HBASE-21042
> URL: https://issues.apache.org/jira/browse/HBASE-21042
> Project: HBase
>  Issue Type: Bug
>Reporter: Thomas D'Silva
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>
> Attachments: 21042.branch-1.txt
>
>
> [~tdsilva] reported at the tail of HBASE-18998 that the fix for HBASE-18998 
> missed finally block of HRegion#processRowsWithLocks
> This is to fix that remaining call.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21069) NPE in StoreScanner.updateReaders causes RS to crash

2018-08-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587250#comment-16587250
 ] 

Hudson commented on HBASE-21069:


Results for branch branch-1
[build #424 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/424/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/424//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/424//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/424//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> NPE in StoreScanner.updateReaders causes RS to crash 
> -
>
> Key: HBASE-21069
> URL: https://issues.apache.org/jira/browse/HBASE-21069
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Thomas D'Silva
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.7
>
> Attachments: HBASE-21069-branch-1.patch, HBASE-21069-branch-1.patch
>
>
> I see the following NPE in the region server log for a table that is taking 
> heavy writes. 
> I am not sure how the {{memStoreScanners}} variable gets set to null.
> {code}
> 2018-08-17 19:59:23,682 DEBUG [MemStoreFlusher.1] 
> regionserver.HRegionFileSystem - Committing store file ...
> 2018-08-17 19:59:23,684 INFO  [MemStoreFlusher.1] regionserver.HStore - Added 
> hdfs://, entries=919170, sequenceid=275114, filesize=22.6 M
> 2018-08-17 19:59:23,689 FATAL [MemStoreFlusher.1] regionserver.HRegionServer 
> - ABORTING region server 
> iotperf1dchbase1a-dnds22-2-prd.eng.sfdc.net,60020,1533915690501: Replay of 
> WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: ..
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2581)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2258)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2220)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2106)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2031)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:508)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:478)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.(ArrayList.java:177)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:827)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1160)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1133)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.access$900(HStore.java:120)
> at 
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2487)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2536)
> ... 9 more
> 2018-08-17 19:59:23,692 FATAL [MemStoreFlusher.1] regionserver.HRegionServer 
> - RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.access.AccessController, 
> org.apache.phoenix.coprocessor.ScanRegionObserver, 
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver, 
> org.apache.phoenix.hbase.index.Indexer, 
> org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver, 
> org.apache.hadoop.hbase.security.token.TokenProvider, 
> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21069) NPE in StoreScanner.updateReaders causes RS to crash

2018-08-21 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587235#comment-16587235
 ] 

Hudson commented on HBASE-21069:


Results for branch branch-1.4
[build #429 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/429/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/429//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/429//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/429//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> NPE in StoreScanner.updateReaders causes RS to crash 
> -
>
> Key: HBASE-21069
> URL: https://issues.apache.org/jira/browse/HBASE-21069
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Thomas D'Silva
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.7
>
> Attachments: HBASE-21069-branch-1.patch, HBASE-21069-branch-1.patch
>
>
> I see the following NPE in the region server log for a table that is taking 
> heavy writes. 
> I am not sure how the {{memStoreScanners}} variable gets set to null.
> {code}
> 2018-08-17 19:59:23,682 DEBUG [MemStoreFlusher.1] 
> regionserver.HRegionFileSystem - Committing store file ...
> 2018-08-17 19:59:23,684 INFO  [MemStoreFlusher.1] regionserver.HStore - Added 
> hdfs://, entries=919170, sequenceid=275114, filesize=22.6 M
> 2018-08-17 19:59:23,689 FATAL [MemStoreFlusher.1] regionserver.HRegionServer 
> - ABORTING region server 
> iotperf1dchbase1a-dnds22-2-prd.eng.sfdc.net,60020,1533915690501: Replay of 
> WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: ..
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2581)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2258)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2220)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2106)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2031)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:508)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:478)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
> at 
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.(ArrayList.java:177)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:827)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1160)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1133)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.access$900(HStore.java:120)
> at 
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2487)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2536)
> ... 9 more
> 2018-08-17 19:59:23,692 FATAL [MemStoreFlusher.1] regionserver.HRegionServer 
> - RegionServer abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.access.AccessController, 
> org.apache.phoenix.coprocessor.ScanRegionObserver, 
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver, 
> org.apache.phoenix.hbase.index.Indexer, 
> org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver, 
> org.apache.hadoop.hbase.security.token.TokenProvider, 
> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure

2018-08-21 Thread Allan Yang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allan Yang updated HBASE-21083:
---
Status: Patch Available  (was: Open)

> Introduce a mechanism to bypass the execution of a stuck procedure
> --
>
> Key: HBASE-21083
> URL: https://issues.apache.org/jira/browse/HBASE-21083
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Affects Versions: 2.0.1, 2.1.0
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21083.branch-2.0.001.patch
>
>
> Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to 
> introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can 
> continue running.
>  we still have some unrevealed bugs hiding in our AMv2 and procedureV2 
> system, we need something to interfere with stuck procedures before HBCK2 can 
> work. This is very crucial for a production ready system. 
> For now, we have little ways to interfere with running procedures. Aborting 
> them is not a good choice, since some procedures are not abort-able. And some 
> procedure may have overridden the abort() method, which will ignore the abort 
> request.
> So, here, I will introduce a mechanism  to bypass the execution of a stuck 
> procedure.
> Basically, I added a field called 'bypass' to Procedure class. If we set this 
> field to true, all the logic in execute/rollback will be skipped, letting 
> this procedure and its ancestors complete normally and releasing the lock 
> resources at last.
> Notice that bypassing a procedure may leave the cluster in a middle state, 
> e.g. the region not assigned, or some hdfs files left behind. 
> The Operators need know the side effect of bypassing and recover the 
> inconsistent state of the cluster themselves, like issuing new procedures to 
> assign the regions.
> A patch will be uploaded and review board will be open. For now, only APIs in 
> ProcedureExecutor are provided. If anything is fine, I will add it to master 
> service and add a shell command to bypass a procedure. Or, maybe we can use 
> dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure

2018-08-21 Thread Allan Yang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allan Yang updated HBASE-21083:
---
Attachment: HBASE-21083.branch-2.0.001.patch

> Introduce a mechanism to bypass the execution of a stuck procedure
> --
>
> Key: HBASE-21083
> URL: https://issues.apache.org/jira/browse/HBASE-21083
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21083.branch-2.0.001.patch
>
>
> Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to 
> introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can 
> continue running.
>  we still have some unrevealed bugs hiding in our AMv2 and procedureV2 
> system, we need something to interfere with stuck procedures before HBCK2 can 
> work. This is very crucial for a production ready system. 
> For now, we have little ways to interfere with running procedures. Aborting 
> them is not a good choice, since some procedures are not abort-able. And some 
> procedure may have overridden the abort() method, which will ignore the abort 
> request.
> So, here, I will introduce a mechanism  to bypass the execution of a stuck 
> procedure.
> Basically, I added a field called 'bypass' to Procedure class. If we set this 
> field to true, all the logic in execute/rollback will be skipped, letting 
> this procedure and its ancestors complete normally and releasing the lock 
> resources at last.
> Notice that bypassing a procedure may leave the cluster in a middle state, 
> e.g. the region not assigned, or some hdfs files left behind. 
> The Operators need know the side effect of bypassing and recover the 
> inconsistent state of the cluster themselves, like issuing new procedures to 
> assign the regions.
> A patch will be uploaded and review board will be open. For now, only APIs in 
> ProcedureExecutor are provided. If anything is fine, I will add it to master 
> service and add a shell command to bypass a procedure. Or, maybe we can use 
> dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-20666) Unsuccessful table creation leaves entry in rsgroup meta table

2018-08-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587190#comment-16587190
 ] 

Hadoop QA commented on HBASE-20666:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
49s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
48s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
9m  5s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}124m 
43s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  5m  
9s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
47s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}178m  2s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20666 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12936385/HBASE-20666.master.002.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 6b1e6d07f560 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / bb3494134e |
| maven | version: Apache Maven 3.5.4 
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_

[jira] [Created] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure

2018-08-21 Thread Allan Yang (JIRA)
Allan Yang created HBASE-21083:
--

 Summary: Introduce a mechanism to bypass the execution of a stuck 
procedure
 Key: HBASE-21083
 URL: https://issues.apache.org/jira/browse/HBASE-21083
 Project: HBase
  Issue Type: Sub-task
  Components: amv2
Affects Versions: 2.0.1, 2.1.0
Reporter: Allan Yang
Assignee: Allan Yang


Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to 
introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can 
continue running.
 we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, 
we need something to interfere with stuck procedures before HBCK2 can work. 
This is very crucial for a production ready system. 

For now, we have little ways to interfere with running procedures. Aborting 
them is not a good choice, since some procedures are not abort-able. And some 
procedure may have overridden the abort() method, which will ignore the abort 
request.

So, here, I will introduce a mechanism  to bypass the execution of a stuck 
procedure.
Basically, I added a field called 'bypass' to Procedure class. If we set this 
field to true, all the logic in execute/rollback will be skipped, letting this 
procedure and its ancestors complete normally and releasing the lock resources 
at last.

Notice that bypassing a procedure may leave the cluster in a middle state, e.g. 
the region not assigned, or some hdfs files left behind. 
The Operators need know the side effect of bypassing and recover the 
inconsistent state of the cluster themselves, like issuing new procedures to 
assign the regions.

A patch will be uploaded and review board will be open. For now, only APIs in 
ProcedureExecutor are provided. If anything is fine, I will add it to master 
service and add a shell command to bypass a procedure. Or, maybe we can use 
dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. 




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-17621) Usability and aesthetic problems with the table view UI

2018-08-21 Thread Xu Cang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587181#comment-16587181
 ] 

Xu Cang commented on HBASE-17621:
-

sorting by full region name with start/stop key is fixed by this 
:https://issues.apache.org/jira/browse/HBASE-20487

> Usability and aesthetic problems with the table view UI
> ---
>
> Key: HBASE-17621
> URL: https://issues.apache.org/jira/browse/HBASE-17621
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.4.0, 2.0.0
>Reporter: Andrew Purtell
>Priority: Blocker
>
> HBASE-15675 changed the display and formatting of the table view in the 
> master UI. After this change by default regions are sorted lexiographically 
> by encoded region name, which has no correspondence with ordering in the 
> keyspace. Previously entries adjacent in the region list would be adjacent in 
> keyspace too. This made it easy to locate boundaries for split and merge 
> requests. This is a big step backwards for usability. For anyone who wants to 
> manage splits and merges (after HBASE-17609), unless they have memorized what 
> encoded names correspond to the adjacent regions of interest, they first have 
> to check the box named "ShowDetailName&Start/End Key" and resort the lists.
> Related, naming that element "ShowDetailName&Start/End Key" is very poor 
> aesthetics.
> Related, the checkboxes are right of the description text, which is an 
> unconventional design. In most UI designs, checkboxes appear left of the 
> description text.
>  
> Related, following that element is an inexplicable string "+ + + + +"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations

2018-08-21 Thread liubangchen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liubangchen updated HBASE-19008:

Attachment: HBASE-19008-16.patch

> Add missing equals or hashCode method(s) to stock Filter implementations
> 
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
>  Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch, 
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch, 
> HBASE-19008-14.patch, HBASE-19008-15.patch, HBASE-19008-16.patch, 
> HBASE-19008-2.patch, HBASE-19008-3.patch, HBASE-19008-4.patch, 
> HBASE-19008-5.patch, HBASE-19008-6.patch, HBASE-19008-7.patch, 
> HBASE-19008-8.patch, HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not 
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock 
> Filter implementations such as KeyOnlyFilter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations

2018-08-21 Thread Reid Chan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587123#comment-16587123
 ] 

Reid Chan commented on HBASE-19008:
---

Please upload the latest patch to review board.

> Add missing equals or hashCode method(s) to stock Filter implementations
> 
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
>  Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch, 
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch, 
> HBASE-19008-14.patch, HBASE-19008-15.patch, HBASE-19008-2.patch, 
> HBASE-19008-3.patch, HBASE-19008-4.patch, HBASE-19008-5.patch, 
> HBASE-19008-6.patch, HBASE-19008-7.patch, HBASE-19008-8.patch, 
> HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not 
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock 
> Filter implementations such as KeyOnlyFilter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations

2018-08-21 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587121#comment-16587121
 ] 

Ted Yu commented on HBASE-19008:


TestMergeTableRegionsProcedure is not related to Filter.

> Add missing equals or hashCode method(s) to stock Filter implementations
> 
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
>  Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch, 
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch, 
> HBASE-19008-14.patch, HBASE-19008-15.patch, HBASE-19008-2.patch, 
> HBASE-19008-3.patch, HBASE-19008-4.patch, HBASE-19008-5.patch, 
> HBASE-19008-6.patch, HBASE-19008-7.patch, HBASE-19008-8.patch, 
> HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not 
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock 
> Filter implementations such as KeyOnlyFilter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations

2018-08-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587118#comment-16587118
 ] 

Hadoop QA commented on HBASE-19008:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 5 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
11s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  3m 
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
37s{color} | {color:red} hbase-client: The patch generated 3 new + 376 
unchanged - 13 fixed = 379 total (was 389) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
18s{color} | {color:green} hbase-server: The patch generated 0 new + 404 
unchanged - 2 fixed = 404 total (was 406) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch hbase-spark passed checkstyle {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
14s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 26s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m  
5s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}116m 35s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m 
54s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
55s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 40s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Doc

[jira] [Commented] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587115#comment-16587115
 ] 

Toshihiro Suzuki commented on HBASE-21015:
--

It seems like it's conflicting with HBASE-20881. Looking into this conflict.

> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some date into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-20968) list_procedures_test fails due to no matching regex

2018-08-21 Thread Duo Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-20968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587112#comment-16587112
 ] 

Duo Zhang commented on HBASE-20968:
---

Add the test back for branch-2.0 and branch-2.1. [~stack] Could you please 
check whether this works for you?

Thanks.

> list_procedures_test fails due to no matching regex
> ---
>
> Key: HBASE-20968
> URL: https://issues.apache.org/jira/browse/HBASE-20968
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20968.v2.txt, HBASE-20968-branch-2.1.patch, 
> HBASE-20968.patch, org.apache.hadoop.hbase.client.TestShell-output.txt
>
>
> From test output against hadoop3:
> {code}
> 2018-07-28 12:04:24,838 DEBUG [Time-limited test] 
> procedure2.ProcedureExecutor(948): Stored pid=12, state=RUNNABLE, 
> hasLock=false; org.apache.hadoop.hbase.client.procedure.  
> ShellTestProcedure
> 2018-07-28 12:04:24,864 INFO  [RS-EventLoopGroup-1-3] 
> ipc.ServerRpcConnection(556): Connection from 172.18.128.12:46918, 
> version=3.0.0-SNAPSHOT, sasl=false, ugi=hbase (auth: SIMPLE), 
> service=MasterService
> 2018-07-28 12:04:24,900 DEBUG [Thread-114] master.MasterRpcServices(1157): 
> Checking to see if procedure is done pid=11
> ^[[38;5;196mF^[[0m
> ===
> Failure: 
> ^[[48;5;124;38;5;231;1mtest_list_procedures(Hbase::ListProceduresTest)^[[0m
> src/test/ruby/shell/list_procedures_test.rb:65:in `block in 
> test_list_procedures'
>  62: end
>  63:   end
>  64:
> ^[[48;5;124;38;5;231;1m  => 65:   assert_equal(1, matching_lines)^[[0m
>  66: end
>  67:   end
>  68: end
> <^[[48;5;34;38;5;231;1m1^[[0m> expected but was
> <^[[48;5;124;38;5;231;1m0^[[0m>
> ===
> ...
> 2018-07-28 12:04:25,374 INFO  [PEWorker-9] 
> procedure2.ProcedureExecutor(1316): Finished pid=12, state=SUCCESS, 
> hasLock=false; org.apache.hadoop.hbase.client.procedure.   
> ShellTestProcedure in 336msec
> {code}
> The completion of the ShellTestProcedure was after the assertion was raised.
> {code}
> def create_procedure_regexp(table_name)
>   regexp_string = '[0-9]+ .*ShellTestProcedure SUCCESS.*' \
> {code}
> The regex used by the test isn't found in test output either.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (HBASE-21082) Reimplement assign/unassign related procedure metrics

2018-08-21 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-21082:
-

 Summary: Reimplement assign/unassign related procedure metrics
 Key: HBASE-21082
 URL: https://issues.apache.org/jira/browse/HBASE-21082
 Project: HBase
  Issue Type: Sub-task
  Components: amv2, metrics
Reporter: Duo Zhang
 Fix For: 3.0.0, 2.2.0


As after HBASE-20881, we only have one TRSP procedure to handle all the 
assign/unassign/move/reopen operations, we need to reimplement(redefine?) the 
metrics for these procedures.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21071) HBaseTestingUtility::startMiniCluster() to use builder pattern

2018-08-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587096#comment-16587096
 ] 

Hadoop QA commented on HBASE-21071:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
21s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
15s{color} | {color:red} hbase-server: The patch generated 2 new + 228 
unchanged - 47 fixed = 230 total (was 275) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
17s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 46s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m 
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
23s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 48s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21071 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12936378/HBASE-21071.004.patch 
|
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 5771a75c3856 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / bb3494134e |
| maven | version: Apache Maven 3.5.4 
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14103/artifact/patchprocess/diff-checkstyle-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14103/testReport/ |
| Max. process+thread count | 4867 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
http

[jira] [Commented] (HBASE-20968) list_procedures_test fails due to no matching regex

2018-08-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-20968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587084#comment-16587084
 ] 

Hadoop QA commented on HBASE-20968:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
 6s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red}  0m  
3s{color} | {color:red} The patch generated 6 new + 0 unchanged - 0 fixed = 6 
total (was 0) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}  
0m  1s{color} | {color:orange} The patch generated 14 new + 0 unchanged - 0 
fixed = 14 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 
46s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 47s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-20968 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12936388/HBASE-20968-branch-2.1.patch
 |
| Optional Tests |  asflicense  unit  rubocop  ruby_lint  |
| uname | Linux d2eb587929ab 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | branch-2.1 / 798cb1d793 |
| maven | version: Apache Maven 3.5.4 
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| rubocop | v0.58.2 |
| rubocop | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14106/artifact/patchprocess/diff-patch-rubocop.txt
 |
| ruby-lint | v2.3.1 |
| ruby-lint | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14106/artifact/patchprocess/diff-patch-ruby-lint.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14106/testReport/ |
| Max. process+thread count | 2168 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14106/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> list_procedures_test fails due to no matching regex
> ---
>
> Key: HBASE-20968
> URL: https://issues.apache.org/jira/browse/HBASE-20968
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20968.v2.txt, HBASE-20968-branch-2.1.patch, 
> HBASE-20968.patch, org.apache.hadoop.hbase.client.TestShell-output.txt
>
>
> From test output against hadoop3:
> {code}
> 2018-07-28 12:04:24,838 DEBUG [Time-limited test] 
> procedure2.ProcedureExecutor(948): Stored pid=12, state=RUNNABLE, 
> hasLock=false; org.apache.hadoop.hbase.client.procedure.  
> ShellTestProcedure
> 2018-07-28 12:04:24,864 INFO  [RS-EventLoopGroup-1-3] 
> ipc.ServerRpcConnection(556): Connection from 172.18.128.12:46918, 
> version=3.0.0-SNAPSHOT, sasl=false, ugi=hbase (auth: SIMPLE), 
> service=MasterService
> 2018-07-28 12:04:24,900 DEBUG [Thread-114] master.MasterRpcServices(1157): 
> Checking to see if procedure is done pid=11
> ^[[38;5;196mF^[[0m
> ===
> Failure: 
> ^[[48;5;124;38;5;231;1mtest_list_procedures(Hbase::ListProceduresTest)^[[0m
> src/test/ruby/shell/list_procedures_test.rb:65:in `block in 
> test_list_procedures'
>  62: end
>  63:   end
>  64:
> ^[[48;5;124;38;5;231;1m  => 65:   assert_equal(1, 

[jira] [Commented] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587074#comment-16587074
 ] 

Hadoop QA commented on HBASE-21015:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  5s{color} 
| {color:red} HBASE-21015 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-21015 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12936392/HBASE-21015.master.001.patch
 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/14107/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some date into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-21078) [amv2] CODE-BUG NPE in RTP doing Unassign

2018-08-21 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587072#comment-16587072
 ] 

stack commented on HBASE-21078:
---

Checking I didn't have dirty env. Will be back.

> [amv2] CODE-BUG NPE in RTP doing Unassign
> -
>
> Key: HBASE-21078
> URL: https://issues.apache.org/jira/browse/HBASE-21078
> Project: HBase
>  Issue Type: Bug
>  Components: amv2
>Affects Versions: 2.0.1
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.2
>
>
> Saw this is a run against tip of branch-2.0. The region had just finished 
> being split when the move goes to run.
> {code}
> 2018-08-18 16:55:14,908 INFO  [PEWorker-2] procedure2.ProcedureExecutor: 
> Finished pid=2028, state=SUCCESS, hasLock=false; SplitTableRegionProcedure 
> table=IntegrationTestBigLinkedList, parent=c3f199b5af62ae2ff8f8b6426b21d95d, 
> daughterA=31ccbf098ae615ce30f28ec84c956b8f, 
> daughterB=1890b4c96736f223f31efef11c817c90 in 9.0090sec
> 2018-08-18 16:55:14,908 INFO  [PEWorker-16] 
> procedure.MasterProcedureScheduler: pid=2038, ppid=2030, 
> state=RUNNABLE:MOVE_REGION_UNASSIGN, hasLock=false; MoveRegionProcedure 
> hri=c3f199b5af62ae2ff8f8b6426b21d95d, 
> source=ve0540.halxg.cloudera.com,16020,1534632630737, 
> destination=ve0540.halxg.cloudera.com,16020,1534632630737 checking lock on 
> c3f199b5af62ae2ff8f8b6426b21d95d
> 2018-08-18 16:55:14,958 INFO  [PEWorker-16] procedure2.ProcedureExecutor: 
> Initialized subprocedures=[{pid=2095, ppid=2038, 
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=false; UnassignProcedure 
> table=IntegrationTestBigLinkedList, region=c3f199b5af62ae2ff8f8b6426b21d95d, 
> server=ve0540.halxg.cloudera.com,16020,1534632630737}]
> 2018-08-18 16:55:15,008 INFO  [PEWorker-3] 
> procedure.MasterProcedureScheduler: pid=2095, ppid=2038, 
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=false; UnassignProcedure 
> table=IntegrationTestBigLinkedList, region=c3f199b5af62ae2ff8f8b6426b21d95d, 
> server=ve0540.halxg.cloudera.com,16020,1534632630737 checking lock on 
> c3f199b5af62ae2ff8f8b6426b21d95d
> 2018-08-18 16:55:15,085 ERROR [PEWorker-3] procedure2.ProcedureExecutor: 
> CODE-BUG: Uncaught runtime exception: pid=2095, ppid=2038, 
> state=RUNNABLE:REGION_TRANSITION_DISPATCH, hasLock=true; UnassignProcedure 
> table=IntegrationTestBigLinkedList, region=c3f199b5af62ae2ff8f8b6426b21d95d, 
> server=ve0540.halxg.cloudera.com,16020,1534632630737
> java.lang.NullPointerException
>   at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionStates.getOrCreateServer(RegionStates.java:1097)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionStates.addRegionToServer(RegionStates.java:1125)
>   at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsClosing(AssignmentManager.java:1477)
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:204)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:345)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:97)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:873)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1556)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1344)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1854)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work stopped] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-21015 stopped by Toshihiro Suzuki.

> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some date into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Toshihiro Suzuki updated HBASE-21015:
-
Status: Patch Available  (was: Open)

> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some date into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work started] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-21015 started by Toshihiro Suzuki.

> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some date into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HBASE-21015) The state of the split parent region is wrong after cloning a snapshot

2018-08-21 Thread Toshihiro Suzuki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Toshihiro Suzuki updated HBASE-21015:
-
Attachment: HBASE-21015.master.001.patch

> The state of the split parent region is wrong after cloning a snapshot
> --
>
> Key: HBASE-21015
> URL: https://issues.apache.org/jira/browse/HBASE-21015
> Project: HBase
>  Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Attachments: HBASE-21015.master.001.patch, Screen Shot 2018-08-07 at 
> 00.40.40.png
>
>
> Steps to reproduce are as follows:
> 1. Create a table
> {code}
> create "test", "cf"
> {code}
> 2. Put some date into the table
> {code}
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> {code}
> 3. Split the table
> {code}
> split "test"
> {code}
> 4. Take a snapshot of the table before CatalogJanitor cleans up the split 
> parent region
> {code}
> snapshot "test", "snap"
> {code}
> 5. Clone the snapshot
> {code}
> clone_snapshot "snap", "cloned_table"
> {code}
> After cloning the snapshot, the state of the split parent region is OFFLINE, 
> but it should be SPLIT.
>  !Screen Shot 2018-08-07 at 00.40.40.png! 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work

2018-08-21 Thread Jack Bearden (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587055#comment-16587055
 ] 

Jack Bearden commented on HBASE-20993:
--

The netty client needs it's sasl negotiate handler updated to react to the new 
response. I'm working on it

> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
>  Issue Type: Bug
>  Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, 
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch, 
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch, 
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch, 
> HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos, 
> hbase.security.authentication:kerberos, 
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal 
> are right set
> A simple auth hbase cluster, a kerberized hbase client application. 
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by 
> GSSException: No valid credentials provided (Mechanism level: Failed to find 
> any Kerberos tgt)]
>   at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
>   at 
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
>   at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
>   at 
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
>   at 
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
>   at 
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
>   at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
>   at 
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
>   at 
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
>   at 
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
>   at 
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
>   at 
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
>   at 
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
>   at 
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
>   at 
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level: 
> Failed to find any Kerberos tgt)
>   at 
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5I

<    1   2   3