date:20200110



[ 
https://issues.apache.org/jira/browse/HBASE-23286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012543#comment-17012543
 ] 

Guanghao Zhang commented on HBASE-23286:


{quote}bq.  I think [~zghao] has already ported the patch to our internal 
branch-2 based branch and tested the feature?
{quote}
Yes. I tested this on our internal branch.

> Improve MTTR: Split WAL to HFile
> 
>
> Key: HBASE-23286
> URL: https://issues.apache.org/jira/browse/HBASE-23286
> Project: HBase
>  Issue Type: Improvement
>  Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> After HBASE-20724, the compaction event marker is not used anymore when 
> failover. So our new proposal is split WAL to HFile to imporve MTTR. It has 3 
> steps:
>  # Read WAL and write HFile to region’s column family’s recovered.hfiles 
> directory.
>  # Open region.
>  # Bulkload the recovered.hfiles for every column family.
> The design doc was attathed by a google doc. Any suggestions are welcomed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23659) BaseLoadBalancer#wouldLowerAvailability should consider region replicas



[ 
https://issues.apache.org/jira/browse/HBASE-23659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012552#comment-17012552
 ] 

Guanghao Zhang commented on HBASE-23659:


Pushed to branch-2.2+. Thanks all for reviewing.

> BaseLoadBalancer#wouldLowerAvailability should consider region replicas
> ---
>
> Key: HBASE-23659
> URL: https://issues.apache.org/jira/browse/HBASE-23659
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
> Found this issue when try to fix the flaky unit test  TestRegionReplicaSplit. 
> It may fail as 
> java.lang.AssertionError: Splitted regions should not be assigned to same 
> region server.
> See 
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.assignment/TestRegionReplicaSplit/testRegionReplicaSplitRegionAssignment/].
>  
> Now  wouldLowerAvailability method only consider the primary regions. The 
> replica region can't assign to same server with primary region. But can be 
> assigned to same server with other replica regions. 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23659) BaseLoadBalancer#wouldLowerAvailability should consider region replicas



 [ 
https://issues.apache.org/jira/browse/HBASE-23659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-23659:
---
Fix Version/s: 2.2.3
   2.3.0
   3.0.0

> BaseLoadBalancer#wouldLowerAvailability should consider region replicas
> ---
>
> Key: HBASE-23659
> URL: https://issues.apache.org/jira/browse/HBASE-23659
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> Found this issue when try to fix the flaky unit test  TestRegionReplicaSplit. 
> It may fail as 
> java.lang.AssertionError: Splitted regions should not be assigned to same 
> region server.
> See 
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.assignment/TestRegionReplicaSplit/testRegionReplicaSplitRegionAssignment/].
>  
> Now  wouldLowerAvailability method only consider the primary regions. The 
> replica region can't assign to same server with primary region. But can be 
> assigned to same server with other replica regions. 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23601) OutputSink.WriterThread exception gets stuck and repeated indefinietly

2020-01-10 Thread Szabolcs Bukros (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012557#comment-17012557
 ] 

Szabolcs Bukros commented on HBASE-23601:
-

[~stack] Thanks for the merge! This doesn't need a master patch. The code there 
was heavily rewritten and as far as I can tell it's not affected by this issue. 
I'll check why it fails on branch-2.

> OutputSink.WriterThread exception gets stuck and repeated indefinietly
> --
>
> Key: HBASE-23601
> URL: https://issues.apache.org/jira/browse/HBASE-23601
> Project: HBase
>  Issue Type: Bug
>  Components: read replicas
>Affects Versions: 2.2.2
>Reporter: Szabolcs Bukros
>Assignee: Szabolcs Bukros
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.9, 2.2.4
>
>
> When a WriterThread runs into an exception (ie: NotServingRegionException), 
> the exception is stored in the controller. It is never removed and can not be 
> overwritten either.
>  
> {code:java}
> public void run()  {
>   try {
> doRun();
>   } catch (Throwable t) {
> LOG.error("Exiting thread", t);
> controller.writerThreadError(t);
>   }
> }{code}
> Thanks to this every time PipelineController.checkForErrors() is called the 
> same old exception is rethrown.
>  
> For example in RegionReplicaReplicationEndpoint.replicate there is a while 
> loop that does the actual replicating. Every time it loops, it calls 
> checkForErrors(), catches the rethrown exception, logs it but does nothing 
> about it. This results in ~2GB log files in ~5min in my experience.
>  
> My proposal would be to clean up the stored exception when it reaches 
> RegionReplicaReplicationEndpoint.replicate and make sure we restart the 
> WriterThread that died throwing it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23673) org.apache.hadoop.hbase.ipc.RpcServer.Listener#closeCurrentConnection did not close the connection properly

2020-01-10 Thread Baiqiang Zhao (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012560#comment-17012560
 ] 

Baiqiang Zhao commented on HBASE-23673:
---

Ping [~stack]

> org.apache.hadoop.hbase.ipc.RpcServer.Listener#closeCurrentConnection did not 
> close the connection properly
> ---
>
> Key: HBASE-23673
> URL: https://issues.apache.org/jira/browse/HBASE-23673
> Project: HBase
>  Issue Type: Bug
>  Components: IPC/RPC
>Affects Versions: 1.6.0, 1.4.12
>Reporter: Baiqiang Zhao
>Priority: Major
>
> The connection is attached to readKey, but the parameter passed into the 
> closeCurrentConnection is acceptedKey. branch-2 is fixed in HBASE-15948, 
> though 2.0+ already uses netty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish



 [ 
https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-23655.

Resolution: Fixed

> Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
> ---
>
> Key: HBASE-23655
> URL: https://issues.apache.org/jira/browse/HBASE-23655
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.2.2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail 
> if not wait SCP to finish.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23659) BaseLoadBalancer#wouldLowerAvailability should consider region replicas



 [ 
https://issues.apache.org/jira/browse/HBASE-23659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-23659.

Resolution: Fixed

> BaseLoadBalancer#wouldLowerAvailability should consider region replicas
> ---
>
> Key: HBASE-23659
> URL: https://issues.apache.org/jira/browse/HBASE-23659
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> Found this issue when try to fix the flaky unit test  TestRegionReplicaSplit. 
> It may fail as 
> java.lang.AssertionError: Splitted regions should not be assigned to same 
> region server.
> See 
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.assignment/TestRegionReplicaSplit/testRegionReplicaSplitRegionAssignment/].
>  
> Now  wouldLowerAvailability method only consider the primary regions. The 
> replica region can't assign to same server with primary region. But can be 
> assigned to same server with other replica regions. 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23658) Fix flaky TestSnapshotFromMaster



 [ 
https://issues.apache.org/jira/browse/HBASE-23658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-23658.

Resolution: Fixed

> Fix flaky TestSnapshotFromMaster
> 
>
> Key: HBASE-23658
> URL: https://issues.apache.org/jira/browse/HBASE-23658
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
> Attachments: HBASE-23658.branch-2.2.001.patch
>
>
> testAsyncSnapshotWillNotBlockSnapshotHFileCleaner is flaky.  The assert may 
> fail.
> {code:java}
> assertTrue(master.getSnapshotManager().isTakingAnySnapshot());
> future.get(); // in branch-2.2, here is Thread.sleep
> assertFalse(master.getSnapshotManager().isTakingAnySnapshot());
> {code}
> See 
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testAsyncSnapshotWillNotBlockSnapshotHFileCleaner/]
>  
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2.2/lastSuccessfulBuild/artifact/dashboard.html]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23663) Allow dot and hyphen in Profiler's URL



 [ 
https://issues.apache.org/jira/browse/HBASE-23663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-23663:
---
Fix Version/s: (was: 2.2.4)
   2.2.3

> Allow dot and hyphen in Profiler's URL
> --
>
> Key: HBASE-23663
> URL: https://issues.apache.org/jira/browse/HBASE-23663
> Project: HBase
>  Issue Type: Bug
>  Components: profiler
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> ProfileOutputServlet sanitizes the input URL but fails when the query string 
> contains dot or hyphen. These are valid characters for the hostname.
> Example URL part:
> /prof-output/async-prof-pid-122466-cpu-1.svg?host=myhost-1.example.com&port=16010
>  
> In this case the user gets the following error message:
> {noformat}
> HTTP ERROR: 500
> Problem accessing /prof-output/async-prof-pid-122466-cpu-1.svg. Reason:
> java.lang.RuntimeException: Non-alphanumeric data found in input, 
> aborting.
> Powered by Jetty:// 9.3.27.v20190418 {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23651) Region balance throttling can be disabled



 [ 
https://issues.apache.org/jira/browse/HBASE-23651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-23651:
---
Fix Version/s: (was: 2.2.4)
   2.2.3

> Region balance throttling can be disabled
> -
>
> Key: HBASE-23651
> URL: https://issues.apache.org/jira/browse/HBASE-23651
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.2.3
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> HBASE-17178 Add region balance throttling, but it can not be disabled, 
> sometimes we need no throttle and balance the cluster as fast as possible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23666) Backport "HBASE-23660 hbase:meta's table.jsp ref to wrong rs address" to branch-2



 [ 
https://issues.apache.org/jira/browse/HBASE-23666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-23666:
---
Fix Version/s: (was: 2.2.3)

> Backport "HBASE-23660 hbase:meta's table.jsp ref to wrong rs address" to 
> branch-2
> -
>
> Key: HBASE-23666
> URL: https://issues.apache.org/jira/browse/HBASE-23666
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 2.2.2
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
> Fix For: 2.3.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23666) Backport "HBASE-23660 hbase:meta's table.jsp ref to wrong rs address" to branch-2



 [ 
https://issues.apache.org/jira/browse/HBASE-23666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-23666:
---
Fix Version/s: 2.2.3

> Backport "HBASE-23660 hbase:meta's table.jsp ref to wrong rs address" to 
> branch-2
> -
>
> Key: HBASE-23666
> URL: https://issues.apache.org/jira/browse/HBASE-23666
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 2.2.2
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
> Fix For: 2.3.0, 2.2.3
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23601) OutputSink.WriterThread exception gets stuck and repeated indefinietly



[ 
https://issues.apache.org/jira/browse/HBASE-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012637#comment-17012637
 ] 

Guanghao Zhang commented on HBASE-23601:


I plan to release 2.2.3 now. Let me revert it from branch-2.2 as this not 
resolved. Will merge this after release. Thanks.

> OutputSink.WriterThread exception gets stuck and repeated indefinietly
> --
>
> Key: HBASE-23601
> URL: https://issues.apache.org/jira/browse/HBASE-23601
> Project: HBase
>  Issue Type: Bug
>  Components: read replicas
>Affects Versions: 2.2.2
>Reporter: Szabolcs Bukros
>Assignee: Szabolcs Bukros
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.9, 2.2.4
>
>
> When a WriterThread runs into an exception (ie: NotServingRegionException), 
> the exception is stored in the controller. It is never removed and can not be 
> overwritten either.
>  
> {code:java}
> public void run()  {
>   try {
> doRun();
>   } catch (Throwable t) {
> LOG.error("Exiting thread", t);
> controller.writerThreadError(t);
>   }
> }{code}
> Thanks to this every time PipelineController.checkForErrors() is called the 
> same old exception is rethrown.
>  
> For example in RegionReplicaReplicationEndpoint.replicate there is a while 
> loop that does the actual replicating. Every time it loops, it calls 
> checkForErrors(), catches the rethrown exception, logs it but does nothing 
> about it. This results in ~2GB log files in ~5min in my experience.
>  
> My proposal would be to clean up the stored exception when it reaches 
> RegionReplicaReplicationEndpoint.replicate and make sure we restart the 
> WriterThread that died throwing it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] infraio merged pull request #992: HBASE-23637 Generate CHANGES.md and RELEASENOTES.md for 2.2.3 (addendum)

infraio merged pull request #992: HBASE-23637 Generate CHANGES.md and 
RELEASENOTES.md for 2.2.3 (addendum)
URL: https://github.com/apache/hbase/pull/992
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #992: HBASE-23637 Generate CHANGES.md and RELEASENOTES.md for 2.2.3 (addendum)

Apache-HBase commented on issue #992: HBASE-23637 Generate CHANGES.md and 
RELEASENOTES.md for 2.2.3 (addendum)
URL: https://github.com/apache/hbase/pull/992#issuecomment-572974195
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m  0s |  Docker mode activated.  |
   | -1 :x: |  patch  |   0m  7s |  https://github.com/apache/hbase/pull/992 
does not apply to branch-2.2. Rebase required? Wrong Branch? See 
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for 
help.  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | GITHUB PR | https://github.com/apache/hbase/pull/992 |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-992/2/console |
   | versions | git=2.17.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1015: HBASE-23668 Master log start filling with "Flush journal status" messages

Apache-HBase commented on issue #1015: HBASE-23668  Master log start filling 
with "Flush journal status" messages
URL: https://github.com/apache/hbase/pull/1015#issuecomment-572995230
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 14s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -0 :warning: |  test4tests  |   0m  0s |  The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch.  |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 52s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 34s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 35s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   3m 51s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   3m 50s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 35s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  1s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  1s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 33s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 38s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  16m 57s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 35s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   3m 57s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 155m 55s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 28s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 216m 10s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1015/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1015 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux d222a57dc4c4 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 
05:24:09 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1015/out/precommit/personality/provided.sh
 |
   | git revision | branch-2 / 1217d57492 |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1015/2/testReport/
 |
   | Max. process+thread count | 4606 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1015/2/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (HBASE-23674) Too many rit page Numbers show confusion

2020-01-10 Thread dingbaosheng (Jira)

dingbaosheng created HBASE-23674:


 Summary: Too many rit page Numbers show confusion
 Key: HBASE-23674
 URL: https://issues.apache.org/jira/browse/HBASE-23674
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 3.0.0
Reporter: dingbaosheng
 Fix For: 3.0.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] mygood opened a new pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern

mygood opened a new pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support 
Merge region by pattern
URL: https://github.com/apache/hbase/pull/1016
 
 
   Design Objective:
1. Merge empty region
2. Neat region
3. merge expired region
   Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] 
[--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] 
[--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=]
   Options:
   --h or --h print help
   --tableName table name must be not null
   --startRegion start region
   --stopRegion stop region
   --maxRegionSize max region size Unit GB
   --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss
   --numMaxMergePlans num MaxMerge Plans
   --targetRegionCount target Region Count
   --configMergePauseTime config Merge Pause Time In milliseconds
   --printExecutionPlan Value default is true print execution plans false is 
execution merge
   
   Examples:
   bin/hbase onlinemerge --tableName=test:test1 
--startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. 
--stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f.
 --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss 
--numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern

2020-01-10 Thread zhengsicheng (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhengsicheng updated HBASE-23656:
-
Attachment: HBASE-23656.branch-1.4.v1.patch

> [MERGETOOL] HBASE Support Merge region by pattern
> -
>
> Key: HBASE-23656
> URL: https://issues.apache.org/jira/browse/HBASE-23656
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Reporter: zhengsicheng
>Assignee: zhengsicheng
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-23656.branch-1.4.v1.patch, 
> HBASE-23656.master.v1.patch
>
>
> Design Objective:
>  # Merge empty region
>  # Neat region
>  # merge expired region
> Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] 
> [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] 
> [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=]
> Options:
>  --h or --h print help
>  --tableName table name must be not null
>  --startRegion start region
>  --stopRegion stop region
>  --maxRegionSize max region size Unit GB
>  --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss
>  --numMaxMergePlans num MaxMerge Plans
>  --targetRegionCount target Region Count
>  --configMergePauseTime config Merge Pause Time In milliseconds
>  --printExecutionPlan Value default is true print execution plans false is 
> execution merge
> Examples:
>  bin/hbase onlinemerge --tableName=test:test1 
> --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. 
> --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f.
>  --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss 
> --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern

2020-01-10 Thread zhengsicheng (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhengsicheng updated HBASE-23656:
-
Fix Version/s: 1.4.13

> [MERGETOOL] HBASE Support Merge region by pattern
> -
>
> Key: HBASE-23656
> URL: https://issues.apache.org/jira/browse/HBASE-23656
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Reporter: zhengsicheng
>Assignee: zhengsicheng
>Priority: Minor
> Fix For: 3.0.0, 1.4.13
>
> Attachments: HBASE-23656.branch-1.4.v1.patch, 
> HBASE-23656.master.v1.patch
>
>
> Design Objective:
>  # Merge empty region
>  # Neat region
>  # merge expired region
> Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] 
> [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] 
> [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=]
> Options:
>  --h or --h print help
>  --tableName table name must be not null
>  --startRegion start region
>  --stopRegion stop region
>  --maxRegionSize max region size Unit GB
>  --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss
>  --numMaxMergePlans num MaxMerge Plans
>  --targetRegionCount target Region Count
>  --configMergePauseTime config Merge Pause Time In milliseconds
>  --printExecutionPlan Value default is true print execution plans false is 
> execution merge
> Examples:
>  bin/hbase onlinemerge --tableName=test:test1 
> --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. 
> --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f.
>  --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss 
> --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23591) Negative memStoreSizing

2020-01-10 Thread Szabolcs Bukros (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012853#comment-17012853
 ] 

Szabolcs Bukros commented on HBASE-23591:
-

[~anoop.hbase] You are right,  we would loose the file ref... Thanks for 
pointing it out!

What do you think about a solution where we send the last x (maybe 10?) 
HStoreFile paths in the FlushDescriptor instead of just the latest one? In the 
StoreFileManager the files are ordered by seqID so grabbing the last few is 
easy. We would have to make sure to filter out storedFiles already listed in 
the store of the replica region in replayFlush too.  This way the first 
successful flush would add all the potentially missing refs.

> Negative memStoreSizing
> ---
>
> Key: HBASE-23591
> URL: https://issues.apache.org/jira/browse/HBASE-23591
> Project: HBase
>  Issue Type: Bug
>  Components: read replicas
>Reporter: Szabolcs Bukros
>Priority: Major
> Fix For: 2.2.2
>
>
> After a flush on the replica region the memStoreSizing becomes negative:
> {code:java}
> 2019-12-17 08:31:59,983 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> 0beaae111b0f6e98bfde31ba35be5408 : Replaying flush marker action: 
> COMMIT_FLUSH table_name: "IntegrationTestRegionReplicaReplicati
> on" encoded_region_name: "544affde3e027454f67c8ea46c8f69ee" 
> flush_sequence_number: 41392 store_flushes { family_name: "f1" 
> store_home_dir: "f1" flush_output: "3c48a23eac784a348a18e10e337d80a2" } 
> store_flushes { family_name: "f2" store_home_dir: "f2" flush_output: 
> "9a5283ec95694667b4ead2398af5f01e" } store_flushes { family_name: "f3" 
> store_home_dir: "f3" flush_output: "e6f25e6b0eca4d22af15d0626d0f8759" } 
> region_name: 
> "IntegrationTestRegionReplicaReplication,,1576599911697.544affde3e027454f67c8ea46c8f69ee."
> 2019-12-17 08:31:59,984 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> 0beaae111b0f6e98bfde31ba35be5408 : Received a flush commit marker with 
> seqId:41392 and a previous prepared snapshot was found
> 2019-12-17 08:31:59,993 INFO org.apache.hadoop.hbase.regionserver.HStore: 
> Region: 0beaae111b0f6e98bfde31ba35be5408 added 
> hdfs://replica-1:8020/hbase/data/default/IntegrationTestRegionReplicaReplication/544affde3e027454f67c8ea46c8f69ee/f1/3c48a23eac784a348a18e10e337d80a2,
>  entries=32445, sequenceid=41392, filesize=27.6 M
> 2019-12-17 08:32:00,016 INFO org.apache.hadoop.hbase.regionserver.HStore: 
> Region: 0beaae111b0f6e98bfde31ba35be5408 added 
> hdfs://replica-1:8020/hbase/data/default/IntegrationTestRegionReplicaReplication/544affde3e027454f67c8ea46c8f69ee/f2/9a5283ec95694667b4ead2398af5f01e,
>  entries=12264, sequenceid=41392, filesize=10.9 M
> 2019-12-17 08:32:00,121 INFO org.apache.hadoop.hbase.regionserver.HStore: 
> Region: 0beaae111b0f6e98bfde31ba35be5408 added 
> hdfs://replica-1:8020/hbase/data/default/IntegrationTestRegionReplicaReplication/544affde3e027454f67c8ea46c8f69ee/f3/e6f25e6b0eca4d22af15d0626d0f8759,
>  entries=32379, sequenceid=41392, filesize=27.5 M
> 2019-12-17 08:32:00,122 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> CustomLog decrMemStoreSize. Current: dataSize=135810071, 
> getHeapSize=148400960, getOffHeapSize=0, getCellsCount=167243 delta: 
> dataSizeDelta=155923644, heapSizeDelta=170112320, offHeapSizeDelta=0, 
> cellsCountDelta=188399
> 2019-12-17 08:32:00,122 ERROR org.apache.hadoop.hbase.regionserver.HRegion: 
> Asked to modify this region's 
> (IntegrationTestRegionReplicaReplication,,1576599911697_0001.0beaae111b0f6e98bfde31ba35be54
> 08.) memStoreSizing to a negative value which is incorrect. Current 
> memStoreSizing=135810071, delta=-155923644
> java.lang.Exception
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.checkNegativeMemStoreDataSize(HRegion.java:1323)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.decrMemStoreSize(HRegion.java:1316)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.decrMemStoreSize(HRegion.java:1303)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALFlushCommitMarker(HRegion.java:5194)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.replayWALFlushMarker(HRegion.java:5025)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doReplayBatchOp(RSRpcServices.java:1143)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.replay(RSRpcServices.java:2232)
> at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:29754)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at 
> org.apache.hadoop.hb

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365221342
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
 ##
 @@ -58,8 +62,19 @@
 
   private final ZNodePaths znodePaths;
 
+  /**
+   * A znode maintained by MirroringTableStateManager.
+   * MirroringTableStateManager is deprecated to be removed in hbase3. It can 
also be disabled.
+   * Make sure it is enabled if you want to alter hbase:meta table in hbase2. 
In hbase3,
+   * TBD how metatable state will be hosted; likely on active hbase master.
 
 Review comment:
   So I do not think we should name this as mirrored, it is the original data. 
The state in master's memory, is a cache, actually.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365230905
 
 

 ##
 File path: 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseMetaEdit.java
 ##
 @@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+
+/**
+ * Test being able to edit hbase:meta.
+ */
+@Category({MiscTests.class, LargeTests.class})
+public class TestHBaseMetaEdit {
+  @ClassRule
+  public static final HBaseClassTestRule CLASS_RULE =
+  HBaseClassTestRule.forClass(TestHBaseMetaEdit.class);
+  @Rule
+  public TestName name = new TestName();
+  private final static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+
+  @Before
+  public void before() throws Exception {
+UTIL.startMiniCluster();
+  }
+
+  @After
+  public void after() throws Exception {
+UTIL.shutdownMiniCluster();
+  }
+
+  /**
+   * Set versions, set HBASE-16213 indexed block encoding, and add a column 
family.
+   * Verify they are all in place by looking at TableDescriptor AND by checking
+   * what the RegionServer sees after opening Region.
+   */
+  @Test
+  public void testEditMeta() throws IOException {
+Admin admin = UTIL.getAdmin();
+admin.tableExists(TableName.META_TABLE_NAME);
+admin.disableTable(TableName.META_TABLE_NAME);
+assertTrue(admin.isTableDisabled(TableName.META_TABLE_NAME));
+TableDescriptor descriptor = 
admin.getDescriptor(TableName.META_TABLE_NAME);
+ColumnFamilyDescriptor cfd = 
descriptor.getColumnFamily(HConstants.CATALOG_FAMILY);
+byte [] extraColumnFamilyName = Bytes.toBytes("xtra");
+ColumnFamilyDescriptor newCfd =
+
ColumnFamilyDescriptorBuilder.newBuilder(extraColumnFamilyName).build();
+int oldVersions = cfd.getMaxVersions();
+// Add '1' to current versions count.
+cfd = 
ColumnFamilyDescriptorBuilder.newBuilder(cfd).setMaxVersions(oldVersions + 1).
+setConfiguration(ColumnFamilyDescriptorBuilder.DATA_BLOCK_ENCODING,
+DataBlockEncoding.ROW_INDEX_V1.toString()).build();
+admin.modifyColumnFamily(TableName.META_TABLE_NAME, cfd);
+admin.addColumnFamily(TableName.META_TABLE_NAME, newCfd);
 
 Review comment:
   Now we allow users to add/remove families of meta table from client side? A 
bit dangerous. I think we should only allow a sub set of alter operations for 
meta table and system tables from client side? At least, we should not let them 
add or remove a column family, it will introduce very critical problems.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365221090
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
 ##
 @@ -58,8 +62,19 @@
 
   private final ZNodePaths znodePaths;
 
+  /**
+   * A znode maintained by MirroringTableStateManager.
+   * MirroringTableStateManager is deprecated to be removed in hbase3. It can 
also be disabled.
+   * Make sure it is enabled if you want to alter hbase:meta table in hbase2. 
In hbase3,
+   * TBD how metatable state will be hosted; likely on active hbase master.
 
 Review comment:
   I think the state will either be on zk or on hdfs? HMaster itself is state 
less...
   
   In general, in the current architecture, I think it the state should be 
placed on zk, of course you could cache it in master and let client go to 
master to ask for the state.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365233018
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
 ##
 @@ -25,51 +25,39 @@
 
 /**
  * Get, remove and modify table descriptors.
- * Used by servers to host descriptors.
  */
 @InterfaceAudience.Private
 public interface TableDescriptors {
   /**
-   * @param tableName
* @return TableDescriptor for tablename
-   * @throws IOException
*/
-  TableDescriptor get(final TableName tableName)
-  throws IOException;
+  TableDescriptor get(final TableName tableName) throws IOException;
 
   /**
* Get Map of all NamespaceDescriptors for a given namespace.
* @return Map of all descriptors.
-   * @throws IOException
*/
-  Map getByNamespace(String name)
-  throws IOException;
+  Map getByNamespace(String name) throws IOException;
 
   /**
* Get Map of all TableDescriptors. Populates the descriptor cache as a
* side effect.
* Notice: the key of map is the table name which contains namespace. It was 
generated by
* {@link TableName#getNameWithNamespaceInclAsString()}.
* @return Map of all descriptors.
-   * @throws IOException
*/
   Map getAll() throws IOException;
 
   /**
* Add or update descriptor
* @param htd Descriptor to set into TableDescriptors
-   * @throws IOException
*/
-  void add(final TableDescriptor htd)
-  throws IOException;
+  void add(final TableDescriptor htd) throws IOException;
 
 Review comment:
   This method is a bit odd. We do not call it in CreateTableProcedure, but 
only in ModifyTableProcedure. Why not just call it update?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365227172
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RejectReplicationRequestStateChecker.java
 ##
 @@ -1,4 +1,6 @@
-/**
+/*
+ * Copyright The Apache Software Foundation
 
 Review comment:
   What's this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365220125
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
 ##
 @@ -666,42 +667,55 @@ public void run(PRESP resp) {
   new DisableTableProcedureBiConsumer(tableName));
   }
 
+  /**
+   * Utility for completing passed TableState {@link CompletableFuture} 
future
+   * using passed parameters.
+   */
+  private static CompletableFuture completeCheckTableState(
 
 Review comment:
   What is the return value used for?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365233838
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
 ##
 @@ -135,16 +136,32 @@ public FSTableDescriptors(final Configuration conf, 
final FileSystem fs,
* TODO: This is a workaround. Should remove this ugly 
code...
*/
   public FSTableDescriptors(final Configuration conf, final FileSystem fs,
-final Path rootdir, final boolean fsreadonly, 
final boolean usecache,
-Function metaObserver) throws IOException {
+   final Path rootdir, final boolean fsreadonly, final boolean usecache,
+   Function metaObserver) 
throws IOException {
 this.fs = fs;
 this.rootdir = rootdir;
 this.fsreadonly = fsreadonly;
 this.usecache = usecache;
-this.metaTableDescriptor = metaObserver == null ? 
createMetaTableDescriptor(conf)
-  : metaObserver.apply(createMetaTableDescriptorBuilder(conf)).build();
+TableDescriptor td = null;
+try {
+  td = getTableDescriptorFromFs(fs, rootdir, TableName.META_TABLE_NAME);
+} catch (TableInfoMissingException e) {
+  td = metaObserver == null? createMetaTableDescriptor(conf):
+metaObserver.apply(createMetaTableDescriptorBuilder(conf)).build();
+  if (!fsreadonly) {
+LOG.info("Creating new hbase:meta table default descriptor/schema {}", 
td);
+updateTableDescriptor(td);
+  }
+}
+this.metaTableDescriptor = td;
   }
 
+  /**
+   *
+   * Should be private
+   * @deprecated Since 2.3.0. Should be for internal use only. Used by testing.
 
 Review comment:
   I do not think this should be deprecated? The class is IA.Private so it is 
OK to put internal methods here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365224489
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
 ##
 @@ -229,6 +244,43 @@ private void 
getMetaRegionLocation(CompletableFuture future,
 });
   }
 
+  @Override
+  public CompletableFuture getMetaTableState() {
+return getAndConvert(this.znodeMirroredMetaTableState, 
ZKAsyncRegistry::getTableState).
+  thenApply(state -> {
+return state == null || 
state.equals(ENABLED_META_TABLE_STATE.getState())?
+  ENABLED_META_TABLE_STATE: new TableState(TableName.META_TABLE_NAME, 
state);
+  }).exceptionally(e -> {
 
 Review comment:
   Currently in HBase, usually we will create a new CompletableFuture and use 
FutureUtils.addListener to complete it. The code in exceptionally are a bit 
tricky, where we throw a CompletionException...


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

Apache9 commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365226730
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
 ##
 @@ -54,8 +53,20 @@
 // TODO: Make this a guava Service
 @InterfaceAudience.Private
 public class TableStateManager {
-
   private static final Logger LOG = 
LoggerFactory.getLogger(TableStateManager.class);
+
+  /**
+   * All table state is kept in hbase:meta except that of hbase:meta itself.
 
 Review comment:
   In general I do not think this is a good design. We should persist it 
somewhere, and just cache it in master's memory. And we do not need to disable 
a table when altering any more? And the solution here just assume that we only 
have one meta region? So the altering operation can be atomic? What if we have 
multiple meta regions and we crash in the middle? I think this patch also aims 
to implement splittable meta in the future? No?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23055) Alter hbase:meta

2020-01-10 Thread Duo Zhang (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012871#comment-17012871
 ] 

Duo Zhang commented on HBASE-23055:
---

Replied on RB, I still do not think this is a good solution. We should design 
for the future, for splittable meta, not for a temporary solution. And I do not 
get the point why we jsut make the state for meta table volatile, why not just 
store it on zk?

And when reviewing the patch, a good thing is that, we do not reference the 
meta table descriptor everywhere in the code. We just reference the region info 
everywhere, which I think it is OK for now as the region info does not include 
the table descriptor. This can be done in the future when implementing 
splittable meta.

So I think we could get the patch in on master, or even on branch-2, but not 
the current one...

Thanks.

> Alter hbase:meta
> 
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
>  Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our 
> being able to set encodings such as the block-with-indexes which will help 
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first 
> step on road to being able to split meta.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23675) Move to Apache parent POM version 22

2020-01-10 Thread Peter Somogyi (Jira)

Peter Somogyi created HBASE-23675:
-

 Summary: Move to Apache parent POM version 22
 Key: HBASE-23675
 URL: https://issues.apache.org/jira/browse/HBASE-23675
 Project: HBase
  Issue Type: Improvement
  Components: dependencies
Affects Versions: 3.0.0, 2.3.0
Reporter: Peter Somogyi
Assignee: Peter Somogyi
 Fix For: 3.0.0, 2.3.0


Apache parent POM version 22 was released on 2020/01/09.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23663) Allow dot and hyphen in Profiler's URL



[ 
https://issues.apache.org/jira/browse/HBASE-23663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012884#comment-17012884
 ] 

Hudson commented on HBASE-23663:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Allow dot and hyphen in Profiler's URL
> --
>
> Key: HBASE-23663
> URL: https://issues.apache.org/jira/browse/HBASE-23663
> Project: HBase
>  Issue Type: Bug
>  Components: profiler
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> ProfileOutputServlet sanitizes the input URL but fails when the query string 
> contains dot or hyphen. These are valid characters for the hostname.
> Example URL part:
> /prof-output/async-prof-pid-122466-cpu-1.svg?host=myhost-1.example.com&port=16010
>  
> In this case the user gets the following error message:
> {noformat}
> HTTP ERROR: 500
> Problem accessing /prof-output/async-prof-pid-122466-cpu-1.svg. Reason:
> java.lang.RuntimeException: Non-alphanumeric data found in input, 
> aborting.
> Powered by Jetty:// 9.3.27.v20190418 {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23655) Fix flaky TestRSGroupsKillRS: should wait the SCP to finish



[ 
https://issues.apache.org/jira/browse/HBASE-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012885#comment-17012885
 ] 

Hudson commented on HBASE-23655:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Fix flaky TestRSGroupsKillRS: should wait the SCP to finish
> ---
>
> Key: HBASE-23655
> URL: https://issues.apache.org/jira/browse/HBASE-23655
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.2.2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> In teardownAfterMethod, it will moveTables and delete RSGroups. It will fail 
> if not wait SCP to finish.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23654) Please add Apache Trafodion and EsgynDB to "Powered by Apache HBase" page



[ 
https://issues.apache.org/jira/browse/HBASE-23654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012886#comment-17012886
 ] 

Hudson commented on HBASE-23654:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Please add Apache Trafodion and EsgynDB to "Powered by Apache HBase" page
> -
>
> Key: HBASE-23654
> URL: https://issues.apache.org/jira/browse/HBASE-23654
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Dave Birdsall
>Assignee: Beata Sudi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> Please add the following two entries to the web page "Powered By Apache 
> HBase™" ([https://hbase.apache.org/poweredbyhbase.html]):
> Trafodion (link [https://trafodion.apache.org/]) 
> Apache Trafodion(TM) is a webscale SQL-on-Hadoop solution enabling 
> transactional or operational workloads. It uses HBase as its storage engine 
> for SQL tables.
> EsgynDB (link [https://esgyn.com/])
> EsgynDB, powered by Apache Trafodion(TM), provides enterprise SQL on Hadoop. 
> It includes full ACID transactions, online transaction processing and online 
> analytic processing, along with enterprise features such as disaster recovery 
> and full backup/restore. Native tables are stored in HBase, but read and 
> write access to various other file formats such as Apache Parquet and ORC is 
> also supported. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23664) Upgrade JUnit to 4.13



[ 
https://issues.apache.org/jira/browse/HBASE-23664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012887#comment-17012887
 ] 

Hudson commented on HBASE-23664:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Upgrade JUnit to 4.13
> -
>
> Key: HBASE-23664
> URL: https://issues.apache.org/jira/browse/HBASE-23664
> Project: HBase
>  Issue Type: Task
>  Components: integration tests, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> New JUnit released a week ago. Let's give it a spin.
> https://github.com/junit-team/junit4/blob/master/doc/ReleaseNotes4.13.md



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23378) Clean Up FSUtil setClusterId



[ 
https://issues.apache.org/jira/browse/HBASE-23378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012889#comment-17012889
 ] 

Hudson commented on HBASE-23378:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Clean Up FSUtil setClusterId
> 
>
> Key: HBASE-23378
> URL: https://issues.apache.org/jira/browse/HBASE-23378
> Project: HBase
>  Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.0.0
>
>
> * Use try-with-resources
> * Remove bad practice of catching one's own Exceptions
> * Method signature 'wait' should be of type long to match JDK API
> * Add additional debugging
> * Do not swallow Interrupt status of thread
> * General cleanup



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-22285) A normalizer which merges very small size regions with adjacent regions.(MergeToNormalize)



[ 
https://issues.apache.org/jira/browse/HBASE-22285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012888#comment-17012888
 ] 

Hudson commented on HBASE-22285:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> A normalizer which merges very small size regions with adjacent 
> regions.(MergeToNormalize)
> --
>
> Key: HBASE-22285
> URL: https://issues.apache.org/jira/browse/HBASE-22285
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Reporter: Aman Poonia
>Assignee: Aman Poonia
>Priority: Minor
> Fix For: 3.0.0
>
>
> There are scenarios where we have seen around 5% of total regions with a size 
> of 0 bytes and another 5-6 % regions with size in a few bytes. These kinds of 
> regions increase with time considering we have TTL over the rows. 
> After exploring the option of RegionNormalizer and doing some quick runs we 
> found that that is not suitable considering it also splits the regions and 
> merges to normalize. What we really want is to split as per Split policy and 
> merge very small regions with adjacent regions to make sure we reduce 0-byte 
> regions.
> We can plugin this normalizer using the property 
> "hbase.master.normalizer.class"



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23055) Alter hbase:meta



[ 
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012890#comment-17012890
 ] 

Hudson commented on HBASE-23055:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Alter hbase:meta
> 
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
>  Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our 
> being able to set encodings such as the block-with-indexes which will help 
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first 
> step on road to being able to split meta.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23659) BaseLoadBalancer#wouldLowerAvailability should consider region replicas



[ 
https://issues.apache.org/jira/browse/HBASE-23659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012891#comment-17012891
 ] 

Hudson commented on HBASE-23659:


Results for branch master
[build #1594 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1594/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1594//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> BaseLoadBalancer#wouldLowerAvailability should consider region replicas
> ---
>
> Key: HBASE-23659
> URL: https://issues.apache.org/jira/browse/HBASE-23659
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> Found this issue when try to fix the flaky unit test  TestRegionReplicaSplit. 
> It may fail as 
> java.lang.AssertionError: Splitted regions should not be assigned to same 
> region server.
> See 
> [https://builds.apache.org/job/HBase-Flaky-Tests/job/master/5227/testReport/junit/org.apache.hadoop.hbase.master.assignment/TestRegionReplicaSplit/testRegionReplicaSplitRegionAssignment/].
>  
> Now  wouldLowerAvailability method only consider the primary regions. The 
> replica region can't assign to same server with primary region. But can be 
> assigned to same server with other replica regions. 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23055) Alter hbase:meta

2020-01-10 Thread Duo Zhang (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012892#comment-17012892
 ] 

Duo Zhang commented on HBASE-23055:
---

Or let's open a follow on issue, and set it as a blocker of 2.3.0, to finish 
the altering meta change, but persisting the state of meta table, and implement 
alter the alter meta table procedure.

> Alter hbase:meta
> 
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
>  Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our 
> being able to set encodings such as the block-with-indexes which will help 
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first 
> step on road to being able to split meta.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] petersomogyi opened a new pull request #1017: HBASE-23675 Move to Apache parent POM version 22

petersomogyi opened a new pull request #1017: HBASE-23675 Move to Apache parent 
POM version 22
URL: https://github.com/apache/hbase/pull/1017
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Work started] (HBASE-23675) Move to Apache parent POM version 22

2020-01-10 Thread Peter Somogyi (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-23675 started by Peter Somogyi.
-
> Move to Apache parent POM version 22
> 
>
> Key: HBASE-23675
> URL: https://issues.apache.org/jira/browse/HBASE-23675
> Project: HBase
>  Issue Type: Improvement
>  Components: dependencies
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Apache parent POM version 22 was released on 2020/01/09.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23652) Move the unsupported procedure type check before migrating to RegionProcedureStore

2020-01-10 Thread Duo Zhang (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-23652:
--
Summary: Move the unsupported procedure type check before migrating to 
RegionProcedureStore  (was: Sort out how to handle v2.1 procedures during 
upgrade)

> Move the unsupported procedure type check before migrating to 
> RegionProcedureStore
> --
>
> Key: HBASE-23652
> URL: https://issues.apache.org/jira/browse/HBASE-23652
> Project: HBase
>  Issue Type: Task
>  Components: master
>Affects Versions: 2.2.2
>Reporter: Nick Dimiduk
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
> Per 
> [discussion|https://lists.apache.org/thread.html/c5f960e6cf3c89ffbf8f8302afb121462a08fd9a89d384c6795a7b81%40%3Cdev.hbase.apache.org%3E]
>  on the list, let's figure out how to make the upgrade from a v2.1 procedure 
> store less error-prone. Could be a simple as documenting runbook steps to 
> execute during the rolling upgrade, but it would be nice if the software 
> could roll over the data versions gracefully.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache9 opened a new pull request #1018: HBASE-23652 Move the unsupported procedure type check before migratin…

Apache9 opened a new pull request #1018: HBASE-23652 Move the unsupported 
procedure type check before migratin…
URL: https://github.com/apache/hbase/pull/1018
 
 
   …g to RegionProcedureStore


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23601) OutputSink.WriterThread exception gets stuck and repeated indefinietly



[ 
https://issues.apache.org/jira/browse/HBASE-23601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17012947#comment-17012947
 ] 

Hudson commented on HBASE-23601:


Results for branch branch-2.1
[build #1768 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1768/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1768//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1768//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1768//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> OutputSink.WriterThread exception gets stuck and repeated indefinietly
> --
>
> Key: HBASE-23601
> URL: https://issues.apache.org/jira/browse/HBASE-23601
> Project: HBase
>  Issue Type: Bug
>  Components: read replicas
>Affects Versions: 2.2.2
>Reporter: Szabolcs Bukros
>Assignee: Szabolcs Bukros
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.9, 2.2.4
>
>
> When a WriterThread runs into an exception (ie: NotServingRegionException), 
> the exception is stored in the controller. It is never removed and can not be 
> overwritten either.
>  
> {code:java}
> public void run()  {
>   try {
> doRun();
>   } catch (Throwable t) {
> LOG.error("Exiting thread", t);
> controller.writerThreadError(t);
>   }
> }{code}
> Thanks to this every time PipelineController.checkForErrors() is called the 
> same old exception is rethrown.
>  
> For example in RegionReplicaReplicationEndpoint.replicate there is a while 
> loop that does the actual replicating. Every time it loops, it calls 
> checkForErrors(), catches the rethrown exception, logs it but does nothing 
> about it. This results in ~2GB log files in ~5min in my experience.
>  
> My proposal would be to clean up the stored exception when it reaches 
> RegionReplicaReplicationEndpoint.replicate and make sure we restart the 
> WriterThread that died throwing it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk merged pull request #1009: HBASE-23664 Upgrade JUnit to 4.13

ndimiduk merged pull request #1009: HBASE-23664 Upgrade JUnit to 4.13
URL: https://github.com/apache/hbase/pull/1009
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on issue #1010: HBASE-23664 Upgrade JUnit to 4.13

ndimiduk commented on issue #1010: HBASE-23664 Upgrade JUnit to 4.13
URL: https://github.com/apache/hbase/pull/1010#issuecomment-573078633
 
 
   That's a lot of failures. Kicking the PR job to try again.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365324010
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
 ##
 @@ -666,42 +667,55 @@ public void run(PRESP resp) {
   new DisableTableProcedureBiConsumer(tableName));
   }
 
+  /**
+   * Utility for completing passed TableState {@link CompletableFuture} 
future
+   * using passed parameters.
+   */
+  private static CompletableFuture completeCheckTableState(
 
 Review comment:
   Looks like this a holdover. I can make it void.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365330389
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
 ##
 @@ -58,8 +62,19 @@
 
   private final ZNodePaths znodePaths;
 
+  /**
+   * A znode maintained by MirroringTableStateManager.
+   * MirroringTableStateManager is deprecated to be removed in hbase3. It can 
also be disabled.
+   * Make sure it is enabled if you want to alter hbase:meta table in hbase2. 
In hbase3,
+   * TBD how metatable state will be hosted; likely on active hbase master.
 
 Review comment:
   The 'mirroring' feature predates this patch. This is an old facility this 
patch just makes use of. Changing this old features name is outside realm of 
this work.
   
   You get the bit that edit of meta is a rare, short-lived event and you get 
the bit that this implementation fails the state back to ENABLED if ever an 
issue, intentionally, to minimize our running into exotic situations. I don't 
want a permanently DISABLED state for hbase:meta, at least not at this stage in 
the game. We can do that later


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365331618
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
 ##
 @@ -229,6 +244,43 @@ private void 
getMetaRegionLocation(CompletableFuture future,
 });
   }
 
+  @Override
+  public CompletableFuture getMetaTableState() {
+return getAndConvert(this.znodeMirroredMetaTableState, 
ZKAsyncRegistry::getTableState).
+  thenApply(state -> {
+return state == null || 
state.equals(ENABLED_META_TABLE_STATE.getState())?
+  ENABLED_META_TABLE_STATE: new TableState(TableName.META_TABLE_NAME, 
state);
+  }).exceptionally(e -> {
 
 Review comment:
   Yeah, saw that. Here, if no znode -- probably because the mirroring table 
state manager is turned off -- then I want the return to be ENABLED (Completed 
CF). Should I change this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365337029
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
 ##
 @@ -54,8 +53,20 @@
 // TODO: Make this a guava Service
 @InterfaceAudience.Private
 public class TableStateManager {
-
   private static final Logger LOG = 
LoggerFactory.getLogger(TableStateManager.class);
+
+  /**
+   * All table state is kept in hbase:meta except that of hbase:meta itself.
 
 Review comment:
   bq. We should persist it somewhere, and just cache it in master's memory.
   
   I explain why state is intentionally transient for now so table falls-back 
to ENABLED if problem like Master crash during alter; its one less thing for 
the operator to decipher and hbck2 navigate starting up a cluster; in other 
words we fallback to the 'known' state rather than persist DISABLED meta, a new 
condition that is sure to have holes in it.
   
   bq. And the solution here just assume that we only have one meta region? 
   
   All of the code base presumes this, not just this patch (this patch is part 
work to undo this presumption letting go of hardcoded meta schema).
   
   bq. I think this patch also aims to implement splittable meta in the future? 
No?
   
   Splittable meta is another project, not this one.
   
   bq. And we do not need to disable a table when altering any more? 
   
   This patch makes hbase:meta dynamic. Maybe the above will work. Let me try.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365337458
 
 

 ##
 File path: 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseMetaEdit.java
 ##
 @@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
+import org.apache.hadoop.hbase.regionserver.Region;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+
+/**
+ * Test being able to edit hbase:meta.
+ */
+@Category({MiscTests.class, LargeTests.class})
+public class TestHBaseMetaEdit {
+  @ClassRule
+  public static final HBaseClassTestRule CLASS_RULE =
+  HBaseClassTestRule.forClass(TestHBaseMetaEdit.class);
+  @Rule
+  public TestName name = new TestName();
+  private final static HBaseTestingUtility UTIL = new HBaseTestingUtility();
+
+  @Before
+  public void before() throws Exception {
+UTIL.startMiniCluster();
+  }
+
+  @After
+  public void after() throws Exception {
+UTIL.shutdownMiniCluster();
+  }
+
+  /**
+   * Set versions, set HBASE-16213 indexed block encoding, and add a column 
family.
+   * Verify they are all in place by looking at TableDescriptor AND by checking
+   * what the RegionServer sees after opening Region.
+   */
+  @Test
+  public void testEditMeta() throws IOException {
+Admin admin = UTIL.getAdmin();
+admin.tableExists(TableName.META_TABLE_NAME);
+admin.disableTable(TableName.META_TABLE_NAME);
+assertTrue(admin.isTableDisabled(TableName.META_TABLE_NAME));
+TableDescriptor descriptor = 
admin.getDescriptor(TableName.META_TABLE_NAME);
+ColumnFamilyDescriptor cfd = 
descriptor.getColumnFamily(HConstants.CATALOG_FAMILY);
+byte [] extraColumnFamilyName = Bytes.toBytes("xtra");
+ColumnFamilyDescriptor newCfd =
+
ColumnFamilyDescriptorBuilder.newBuilder(extraColumnFamilyName).build();
+int oldVersions = cfd.getMaxVersions();
+// Add '1' to current versions count.
+cfd = 
ColumnFamilyDescriptorBuilder.newBuilder(cfd).setMaxVersions(oldVersions + 1).
+setConfiguration(ColumnFamilyDescriptorBuilder.DATA_BLOCK_ENCODING,
+DataBlockEncoding.ROW_INDEX_V1.toString()).build();
+admin.modifyColumnFamily(TableName.META_TABLE_NAME, cfd);
+admin.addColumnFamily(TableName.META_TABLE_NAME, newCfd);
 
 Review comment:
   This is a good point.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365337598
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
 ##
 @@ -25,51 +25,39 @@
 
 /**
  * Get, remove and modify table descriptors.
- * Used by servers to host descriptors.
  */
 @InterfaceAudience.Private
 public interface TableDescriptors {
   /**
-   * @param tableName
* @return TableDescriptor for tablename
-   * @throws IOException
*/
-  TableDescriptor get(final TableName tableName)
-  throws IOException;
+  TableDescriptor get(final TableName tableName) throws IOException;
 
   /**
* Get Map of all NamespaceDescriptors for a given namespace.
* @return Map of all descriptors.
-   * @throws IOException
*/
-  Map getByNamespace(String name)
-  throws IOException;
+  Map getByNamespace(String name) throws IOException;
 
   /**
* Get Map of all TableDescriptors. Populates the descriptor cache as a
* side effect.
* Notice: the key of map is the table name which contains namespace. It was 
generated by
* {@link TableName#getNameWithNamespaceInclAsString()}.
* @return Map of all descriptors.
-   * @throws IOException
*/
   Map getAll() throws IOException;
 
   /**
* Add or update descriptor
* @param htd Descriptor to set into TableDescriptors
-   * @throws IOException
*/
-  void add(final TableDescriptor htd)
-  throws IOException;
+  void add(final TableDescriptor htd) throws IOException;
 
 Review comment:
   Let me fix.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365338167
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
 ##
 @@ -135,16 +136,32 @@ public FSTableDescriptors(final Configuration conf, 
final FileSystem fs,
* TODO: This is a workaround. Should remove this ugly 
code...
*/
   public FSTableDescriptors(final Configuration conf, final FileSystem fs,
-final Path rootdir, final boolean fsreadonly, 
final boolean usecache,
-Function metaObserver) throws IOException {
+   final Path rootdir, final boolean fsreadonly, final boolean usecache,
+   Function metaObserver) 
throws IOException {
 this.fs = fs;
 this.rootdir = rootdir;
 this.fsreadonly = fsreadonly;
 this.usecache = usecache;
-this.metaTableDescriptor = metaObserver == null ? 
createMetaTableDescriptor(conf)
-  : metaObserver.apply(createMetaTableDescriptorBuilder(conf)).build();
+TableDescriptor td = null;
+try {
+  td = getTableDescriptorFromFs(fs, rootdir, TableName.META_TABLE_NAME);
+} catch (TableInfoMissingException e) {
+  td = metaObserver == null? createMetaTableDescriptor(conf):
+metaObserver.apply(createMetaTableDescriptorBuilder(conf)).build();
+  if (!fsreadonly) {
+LOG.info("Creating new hbase:meta table default descriptor/schema {}", 
td);
+updateTableDescriptor(td);
+  }
+}
+this.metaTableDescriptor = td;
   }
 
+  /**
+   *
+   * Should be private
+   * @deprecated Since 2.3.0. Should be for internal use only. Used by testing.
 
 Review comment:
   You are right. I wanted to take it private/protected but as you note, its 
Private so I can just do it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1011: HBASE-23055 Alter hbase:meta

saintstack commented on a change in pull request #1011: HBASE-23055 Alter 
hbase:meta
URL: https://github.com/apache/hbase/pull/1011#discussion_r365340349
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RejectReplicationRequestStateChecker.java
 ##
 @@ -1,4 +1,6 @@
-/**
+/*
+ * Copyright The Apache Software Foundation
 
 Review comment:
   We've had this discussion before. Its not javadoc so seems right to remove 
the extra '*'. We should do up an agreement on way to go. Did you say why you 
think we should preserve the '**'? Is it because its in the formatter?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack opened a new pull request #1011: HBASE-23055 Alter hbase:meta

saintstack opened a new pull request #1011: HBASE-23055 Alter hbase:meta
URL: https://github.com/apache/hbase/pull/1011

Make it so hbase:meta can be altered. TableState for hbase:meta
was hardcoded ENABLED. Make it dynamic. State is now kept in
current active Master. It is transient so falls back to default
if Master crashes. Add to registry a getMetaTableState
which reads mirrored state from zookeeper (NOT from Master and
defaults ENABLED if no implementation or error fetching state).
hbase:meta schema will be bootstrapped from the filesystem.
Changes to filesystem schema are atomic so we should be ok if
Master fails mid-edit (TBD). Undoes a bunch of guards that
prevented our being able to edit hbase:meta.

TODO: Tests, more clarity around hbase:meta table state, and undoing
references to hard-coded hbase:meta regioninfo.

M
hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Throw illegal access exception if you try to use MetaTableAccessor
getting state of the hbase:meta table.

M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
Add fetching of hbase:meta table state from registry. Adds cache of
tablestates w/ a ttl of 1 second (adjustable).

M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
Add querying registry for hbase:meta table state.

M
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
Add querying of mirrored table state for hbase:meta table.

M
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
Shutdown access.

M
hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
Just cleanup.

M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Add state holder for hbase:meta. Removed unused methods.

M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
Shut down access.

M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
Allow hbase:meta to be disabled.

M
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
Allow hbase:meta to be enabled.

Signed-off-by: Bharath Vissapragada

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on issue #1011: HBASE-23055 Alter hbase:meta

saintstack commented on issue #1011: HBASE-23055 Alter hbase:meta
URL: https://github.com/apache/hbase/pull/1011#issuecomment-573125703
 
 
   @Apache9 Thanks for the review. Let me open a follow-on to address helpful 
feedback (Boolean => Void, Preventing CF deletion, etc.).
   
   On the 'bad design', your objections seem a little out-of-place ('... 
presumes single meta region...') or easy to counter and your worries that this 
patch unsettles the codebase seem to pale compared to what has already gone 
into this minor branch. How to proceed? A discussion on dev list? Or would a 
write up on how this works help?  A one-pager? Let me know.
   
   Let me reopen this while we are commenting.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23055) Alter hbase:meta



[ 
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013079#comment-17013079
 ] 

Sean Busbey commented on HBASE-23055:
-

I agree that this change looks much lower risk than the procedure store 
refactoring. (I also want both of them on branch-2 if they're not in feature 
branches.)

> Alter hbase:meta
> 
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
>  Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our 
> being able to set encodings such as the block-with-indexes which will help 
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first 
> step on road to being able to split meta.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23055) Alter hbase:meta



[ 
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013083#comment-17013083
 ] 

Sean Busbey commented on HBASE-23055:
-

bq. Or let's open a follow on issue, and set it as a blocker of 2.3.0, to 
finish the altering meta change, but persisting the state of meta table, and 
implement alter the alter meta table procedure.

we're overdue on the 2.3.z release line starting. an issue for dev@, but maybe 
we should start RCs off an earlier point in the branch-2 history.

> Alter hbase:meta
> 
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
>  Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our 
> being able to set encodings such as the block-with-indexes which will help 
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first 
> step on road to being able to split meta.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23676) Address feedback on HBASE-23055 Alter hbase:meta.

Michael Stack created HBASE-23676:
-

 Summary: Address feedback on HBASE-23055 Alter hbase:meta.
 Key: HBASE-23676
 URL: https://issues.apache.org/jira/browse/HBASE-23676
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.3.0
Reporter: Michael Stack
Assignee: Michael Stack
 Fix For: 2.3.0


Good feedback on HBASE-23055 came in after merge from [~zhangduo]. Opening this 
issue to address it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern

Apache-HBase commented on issue #1016: HBASE-23656 [MERGETOOL] HBASE Support 
Merge region by pattern
URL: https://github.com/apache/hbase/pull/1016#issuecomment-573138221
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   2m 17s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +0 :ok: |  shelldocs  |   0m  0s |  Shelldocs was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
1 new or modified test files.  |
   ||| _ branch-1.4 Compile Tests _ |
   | +0 :ok: |  mvndep  |   1m 30s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   9m  0s |  branch-1.4 passed  |
   | +1 :green_heart: |  compile  |   3m  4s |  branch-1.4 passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  compile  |   2m 48s |  branch-1.4 passed with JDK 
v1.7.0_242  |
   | +1 :green_heart: |  checkstyle  |   8m 22s |  branch-1.4 passed  |
   | +1 :green_heart: |  shadedjars  |   2m 45s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 35s |  branch-1.4 passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  javadoc  |   3m 37s |  branch-1.4 passed with JDK 
v1.7.0_242  |
   | +0 :ok: |  spotbugs  |   2m 29s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |  13m  5s |  branch-1.4 passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 57s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 43s |  the patch passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  javac  |   1m 43s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 46s |  the patch passed with JDK 
v1.7.0_242  |
   | +1 :green_heart: |  javac  |   1m 46s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   6m 39s |  the patch passed  |
   | +1 :green_heart: |  shellcheck  |   0m  2s |  There were no new shellcheck 
issues.  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   2m 44s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   2m 32s |  Patch does not cause any 
errors with Hadoop 2.7.7.  |
   | +1 :green_heart: |  javadoc  |   2m 34s |  the patch passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  javadoc  |   3m 38s |  the patch passed with JDK 
v1.7.0_242  |
   | +1 :green_heart: |  findbugs  |  13m 21s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 208m 46s |  root in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m  9s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 300m 47s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1016 |
   | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc 
unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 709cf1c4a6d2 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1016/out/precommit/personality/provided.sh
 |
   | git revision | branch-1.4 / 879b7ea |
   | Default Java | 1.7.0_242 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 
/usr/lib/jvm/zulu-7-amd64:1.7.0_242 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/testReport/
 |
   | Max. process+thread count | 4130 (vs. ulimit of 1) |
   | modules | C: hbase-server . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/console |
   | versions | git=1.9.1 maven=3.0.5 shellcheck=0.7.0 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Gi

[jira] [Created] (HBASE-23677) region.jsp returns 500/NPE when provided encoded region name is not online

Nick Dimiduk created HBASE-23677:


 Summary: region.jsp returns 500/NPE when provided encoded region 
name is not online
 Key: HBASE-23677
 URL: https://issues.apache.org/jira/browse/HBASE-23677
 Project: HBase
  Issue Type: Bug
  Components: regionserver, UI
Affects Versions: 2.2.3
Reporter: Nick Dimiduk


Noticed while testing for HBASE-23653. A link off to a region that is closed or 
otherwise not in the region server's online region map results in a 500.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1017: HBASE-23675 Move to Apache parent POM version 22

Apache-HBase commented on issue #1017: HBASE-23675 Move to Apache parent POM 
version 22
URL: https://github.com/apache/hbase/pull/1017#issuecomment-573145861
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 21s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -0 :warning: |  test4tests  |   0m  0s |  The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 42s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 53s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 17s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  5s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   3m  3s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 21s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 21s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 21s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  2s |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  shadedjars  |   5m  0s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 20s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   3m  4s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 179m 46s |  root in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 59s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 241m 42s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hbase.client.TestAsyncTableGetMultiThreaded |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1017/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1017 |
   | Optional Tests | dupname asflicense javac javadoc unit shadedjars 
hadoopcheck xml compile |
   | uname | Linux 7d1c60a90af4 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 
05:24:09 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1017/out/precommit/personality/provided.sh
 |
   | git revision | master / 56842d04d7 |
   | Default Java | 1.8.0_181 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1017/1/artifact/out/patch-unit-root.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1017/1/testReport/
 |
   | Max. process+thread count | 5375 (vs. ulimit of 1) |
   | modules | C: hbase-build-configuration . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1017/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…

saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on 
meta table (fix bloomfilters…
URL: https://github.com/apache/hbase/pull/1012#issuecomment-573148773
 
 
   So, the ROW_INDEX_V1 encoding seems to mess up the tests. Here is a patch 
with ROWCOL bloom to see how it does.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1018: HBASE-23652 Move the unsupported procedure type check before migratin…

Apache-HBase commented on issue #1018: HBASE-23652 Move the unsupported 
procedure type check before migratin…
URL: https://github.com/apache/hbase/pull/1018#issuecomment-573151752
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 18s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -0 :warning: |  test4tests  |   0m  0s |  The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m  9s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 33s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  4s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 53s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   4m 50s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 58s |  the patch passed  |
   | -1 :x: |  checkstyle  |   1m 27s |  hbase-server: The patch generated 5 
new + 103 unchanged - 0 fixed = 108 total (was 103)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   5m  5s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 45s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   4m 57s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 162m 52s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 30s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 227m 14s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1018/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1018 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux b1b05b15086e 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 
10:36:11 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1018/out/precommit/personality/provided.sh
 |
   | git revision | master / 56842d04d7 |
   | Default Java | 1.8.0_181 |
   | checkstyle | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1018/1/artifact/out/diff-checkstyle-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1018/1/testReport/
 |
   | Max. process+thread count | 4879 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1018/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack closed pull request #1015: HBASE-23668 Master log start filling with "Flush journal status" messages

saintstack closed pull request #1015: HBASE-23668  Master log start filling 
with "Flush journal status" messages
URL: https://github.com/apache/hbase/pull/1015
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on issue #1015: HBASE-23668 Master log start filling with "Flush journal status" messages

saintstack commented on issue #1015: HBASE-23668  Master log start filling with 
"Flush journal status" messages
URL: https://github.com/apache/hbase/pull/1015#issuecomment-573152004
 
 
   Manually merged to branch-2 and master.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (HBASE-23668) Master log start filling with "Flush journal status" messages



 [ 
https://issues.apache.org/jira/browse/HBASE-23668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-23668.
---
Hadoop Flags: Reviewed
Assignee: Michael Stack
  Resolution: Fixed

Merged to branch-2+. Thanks for review [~zhangduo] (reopen to add your UT?).

> Master log start filling with "Flush journal status" messages
> -
>
> Key: HBASE-23668
> URL: https://issues.apache.org/jira/browse/HBASE-23668
> Project: HBase
>  Issue Type: Improvement
>  Components: proc-v2, RegionProcedureStore
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Takes a while to get into this condition. Not each to tell how because all 
> logs have rolled off and I only have logs filled w/ below:
> {code}
> 2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> Flush status journal:
>   Acquiring readlock on region at 1578553261723
>   Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, 
> failureReason:Nothing to flush,flush seq id45226854 at 1578553261723
> 2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> Flush status journal:
>   Acquiring readlock on region at 1578553261723
>   Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, 
> failureReason:Nothing to flush,flush seq id45226855 at 1578553261723
> 2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> Flush status journal:
>   Acquiring readlock on region at 1578553261723
>   Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, 
> failureReason:Nothing to flush,flush seq id45226856 at 1578553261723
> 2020-01-09 07:01:01,723 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
> Flush status journal:
>   Acquiring readlock on region at 1578553261723
>   Flush successful flush result:CANNOT_FLUSH_MEMSTORE_EMPTY, 
> failureReason:Nothing to flush,flush seq id45226857 at 1578553261723
> {code}
> ... I added the printing of flushresult... i.e. cannot flush because store is 
> empty.
> Digging.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1010: HBASE-23664 Upgrade JUnit to 4.13

Apache-HBase commented on issue #1010: HBASE-23664 Upgrade JUnit to 4.13
URL: https://github.com/apache/hbase/pull/1010#issuecomment-573157065
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   7m 11s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -0 :warning: |  test4tests  |   0m  0s |  The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   8m 43s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   1m 42s |  branch-1 passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  compile  |   1m 49s |  branch-1 passed with JDK 
v1.7.0_242  |
   | +1 :green_heart: |  shadedjars  |   2m 49s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 13s |  branch-1 passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  javadoc  |   3m 46s |  branch-1 passed with JDK 
v1.7.0_242  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m  1s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 42s |  the patch passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  javac  |   1m 42s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 45s |  the patch passed with JDK 
v1.7.0_242  |
   | +1 :green_heart: |  javac  |   1m 45s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | -1 :x: |  xml  |   0m  1s |  The patch has 1 ill-formed XML file(s).  |
   | +1 :green_heart: |  shadedjars  |   2m 43s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   5m  0s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   2m  7s |  the patch passed with JDK 
v1.8.0_232  |
   | +1 :green_heart: |  javadoc  |   3m 48s |  the patch passed with JDK 
v1.7.0_242  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 147m 38s |  root in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 38s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 198m 52s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | XML | Parsing Error(s): |
   |   | pom.xml |
   | Failed junit tests | hadoop.hbase.client.TestAdmin2 |
   |   | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas |
   |   | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory |
   |   | hadoop.hbase.regionserver.TestCompactionState |
   |   | hadoop.hbase.security.token.TestGenerateDelegationToken |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1010/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1010 |
   | Optional Tests | dupname asflicense javac javadoc unit shadedjars 
hadoopcheck xml compile |
   | uname | Linux 84529f3a5ea7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1010/out/precommit/personality/provided.sh
 |
   | git revision | branch-1 / 6b940ae |
   | Default Java | 1.7.0_242 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 
/usr/lib/jvm/zulu-7-amd64:1.7.0_242 |
   | xml | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1010/2/artifact/out/xml.txt
 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1010/2/artifact/out/patch-unit-root.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1010/2/testReport/
 |
   | Max. process+thread count | 4560 (vs. ulimit of 1) |
   | modules | C: . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1010/2/console |
   | versions | git=1.9.1 maven=3.0.5 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23279) Switch default block encoding to ROW_INDEX_V1



[ 
https://issues.apache.org/jira/browse/HBASE-23279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013140#comment-17013140
 ] 

Michael Stack commented on HBASE-23279:
---

How we doing here [~vjasani]? [~anoop.hbase] reminded me this was outstanding 
in his review of HBASE-21065. In my test runs enabling ROW_INDEX_V1 on 
hbase:meta, the TestGetClosestAtOrBefore fails (commented out here) and then 
the TestProcedurePriority fails too as it does for you here. What you thinking? 
I was going to dig in but asking here first.

> Switch default block encoding to ROW_INDEX_V1
> -
>
> Key: HBASE-23279
> URL: https://issues.apache.org/jira/browse/HBASE-23279
> Project: HBase
>  Issue Type: Wish
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Lars Hofhansl
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23279.master.000.patch, 
> HBASE-23279.master.001.patch, HBASE-23279.master.002.patch, 
> HBASE-23279.master.003.patch
>
>
> Currently we set both block encoding and compression to NONE.
> ROW_INDEX_V1 has many advantages and (almost) no disadvantages (the hfiles 
> are slightly larger about 3% or so). I think that would a better default than 
> NONE.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23678) Literate builder API for version management in schema

Andrew Kyle Purtell created HBASE-23678:
---

 Summary: Literate builder API for version management in schema
 Key: HBASE-23678
 URL: https://issues.apache.org/jira/browse/HBASE-23678
 Project: HBase
  Issue Type: Improvement
Reporter: Andrew Kyle Purtell


Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions. This can be achieved 
with VERSIONS=INT_MAX, TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, 
MIN_VERSION=_num_versions_ . This is not intuitive though because VERSIONS has 
been used to specify _num_versions_ in this example since version 0.1.

A literate builder API, by way if its method names, could let a user describe 
more or less in speaking language how they want version retention to work, and 
internally the builder API could set the low level schema attributes. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23678) Literate builder API for version management in schema



 [ 
https://issues.apache.org/jira/browse/HBASE-23678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-23678:

Description: 
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions. This can be achieved 
with VERSIONS=INT_MAX, TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, 
MIN_VERSION=_num_versions_ . This is not intuitive though because VERSIONS has 
been used to specify the number of versions to retain (_num_versions_ in this 
example) since HBase version 0.1, so this is going to be a source of confusion 
- I've seen it in practice. 

A literate builder API, by way if its method names, could let a user describe 
more or less in speaking language how they want version retention to work, and 
internally the builder API could set the low level schema attributes. 

  was:
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions. This can be achieved 
with VERSIONS=INT_MAX, TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, 
MIN_VERSION=_num_versions_ . This is not intuitive though because VERSIONS has 
been used to specify _num_versions_ in this example since version 0.1.

A literate builder API, by way if its method names, could let a user describe 
more or less in speaking language how they want version retention to work, and 
internally the builder API could set the low level schema attributes. 


> Literate builder API for version management in schema
> -
>
> Key: HBASE-23678
> URL: https://issues.apache.org/jira/browse/HBASE-23678
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
> KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
> regarding their usage. Almost all combinations of these four settings make 
> sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
> KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the 
> behavior with TTL easier to conceive when creating the schema. This could 
> take the form of a literate builder API for ColumnDescriptor or an extension 
> to an existing one. 
> Let me give you a motivating example: We may want to retain all versions for 
> a given TTL, and then only a specific number of versions. This can be 
> achieved with VERSIONS=INT_MAX, TTL=_retention_interval_, 
> KEEP_DELETED_CELLS=TTL, MIN_VERSION=_num_versions_ . This is not intuitive 
> though because VERSIONS has been used to specify the number of versions to 
> retain (_num_versions_ in this example) since HBase version 0.1, so this is 
> going to be a source of confusion - I've seen it in practice. 
> A literate builder API, by way if its method names, could let a user describe 
> more or less in speaking language how they want version retention to work, 
> and internally the builder API could set the low level schema attributes. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23678) Literate builder API for version management in schema



 [ 
https://issues.apache.org/jira/browse/HBASE-23678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-23678:

Description: 
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions. This can be achieved 
with VERSIONS=INT_MAX, TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, 
MIN_VERSION=_num_versions_ . This is not intuitive though because VERSIONS has 
been used to specify the number of versions to retain (_num_versions_ in this 
example) since HBase version 0.1, so this is going to be a source of confusion 
- I've seen it in practice. 

A literate builder API, by way of the way we design its method names, could let 
a user describe more or less in speaking language how they want version 
retention to work, and internally the builder API could set the low level 
schema attributes. 

  was:
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions. This can be achieved 
with VERSIONS=INT_MAX, TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, 
MIN_VERSION=_num_versions_ . This is not intuitive though because VERSIONS has 
been used to specify the number of versions to retain (_num_versions_ in this 
example) since HBase version 0.1, so this is going to be a source of confusion 
- I've seen it in practice. 

A literate builder API, by way if its method names, could let a user describe 
more or less in speaking language how they want version retention to work, and 
internally the builder API could set the low level schema attributes. 


> Literate builder API for version management in schema
> -
>
> Key: HBASE-23678
> URL: https://issues.apache.org/jira/browse/HBASE-23678
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
> KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
> regarding their usage. Almost all combinations of these four settings make 
> sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
> KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the 
> behavior with TTL easier to conceive when creating the schema. This could 
> take the form of a literate builder API for ColumnDescriptor or an extension 
> to an existing one. 
> Let me give you a motivating example: We may want to retain all versions for 
> a given TTL, and then only a specific number of versions. This can be 
> achieved with VERSIONS=INT_MAX, TTL=_retention_interval_, 
> KEEP_DELETED_CELLS=TTL, MIN_VERSION=_num_versions_ . This is not intuitive 
> though because VERSIONS has been used to specify the number of versions to 
> retain (_num_versions_ in this example) since HBase version 0.1, so this is 
> going to be a source of confusion - I've seen it in practice. 
> A literate builder API, by way of the way we design its method names, could 
> let a user describe more or less in speaking language how they want version 
> retention to work, and internally the builder API could set the low level 
> schema attributes. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23678) Literate builder API for version management in schema



 [ 
https://issues.apache.org/jira/browse/HBASE-23678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-23678:

Description: 
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions after that interval 
elapses. This can be achieved with VERSIONS=INT_MAX, TTL=_retention_interval_, 
KEEP_DELETED_CELLS=TTL, MIN_VERSION=_num_versions_ . This is not intuitive 
though because VERSIONS has been used to specify the number of versions to 
retain (_num_versions_ in this example) since HBase version 0.1, so this is 
going to be a source of confusion - I've seen it in practice. 

A literate builder API, by way of the way we design its method names, could let 
a user describe more or less in speaking language how they want version 
retention to work, and internally the builder API could set the low level 
schema attributes. 

  was:
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions. This can be achieved 
with VERSIONS=INT_MAX, TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, 
MIN_VERSION=_num_versions_ . This is not intuitive though because VERSIONS has 
been used to specify the number of versions to retain (_num_versions_ in this 
example) since HBase version 0.1, so this is going to be a source of confusion 
- I've seen it in practice. 

A literate builder API, by way of the way we design its method names, could let 
a user describe more or less in speaking language how they want version 
retention to work, and internally the builder API could set the low level 
schema attributes. 


> Literate builder API for version management in schema
> -
>
> Key: HBASE-23678
> URL: https://issues.apache.org/jira/browse/HBASE-23678
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
> KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
> regarding their usage. Almost all combinations of these four settings make 
> sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
> KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the 
> behavior with TTL easier to conceive when creating the schema. This could 
> take the form of a literate builder API for ColumnDescriptor or an extension 
> to an existing one. 
> Let me give you a motivating example: We may want to retain all versions for 
> a given TTL, and then only a specific number of versions after that interval 
> elapses. This can be achieved with VERSIONS=INT_MAX, 
> TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, MIN_VERSION=_num_versions_ 
> . This is not intuitive though because VERSIONS has been used to specify the 
> number of versions to retain (_num_versions_ in this example) since HBase 
> version 0.1, so this is going to be a source of confusion - I've seen it in 
> practice. 
> A literate builder API, by way of the way we design its method names, could 
> let a user describe more or less in speaking language how they want version 
> retention to work, and internally the builder API could set the low level 
> schema attributes. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23678) Literate builder API for version management in schema



 [ 
https://issues.apache.org/jira/browse/HBASE-23678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Kyle Purtell updated HBASE-23678:

Description: 
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions after that interval 
elapses. This can be achieved with VERSIONS=INT_MAX, TTL=_retention_interval_, 
KEEP_DELETED_CELLS=TTL, MIN_VERSIONS=_num_versions_ . This is not intuitive 
though because VERSIONS has been used to specify the number of versions to 
retain (_num_versions_ in this example) since HBase version 0.1, so this is 
going to be a source of confusion - I've seen it in practice. 

A literate builder API, by way of the way we design its method names, could let 
a user describe more or less in speaking language how they want version 
retention to work, and internally the builder API could set the low level 
schema attributes. 

  was:
Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
regarding their usage. Almost all combinations of these four settings make 
sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the behavior 
with TTL easier to conceive when creating the schema. This could take the form 
of a literate builder API for ColumnDescriptor or an extension to an existing 
one. 

Let me give you a motivating example: We may want to retain all versions for a 
given TTL, and then only a specific number of versions after that interval 
elapses. This can be achieved with VERSIONS=INT_MAX, TTL=_retention_interval_, 
KEEP_DELETED_CELLS=TTL, MIN_VERSION=_num_versions_ . This is not intuitive 
though because VERSIONS has been used to specify the number of versions to 
retain (_num_versions_ in this example) since HBase version 0.1, so this is 
going to be a source of confusion - I've seen it in practice. 

A literate builder API, by way of the way we design its method names, could let 
a user describe more or less in speaking language how they want version 
retention to work, and internally the builder API could set the low level 
schema attributes. 


> Literate builder API for version management in schema
> -
>
> Key: HBASE-23678
> URL: https://issues.apache.org/jira/browse/HBASE-23678
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> Lars designed the combination of VERSIONS, TTL, MIN_VERSIONS, and 
> KEEP_DELETED_CELLS with a maximum of flexibility. There is a lot of nuance 
> regarding their usage. Almost all combinations of these four settings make 
> sense for some use cases (exceptions are MIN_VERSIONS > 0 without TTL, and 
> KEEP_DELETED_CELLS=TTL without TTL). There should be a way to make the 
> behavior with TTL easier to conceive when creating the schema. This could 
> take the form of a literate builder API for ColumnDescriptor or an extension 
> to an existing one. 
> Let me give you a motivating example: We may want to retain all versions for 
> a given TTL, and then only a specific number of versions after that interval 
> elapses. This can be achieved with VERSIONS=INT_MAX, 
> TTL=_retention_interval_, KEEP_DELETED_CELLS=TTL, MIN_VERSIONS=_num_versions_ 
> . This is not intuitive though because VERSIONS has been used to specify the 
> number of versions to retain (_num_versions_ in this example) since HBase 
> version 0.1, so this is going to be a source of confusion - I've seen it in 
> practice. 
> A literate builder API, by way of the way we design its method names, could 
> let a user describe more or less in speaking language how they want version 
> retention to work, and internally the builder API could set the low level 
> schema attributes. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23679) FileSystem instance leaks due to bulk loads with Kerberos enabled

2020-01-10 Thread Josh Elser (Jira)

Josh Elser created HBASE-23679:
--

 Summary: FileSystem instance leaks due to bulk loads with Kerberos 
enabled
 Key: HBASE-23679
 URL: https://issues.apache.org/jira/browse/HBASE-23679
 Project: HBase
  Issue Type: Bug
Reporter: Josh Elser
Assignee: Josh Elser


Spent the better part of a week chasing an issue on HBase 2.x where the number 
of DistributedFileSystem instances on the heap of a RegionServer would grow 
unbounded. Looking at multiple heap-dumps, it was obvious to see that we had an 
immense number of DFS instances cached (in FileSystem$Cache) for the same user, 
with the unique number of Tokens contained in that DFS's UGI member (one hbase 
delegation token, and two HDFS delegation tokens – we only do this for bulk 
loads). For the user's clusters, they eventually experienced 10x perf 
degradation as RegionServers spent all of their time in JVM GC (they were 
unlucky to not have RegionServers crash outright, as this would've, albeit 
temporarily, fixed the issue).

The problem seems to be two-fold with changes by HBASE-15291 being largely the 
cause. This issue tried to close
 FileSystem instances which were being leaked – however, it did this by 
instrumenting the method
 {{SecureBulkLoadManager.cleanupBulkLoad(..)}}. Two big issues with this 
approach:
 1. It relies on clients to call this method (client's hanging up will leak 
resources in RegionServers)
 2. This method is only called on the RegionServer hosting the first Region of 
the table which was bulk-loaded into. For
 multiple RegionServers, they are left to leak resources.

HBASE-21342 later tried to fix an issue where FS objects were now being closed 
prematurely via reference-counting (which appears to work fine), but does not 
address the other two issues above.

Through all of this, I (re)learned the dirty history of UGI and how its caching 
doesn't work so great HADOOP-6670. I see trying to continue to leverage the 
FileSystem$CACHE as a potentially dangerous thing (we've been back here 
multiple times already). My opinion at this point is that we should cleanly 
create a new FileSystem instance during the call to 
{{SecureBulkLoadManager#secureBulkLoadHFiles(..)}} and close it in a finally 
block in that same method. This both simplifies the lifecycle of a FileSystem 
instance in the bulk-load codepath but also helps us avoid future problems with 
UGI and FS caching. The one downside is that we pay the penalty to create a new 
FileSystem instance, but I'm of the opinion that we cross that bridge when we 
get there.

Thanks for [~jdcryans] and [~busbey] for their help along the way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23679) FileSystem instance leaks due to bulk loads with Kerberos enabled

2020-01-10 Thread Josh Elser (Jira)

[
https://issues.apache.org/jira/browse/HBASE-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Elser updated HBASE-23679:
---
Description:
Spent the better part of a week chasing an issue on HBase 2.x where the number
of DistributedFileSystem instances on the heap of a RegionServer would grow
unbounded. Looking at multiple heap-dumps, it was obvious to see that we had an
immense number of DFS instances cached (in FileSystem$Cache) for the same user,
with the unique number of Tokens contained in that DFS's UGI member (one hbase
delegation token, and two HDFS delegation tokens – we only do this for bulk
loads). For the user's clusters, they eventually experienced 10x perf
degradation as RegionServers spent all of their time in JVM GC (they were
unlucky to not have RegionServers crash outright, as this would've, albeit
temporarily, fixed the issue).

The problem seems to be two-fold with changes by HBASE-15291 being largely the
cause. This issue tried to close FileSystem instances which were being leaked –
however, it did this by instrumenting the method
{{SecureBulkLoadManager.cleanupBulkLoad(..)}}. Two big issues with this
approach:
# It relies on clients to call this method (client's hanging up will leak
resources in RegionServers)
# This method is only called on the RegionServer hosting the first Region of
the table which was bulk-loaded into. For multiple RegionServers, they are left
to leak resources.

HBASE-21342 later tried to fix an issue where FS objects were now being closed
prematurely via reference-counting (which appears to work fine), but does not
address the other two issues above. Point #2 makes debugging this issue harder
than normal because it doesn't manifest on a single node instance :)

Through all of this, I (re)learned the dirty history of UGI and how its caching
doesn't work so great HADOOP-6670. I see trying to continue to leverage the
FileSystem$CACHE as a potentially dangerous thing (we've been back here
multiple times already). My opinion at this point is that we should cleanly
create a new FileSystem instance during the call to
{{SecureBulkLoadManager#secureBulkLoadHFiles(..)}} and close it in a finally
block in that same method. This both simplifies the lifecycle of a FileSystem
instance in the bulk-load codepath but also helps us avoid future problems with
UGI and FS caching. The one downside is that we pay the penalty to create a new
FileSystem instance, but I'm of the opinion that we cross that bridge when we
get there.

Thanks for [~jdcryans] and [~busbey] for their help along the way.

was:
Spent the better part of a week chasing an issue on HBase 2.x where the number
of DistributedFileSystem instances on the heap of a RegionServer would grow
unbounded. Looking at multiple heap-dumps, it was obvious to see that we had an
immense number of DFS instances cached (in FileSystem$Cache) for the same user,
with the unique number of Tokens contained in that DFS's UGI member (one hbase
delegation token, and two HDFS delegation tokens – we only do this for bulk
loads). For the user's clusters, they eventually experienced 10x perf
degradation as RegionServers spent all of their time in JVM GC (they were
unlucky to not have RegionServers crash outright, as this would've, albeit
temporarily, fixed the issue).

The problem seems to be two-fold with changes by HBASE-15291 being largely the
cause. This issue tried to close
FileSystem instances which were being leaked – however, it did this by
instrumenting the method
{{SecureBulkLoadManager.cleanupBulkLoad(..)}}. Two big issues with this
approach:
1. It relies on clients to call this method (client's hanging up will leak
resources in RegionServers)
2. This method is only called on the RegionServer hosting the first Region of
the table which was bulk-loaded into. For
multiple RegionServers, they are left to leak resources.

HBASE-21342 later tried to fix an issue where FS objects were now being closed
prematurely via reference-counting (which appears to work fine), but does not
address the other two issues above.

Thanks for [~jdcryans] and [~busbey] for their

[GitHub] [hbase] ndimiduk merged pull request #1010: HBASE-23664 Upgrade JUnit to 4.13

ndimiduk merged pull request #1010: HBASE-23664 Upgrade JUnit to 4.13
URL: https://github.com/apache/hbase/pull/1010
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23628) Remove Apache Commons Digest Base64



 [ 
https://issues.apache.org/jira/browse/HBASE-23628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-23628:

Component/s: dependencies

> Remove Apache Commons Digest Base64
> ---
>
> Key: HBASE-23628
> URL: https://issues.apache.org/jira/browse/HBASE-23628
> Project: HBase
>  Issue Type: Improvement
>  Components: dependencies
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.0.0
>
>
> Use the native JDK Base64 implementation instead.  Most places are using the 
> JDK version, but a couple of spots were missed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] joshelser opened a new pull request #1019: HBASE-23679 Use new FileSystem objects during bulk loads

joshelser opened a new pull request #1019: HBASE-23679 Use new FileSystem 
objects during bulk loads
URL: https://github.com/apache/hbase/pull/1019
 
 
   Undoes lots of fanciness about FileSystem caching because of
   an explicit bug that was present (FileSystem only closed on one
   out of N regionservers, and not every time), but also because
   the FS caching itself is dodgy and prone to error.
   
   Each BulkLoad request will now get its own FileSystem instance
   that it is responsible for closing.
   
   The change to HRegion is because ${hbase.rootdir}/data is `chmod 700` which 
means that a normal user cannot get the size of those files (you'll see lots of 
AccessDenied exceptions in the RS log). Using HRegionFilesystem instead keeps 
this from being an issue (reading the filesize as HBase instead of the user).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (HBASE-23664) Upgrade JUnit to 4.13



 [ 
https://issues.apache.org/jira/browse/HBASE-23664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-23664.
--
Resolution: Fixed

> Upgrade JUnit to 4.13
> -
>
> Key: HBASE-23664
> URL: https://issues.apache.org/jira/browse/HBASE-23664
> Project: HBase
>  Issue Type: Task
>  Components: integration tests, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> New JUnit released a week ago. Let's give it a spin.
> https://github.com/junit-team/junit4/blob/master/doc/ReleaseNotes4.13.md



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23676) Address feedback on HBASE-23055 Alter hbase:meta.



[ 
https://issues.apache.org/jira/browse/HBASE-23676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013220#comment-17013220
 ] 

Michael Stack commented on HBASE-23676:
---

{code}
Duo Zhang 

5:33 AM (8 hours ago)

to State, apache/hbase, Michael

@Apache9 commented on this pull request.

In 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java:

> @@ -666,42 +667,55 @@ public void run(PRESP resp) {
   new DisableTableProcedureBiConsumer(tableName));
   }
 
+  /**
+   * Utility for completing passed TableState {@link CompletableFuture} 
future
+   * using passed parameters.
+   */
+  private static CompletableFuture completeCheckTableState(

What is the return value used for?

In 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java:

> @@ -58,8 +62,19 @@
 
   private final ZNodePaths znodePaths;
 
+  /**
+   * A znode maintained by MirroringTableStateManager.
+   * MirroringTableStateManager is deprecated to be removed in hbase3. It can 
also be disabled.
+   * Make sure it is enabled if you want to alter hbase:meta table in hbase2. 
In hbase3,
+   * TBD how metatable state will be hosted; likely on active hbase master.

I think the state will either be on zk or on hdfs? HMaster itself is state 
less...

In general, in the current architecture, I think it the state should be placed 
on zk, of course you could cache it in master and let client go to master to 
ask for the state.

In 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java:

> @@ -58,8 +62,19 @@
 
   private final ZNodePaths znodePaths;
 
+  /**
+   * A znode maintained by MirroringTableStateManager.
+   * MirroringTableStateManager is deprecated to be removed in hbase3. It can 
also be disabled.
+   * Make sure it is enabled if you want to alter hbase:meta table in hbase2. 
In hbase3,
+   * TBD how metatable state will be hosted; likely on active hbase master.

So I do not think we should name this as mirrored, it is the original data. The 
state in master's memory, is a cache, actually.

In 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java:

> @@ -229,6 +244,43 @@ private void 
> getMetaRegionLocation(CompletableFuture future,
 });
   }
 
+  @Override
+  public CompletableFuture getMetaTableState() {
+return getAndConvert(this.znodeMirroredMetaTableState, 
ZKAsyncRegistry::getTableState).
+  thenApply(state -> {
+return state == null || 
state.equals(ENABLED_META_TABLE_STATE.getState())?
+  ENABLED_META_TABLE_STATE: new TableState(TableName.META_TABLE_NAME, 
state);
+  }).exceptionally(e -> {

Currently in HBase, usually we will create a new CompletableFuture and use 
FutureUtils.addListener to complete it. The code in exceptionally are a bit 
tricky, where we throw a CompletionException...

In 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java:

>private static final Logger LOG = 
> LoggerFactory.getLogger(TableStateManager.class);
+
+  /**
+   * All table state is kept in hbase:meta except that of hbase:meta itself.

In general I do not think this is a good design. We should persist it 
somewhere, and just cache it in master's memory. And we do not need to disable 
a table when altering any more? And the solution here just assume that we only 
have one meta region? So the altering operation can be atomic? What if we have 
multiple meta regions and we crash in the middle? I think this patch also aims 
to implement splittable meta in the future? No?

In 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RejectReplicationRequestStateChecker.java:

> @@ -1,4 +1,6 @@
-/**
+/*
+ * Copyright The Apache Software Foundation

What's this?

In hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseMetaEdit.java:

> +Admin admin = UTIL.getAdmin();
+admin.tableExists(TableName.META_TABLE_NAME);
+admin.disableTable(TableName.META_TABLE_NAME);
+assertTrue(admin.isTableDisabled(TableName.META_TABLE_NAME));
+TableDescriptor descriptor = 
admin.getDescriptor(TableName.META_TABLE_NAME);
+ColumnFamilyDescriptor cfd = 
descriptor.getColumnFamily(HConstants.CATALOG_FAMILY);
+byte [] extraColumnFamilyName = Bytes.toBytes("xtra");
+ColumnFamilyDescriptor newCfd =
+
ColumnFamilyDescriptorBuilder.newBuilder(extraColumnFamilyName).build();
+int oldVersions = cfd.getMaxVersions();
+// Add '1' to current versions count.
+cfd = 
ColumnFamilyDescriptorBuilder.newBuilder(cfd).setMaxVersions(oldVersions + 1).
+setConfiguration(ColumnFamilyDescriptorBuilder.DATA_BLOCK_ENCODING,
+DataBlockEncoding.ROW_INDEX_V1.toString()).build();
+admin.modifyColumnFamily(TableName.META_TABLE_NAME, cfd);
+admin.addColumnFamily(TableName.META_TABLE_NAME, newCfd);

Now we allow users to add/remove fam

[jira] [Reopened] (HBASE-23628) Remove Apache Commons Digest Base64



 [ 
https://issues.apache.org/jira/browse/HBASE-23628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened HBASE-23628:
-

reopening for backport(s)

> Remove Apache Commons Digest Base64
> ---
>
> Key: HBASE-23628
> URL: https://issues.apache.org/jira/browse/HBASE-23628
> Project: HBase
>  Issue Type: Improvement
>  Components: dependencies
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.0.0
>
>
> Use the native JDK Base64 implementation instead.  Most places are using the 
> JDK version, but a couple of spots were missed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23628) Remove Apache Commons Digest Base64



 [ 
https://issues.apache.org/jira/browse/HBASE-23628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-23628.
-
Fix Version/s: 2.3.0
   Resolution: Fixed

> Remove Apache Commons Digest Base64
> ---
>
> Key: HBASE-23628
> URL: https://issues.apache.org/jira/browse/HBASE-23628
> Project: HBase
>  Issue Type: Improvement
>  Components: dependencies
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Use the native JDK Base64 implementation instead.  Most places are using the 
> JDK version, but a couple of spots were missed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23628) Replace Apache Commons Digest Base64 with JDK8 Base64



 [ 
https://issues.apache.org/jira/browse/HBASE-23628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-23628:

Summary: Replace Apache Commons Digest Base64 with JDK8 Base64  (was: 
Remove Apache Commons Digest Base64)

> Replace Apache Commons Digest Base64 with JDK8 Base64
> -
>
> Key: HBASE-23628
> URL: https://issues.apache.org/jira/browse/HBASE-23628
> Project: HBase
>  Issue Type: Improvement
>  Components: dependencies
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Use the native JDK Base64 implementation instead.  Most places are using the 
> JDK version, but a couple of spots were missed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…

Apache-HBase commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on 
meta table (fix bloomfilters…
URL: https://github.com/apache/hbase/pull/1012#issuecomment-573226238
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 37s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -0 :warning: |  test4tests  |   0m  0s |  The patch doesn't appear to 
include any new or modified tests. Please justify why no new tests are needed 
for this patch. Also please list what manual steps were performed to verify 
this patch.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 31s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 44s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 25s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 40s |  master passed  |
   | +0 :ok: |  spotbugs  |   5m  4s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   5m  0s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 52s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  5s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  5s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 39s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   5m 38s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  18m 30s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   4m 52s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 156m  1s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 38s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 223m 37s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1012 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 888eea02fdcf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1012/out/precommit/personality/provided.sh
 |
   | git revision | master / 8ca614857d |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/2/testReport/
 |
   | Max. process+thread count | 5889 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/2/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23326) Implement a ProcedureStore which stores procedures in a HRegion



[ 
https://issues.apache.org/jira/browse/HBASE-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013235#comment-17013235
 ] 

Michael Stack commented on HBASE-23326:
---

[~zhangduo] I see that the regionprocedurestore WAL files are being moved to 
the /hbase/archive but I also see build up at /hbase/MasterProcs/archive of old 
procedure hfiles. I have 1500 files in it currently (running about 24 hours 
under load). Poking in PR I don't see obvious handling of this Region archive 
dir. What am I missing? Thanks.

> Implement a ProcedureStore which stores procedures in a HRegion
> ---
>
> Key: HBASE-23326
> URL: https://issues.apache.org/jira/browse/HBASE-23326
> Project: HBase
>  Issue Type: Improvement
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> So we can resue the code in HRegion for persisting the procedures, and also 
> the optimized WAL implementation for better performance.
> This requires we merge the hbase-procedure module to hbase-server, which is 
> an anti-pattern as we make the hbase-server module more overloaded. But I 
> think later we can first try to move the WAL stuff out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…

saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on 
meta table (fix bloomfilters…
URL: https://github.com/apache/hbase/pull/1012#issuecomment-573230752
 
 
   Ok. So changing blooms seems fine. Let me retry with the encoder.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client



[ 
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013250#comment-17013250
 ] 

Hudson commented on HBASE-18095:


Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #35 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/35/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/35//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/35//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/35//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Provide an option for clients to find the server hosting META that does not 
> involve the ZooKeeper client
> 
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the 
> regionserver hosting the meta table region. Site configuration provides the 
> client a list of ZK quorum peers and the client uses an embedded ZK client to 
> query meta location. Timeouts and retry behavior of this embedded ZK client 
> are managed orthogonally to HBase layer settings and in some cases the ZK 
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage 
> or network partition.
> We should consider new configuration settings that provide a list of 
> well-known master and backup master locations, and with this information the 
> client can contact any of the master processes directly. Any master in either 
> active or passive state will track meta location and respond to requests for 
> it with its cached last known location. If this location is stale, the client 
> can ask again with a flag set that requests the master refresh its location 
> cache and return the up-to-date location. Every client interaction with the 
> cluster thus uses only HBase RPC as transport, with appropriate settings 
> applied to the connection. The configuration toggle that enables this 
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and 
> contact the ZK service directly at the beginning of the connection lifecycle. 
> This has several benefits. ZK service need not be exposed to clients, and 
> their potential abuse, yet no benefit ZK provides the HBase server cluster is 
> compromised. Normalizing HBase client and ZK client timeout settings and 
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer 
> necessary. 
> And, from [~ghelmling]: There is an additional complication here for 
> token-based authentication. When a delegation token is used for SASL 
> authentication, the client uses the cluster ID obtained from Zookeeper to 
> select the token identifier to use. So there would also need to be some 
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23349) Low refCount preventing archival of compacted away files



[ 
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013255#comment-17013255
 ] 

Andrew Kyle Purtell commented on HBASE-23349:
-

{quote}
I think we should step back and remember why we have the ref counting in the 
first place. This came from a discussion started in HBASE-13082 and 
HBASE-10060, namely too much synchronization.

If any changes we make now needs new synchronization in the scanner.next(...) 
path we're back to where we started and in that case we should remove the ref 
counting and bring back the old notification and scanner switching we had 
before.
{quote}

I made a similar comment on an internal discussion yesterday. If we have to 
walk back the StoreScanner "lock coarsening" work, then let's not be afraid to 
do it. There is a nuanced decision we would have to make, but let's not be 
concerned about sunk costs. 

> Low refCount preventing archival of compacted away files
> 
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0, 2.3.0, 1.6.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> We have observed that refCount on compacted away store files as low as 1 is 
> prevent archival.
> {code:java}
> regionserver.HStore - Can't archive compacted file 
> hdfs://{{root-dir}}/hbase/data/default/t1/12a9e1112e0371955b3db8d3ebb2d298/cf1/73b72f5ddfce4a34a9e01afe7b83c1f9
>  because of either isCompactedAway=true or file has reference, 
> isReferencedInReads=true, refCount=1, skipping for now.
> {code}
> We should come up with core code (run as part of discharger thread) 
> gracefully resolve reader lock issue by resetting ongoing scanners to start 
> pointing to new store files instead of compacted away store files.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk opened a new pull request #1020: HBASE-23653 Expose content of meta table in web ui

ndimiduk opened a new pull request #1020: HBASE-23653 Expose content of meta 
table in web ui
URL: https://github.com/apache/hbase/pull/1020
 
 
   Adds a display of the content of 'hbase:meta' to the Master's
   table.jsp, when that table is selected. Supports basic pagination,
   filtering, &c.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23653) Expose content of meta table in web ui



 [ 
https://issues.apache.org/jira/browse/HBASE-23653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-23653:
-
Attachment: 23653.1.png
23653.0.png

> Expose content of meta table in web ui
> --
>
> Key: HBASE-23653
> URL: https://issues.apache.org/jira/browse/HBASE-23653
> Project: HBase
>  Issue Type: New Feature
>  Components: master, Operability, UI
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Attachments: 23653.0.png, 23653.1.png
>
>
> While testing improvements to hbck2, breaking and fixing a cluster's region 
> assignments, we've found that quite a few questions can only be answered by 
> dropping into a shell and scanning meta. This is in contrast to the nice web 
> view we have on procedures and their locks. It would be nice to have a 
> similar view into the content of meta. I've put together and extension to the 
> existing {{table.jsp}} that does this. It's had a couple rounds of operator 
> feedback. Will post up some screenshots along with the PR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23679) FileSystem instance leaks due to bulk loads with Kerberos enabled

2020-01-10 Thread Josh Elser (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013260#comment-17013260
 ] 

Josh Elser commented on HBASE-23679:


Dropping some more details in here, should anyone else ever find themselves 
back here.
 * Funny that this user's RegionServers never (rarely?) actually OOME'ed. They 
were left in this crap state because the JVM just kept hanging on. GC verbose 
logging showed us that heap was just sitting completely full. We just sat with 
30-60s GC pauses, getting next to nothing done.
 * A heapdump showed (at worst) 60k+ instances of DistributedFileSystem sitting 
on the heap, nearly all of them referenced by {{FileSystem$CACHE}}. It takes 
some digging to realize this.
 * Digging into each DFS -> DFSClient -> UGI -> Tokens, you can see the 
delegation tokens which let us confirm that Bulk Load was the culprit 
creating/leaking these.
 * For the user, it was an extremely slow growth. One RegionServer would get 
into this state every ~20 days across about that many RegionServers. So, in 
reality, we were probably looking at even longer than 20days for a single 
RegionServer to leak enough of these to cause issues.
 * UGI being a part of FileSystem caching is crap, and we should avoid it for 
anything other than "just the HBase user". It's extremely obtuse and error 
prone, likely to just give us more heartache (lots of issues around this in the 
past).

> FileSystem instance leaks due to bulk loads with Kerberos enabled
> -
>
> Key: HBASE-23679
> URL: https://issues.apache.org/jira/browse/HBASE-23679
> Project: HBase
>  Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
>
> Spent the better part of a week chasing an issue on HBase 2.x where the 
> number of DistributedFileSystem instances on the heap of a RegionServer would 
> grow unbounded. Looking at multiple heap-dumps, it was obvious to see that we 
> had an immense number of DFS instances cached (in FileSystem$Cache) for the 
> same user, with the unique number of Tokens contained in that DFS's UGI 
> member (one hbase delegation token, and two HDFS delegation tokens – we only 
> do this for bulk loads). For the user's clusters, they eventually experienced 
> 10x perf degradation as RegionServers spent all of their time in JVM GC (they 
> were unlucky to not have RegionServers crash outright, as this would've, 
> albeit temporarily, fixed the issue).
> The problem seems to be two-fold with changes by HBASE-15291 being largely 
> the cause. This issue tried to close FileSystem instances which were being 
> leaked – however, it did this by instrumenting the method 
> {{SecureBulkLoadManager.cleanupBulkLoad(..)}}. Two big issues with this 
> approach:
>  # It relies on clients to call this method (client's hanging up will leak 
> resources in RegionServers)
>  # This method is only called on the RegionServer hosting the first Region of 
> the table which was bulk-loaded into. For multiple RegionServers, they are 
> left to leak resources.
> HBASE-21342 later tried to fix an issue where FS objects were now being 
> closed prematurely via reference-counting (which appears to work fine), but 
> does not address the other two issues above. Point #2 makes debugging this 
> issue harder than normal because it doesn't manifest on a single node 
> instance :)
> Through all of this, I (re)learned the dirty history of UGI and how its 
> caching doesn't work so great HADOOP-6670. I see trying to continue to 
> leverage the FileSystem$CACHE as a potentially dangerous thing (we've been 
> back here multiple times already). My opinion at this point is that we should 
> cleanly create a new FileSystem instance during the call to 
> {{SecureBulkLoadManager#secureBulkLoadHFiles(..)}} and close it in a finally 
> block in that same method. This both simplifies the lifecycle of a FileSystem 
> instance in the bulk-load codepath but also helps us avoid future problems 
> with UGI and FS caching. The one downside is that we pay the penalty to 
> create a new FileSystem instance, but I'm of the opinion that we cross that 
> bridge when we get there.
> Thanks for [~jdcryans] and [~busbey] for their help along the way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23653) Expose content of meta table in web ui



[ 
https://issues.apache.org/jira/browse/HBASE-23653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013262#comment-17013262
 ] 

Nick Dimiduk commented on HBASE-23653:
--

Here's a couple screen shots of the new meta browser.

> Expose content of meta table in web ui
> --
>
> Key: HBASE-23653
> URL: https://issues.apache.org/jira/browse/HBASE-23653
> Project: HBase
>  Issue Type: New Feature
>  Components: master, Operability, UI
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Attachments: 23653.0.png, 23653.1.png
>
>
> While testing improvements to hbck2, breaking and fixing a cluster's region 
> assignments, we've found that quite a few questions can only be answered by 
> dropping into a shell and scanning meta. This is in contrast to the nice web 
> view we have on procedures and their locks. It would be nice to have a 
> similar view into the content of meta. I've put together and extension to the 
> existing {{table.jsp}} that does this. It's had a couple rounds of operator 
> feedback. Will post up some screenshots along with the PR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing

ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] 
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r365466674
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 ##
 @@ -86,74 +84,133 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to 
hbase:meta.
* Does not assign.
*/
-  void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+  void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
 if (holes.isEmpty()) {
-  LOG.debug("No holes.");
+  LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
 
 Review comment:
   Good sit, this log line is emitted by `MetaFixer`, speaking of its input. 
It'll be run just when an operator has invoked hbck2.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing

ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] 
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r365466933
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 ##
 @@ -86,74 +84,133 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to 
hbase:meta.
* Does not assign.
*/
-  void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+  void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
 if (holes.isEmpty()) {
-  LOG.debug("No holes.");
+  LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
   return;
 }
-for (Pair p: holes) {
-  RegionInfo ri = getHoleCover(p);
-  if (ri == null) {
-continue;
-  }
-  Configuration configuration = this.masterServices.getConfiguration();
-  HRegion.createRegionDir(configuration, ri, 
FSUtils.getRootDir(configuration));
-  // If an error here, then we'll have a region in the filesystem but not
-  // in hbase:meta (if the below fails). Should be able to rerun the fix.
-  // Add to hbase:meta and then update in-memory state so it knows of new
-  // Region; addRegionToMeta adds region and adds a state column set to 
CLOSED.
-  MetaTableAccessor.addRegionToMeta(this.masterServices.getConnection(), 
ri);
-  this.masterServices.getAssignmentManager().getRegionStates().
-  updateRegionState(ri, RegionState.State.CLOSED);
-  LOG.info("Fixed hole by adding {} in CLOSED state; region NOT assigned 
(assign to ONLINE).",
-  ri);
-}
+
+LOG.info("Identified {} region holes to fix. Detailed fixup progress 
logged at DEBUG.",
 
 Review comment:
   I want the system to log at least acknowledgement that the operator 
requested a run. That's the above log line -- an empty CatalogJanitor Report 
means no work to do. This line is setting the stage for how much work it thinks 
needs done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing

ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] 
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r365467011
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 ##
 @@ -86,74 +84,133 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to 
hbase:meta.
* Does not assign.
*/
-  void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+  void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
 if (holes.isEmpty()) {
-  LOG.debug("No holes.");
+  LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
   return;
 }
-for (Pair p: holes) {
-  RegionInfo ri = getHoleCover(p);
-  if (ri == null) {
-continue;
-  }
-  Configuration configuration = this.masterServices.getConfiguration();
-  HRegion.createRegionDir(configuration, ri, 
FSUtils.getRootDir(configuration));
-  // If an error here, then we'll have a region in the filesystem but not
-  // in hbase:meta (if the below fails). Should be able to rerun the fix.
-  // Add to hbase:meta and then update in-memory state so it knows of new
-  // Region; addRegionToMeta adds region and adds a state column set to 
CLOSED.
-  MetaTableAccessor.addRegionToMeta(this.masterServices.getConnection(), 
ri);
-  this.masterServices.getAssignmentManager().getRegionStates().
-  updateRegionState(ri, RegionState.State.CLOSED);
-  LOG.info("Fixed hole by adding {} in CLOSED state; region NOT assigned 
(assign to ONLINE).",
-  ri);
-}
+
+LOG.info("Identified {} region holes to fix. Detailed fixup progress 
logged at DEBUG.",
+  holes.size());
+
+final List newRegionInfos = createRegionInfosForHoles(holes);
+final List newMetaEntries = createMetaEntries(masterServices, 
newRegionInfos);
+final TransitRegionStateProcedure[] assignProcedures = masterServices
+  .getAssignmentManager()
+  .createRoundRobinAssignProcedures(newMetaEntries);
+
+
masterServices.getMasterProcedureExecutor().submitProcedures(assignProcedures);
+LOG.info(
 
 Review comment:
   The procedure engine logs a ton, but how will the operator know which 
procIds are the the ones that've been requested by this fixup? I'd think it's 
good to keep this message, it's an effective summary of the work done.
   
   Make it DEBUG though?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23326) Implement a ProcedureStore which stores procedures in a HRegion



[ 
https://issues.apache.org/jira/browse/HBASE-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013270#comment-17013270
 ] 

Michael Stack commented on HBASE-23326:
---

And this seem to be hanging around... 
%2C16000%2C1578524235076.1578524242389$masterproc$ though straight WALs are 
cleared.

> Implement a ProcedureStore which stores procedures in a HRegion
> ---
>
> Key: HBASE-23326
> URL: https://issues.apache.org/jira/browse/HBASE-23326
> Project: HBase
>  Issue Type: Improvement
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> So we can resue the code in HRegion for persisting the procedures, and also 
> the optimized WAL implementation for better performance.
> This requires we merge the hbase-procedure module to hbase-server, which is 
> an anti-pattern as we make the hbase-server module more overloaded. But I 
> think later we can first try to move the WAL stuff out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk opened a new pull request #1021: Backport HBASE-23653 Expose content of meta table in web ui