date:20200309

[GitHub] [hbase] infraio commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile enabled

2020-03-09 Thread GitBox

infraio commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile 
enabled
URL: https://github.com/apache/hbase/pull/1254#issuecomment-596930879
 
 
   > Mind explaining what is the problem here?
   > 
   > Thanks.
   
   The problem is that the new OutputSink didn't write seqid to cells of the 
recoved hfiles. So when read cell with same timestamp, the recovered hfile's 
cell's seqid is 0. Then the ut failed when write with same timestamp data.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257#issuecomment-596925870
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  7s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 18s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 11s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 39s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 41s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m  4s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 14s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 30s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 40s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | -0 :warning: |  unit  |  77m 55s |  hbase-server in the patch failed.  |
   |  |   | 110m  7s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1257 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 0673ad7d0725 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/testReport/
 |
   | Max. process+thread count | 6050 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257#issuecomment-596925099
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   6m 56s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m  6s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  1s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 13s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 40s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m  9s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  72m 50s |  hbase-server in the patch passed.  
|
   |  |   | 106m 52s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1257 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux aeffdf28a84b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/testReport/
 |
   | Max. process+thread count | 5279 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Assigned] (HBASE-23684) NPE HFilesOutputSink

2020-03-09 Thread Duo Zhang (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reassigned HBASE-23684:
-

Assignee: (was: Guanghao Zhang)

> NPE HFilesOutputSink
> 
>
> Key: HBASE-23684
> URL: https://issues.apache.org/jira/browse/HBASE-23684
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR, wal
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Priority: Critical
>
> Enabling the new split to hfiles feature, HBASE-23286, running branch-2 tip, 
> I see this out on RegionServers:
> {code}
>  2020-01-13 17:37:08,204 INFO org.apache.hadoop.hbase.wal.OutputSink: 3 split 
> writer threads finished
>  2020-01-13 17:37:08,233 INFO org.apache.hadoop.hbase.wal.WALSplitter: 
> Processed 1007 edits across 0 regions cost 284 ms; edits skipped=76; 
> WAL=hdfs://nameservice1/hbase/genie/WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832,
>  size=128.5 M, length=134708720, corrupted=false, progress failed=true
>  2020-01-13 17:37:08,234 WARN 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: log splitting of 
> WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832
>  failed, returning error
>  java.io.IOException: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:173)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.close(BoundedRecoveredHFilesOutputSink.java:140)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:339)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:181)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:105)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.lambda$new$0(SplitLogWorker.java:84)
>  at 
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:70)
>  at 
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834)
>  Caused by: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.configContextForNonMetaWriter(BoundedRecoveredHFilesOutputSink.java:225)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:213)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:117)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:155)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>  at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> {code}
> It is a bit odd because log says there were zero regions. Not sure what that 
> was about.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23684) NPE HFilesOutputSink

2020-03-09 Thread Duo Zhang (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-23684:
--
Fix Version/s: (was: 2.3.0)
   (was: 3.0.0)

> NPE HFilesOutputSink
> 
>
> Key: HBASE-23684
> URL: https://issues.apache.org/jira/browse/HBASE-23684
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR, wal
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Assignee: Guanghao Zhang
>Priority: Critical
>
> Enabling the new split to hfiles feature, HBASE-23286, running branch-2 tip, 
> I see this out on RegionServers:
> {code}
>  2020-01-13 17:37:08,204 INFO org.apache.hadoop.hbase.wal.OutputSink: 3 split 
> writer threads finished
>  2020-01-13 17:37:08,233 INFO org.apache.hadoop.hbase.wal.WALSplitter: 
> Processed 1007 edits across 0 regions cost 284 ms; edits skipped=76; 
> WAL=hdfs://nameservice1/hbase/genie/WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832,
>  size=128.5 M, length=134708720, corrupted=false, progress failed=true
>  2020-01-13 17:37:08,234 WARN 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: log splitting of 
> WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832
>  failed, returning error
>  java.io.IOException: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:173)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.close(BoundedRecoveredHFilesOutputSink.java:140)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:339)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:181)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:105)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.lambda$new$0(SplitLogWorker.java:84)
>  at 
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:70)
>  at 
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834)
>  Caused by: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.configContextForNonMetaWriter(BoundedRecoveredHFilesOutputSink.java:225)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:213)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:117)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:155)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>  at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> {code}
> It is a bit odd because log says there were zero regions. Not sure what that 
> was about.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23684) NPE HFilesOutputSink

2020-03-09 Thread Guanghao Zhang (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-23684.

Resolution: Duplicate

Resolved as duplicate.

> NPE HFilesOutputSink
> 
>
> Key: HBASE-23684
> URL: https://issues.apache.org/jira/browse/HBASE-23684
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR, wal
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> Enabling the new split to hfiles feature, HBASE-23286, running branch-2 tip, 
> I see this out on RegionServers:
> {code}
>  2020-01-13 17:37:08,204 INFO org.apache.hadoop.hbase.wal.OutputSink: 3 split 
> writer threads finished
>  2020-01-13 17:37:08,233 INFO org.apache.hadoop.hbase.wal.WALSplitter: 
> Processed 1007 edits across 0 regions cost 284 ms; edits skipped=76; 
> WAL=hdfs://nameservice1/hbase/genie/WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832,
>  size=128.5 M, length=134708720, corrupted=false, progress failed=true
>  2020-01-13 17:37:08,234 WARN 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: log splitting of 
> WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832
>  failed, returning error
>  java.io.IOException: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:173)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.close(BoundedRecoveredHFilesOutputSink.java:140)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:339)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:181)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:105)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.lambda$new$0(SplitLogWorker.java:84)
>  at 
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:70)
>  at 
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834)
>  Caused by: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.configContextForNonMetaWriter(BoundedRecoveredHFilesOutputSink.java:225)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:213)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:117)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:155)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>  at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> {code}
> It is a bit odd because log says there were zero regions. Not sure what that 
> was about.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HBASE-23684) NPE HFilesOutputSink

2020-03-09 Thread Guanghao Zhang (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang reassigned HBASE-23684:
--

Assignee: Guanghao Zhang

> NPE HFilesOutputSink
> 
>
> Key: HBASE-23684
> URL: https://issues.apache.org/jira/browse/HBASE-23684
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR, wal
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> Enabling the new split to hfiles feature, HBASE-23286, running branch-2 tip, 
> I see this out on RegionServers:
> {code}
>  2020-01-13 17:37:08,204 INFO org.apache.hadoop.hbase.wal.OutputSink: 3 split 
> writer threads finished
>  2020-01-13 17:37:08,233 INFO org.apache.hadoop.hbase.wal.WALSplitter: 
> Processed 1007 edits across 0 regions cost 284 ms; edits skipped=76; 
> WAL=hdfs://nameservice1/hbase/genie/WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832,
>  size=128.5 M, length=134708720, corrupted=false, progress failed=true
>  2020-01-13 17:37:08,234 WARN 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: log splitting of 
> WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832
>  failed, returning error
>  java.io.IOException: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:173)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.close(BoundedRecoveredHFilesOutputSink.java:140)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:339)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:181)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:105)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.lambda$new$0(SplitLogWorker.java:84)
>  at 
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:70)
>  at 
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834)
>  Caused by: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.configContextForNonMetaWriter(BoundedRecoveredHFilesOutputSink.java:225)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:213)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:117)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:155)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>  at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> {code}
> It is a bit odd because log says there were zero regions. Not sure what that 
> was about.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23684) NPE HFilesOutputSink

2020-03-09 Thread Guanghao Zhang (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055610#comment-17055610
 ] 

Guanghao Zhang commented on HBASE-23684:


The NPE is realted to the "tableDescpritor". It should be fixed by HBASE-23739.

> NPE HFilesOutputSink
> 
>
> Key: HBASE-23684
> URL: https://issues.apache.org/jira/browse/HBASE-23684
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR, wal
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> Enabling the new split to hfiles feature, HBASE-23286, running branch-2 tip, 
> I see this out on RegionServers:
> {code}
>  2020-01-13 17:37:08,204 INFO org.apache.hadoop.hbase.wal.OutputSink: 3 split 
> writer threads finished
>  2020-01-13 17:37:08,233 INFO org.apache.hadoop.hbase.wal.WALSplitter: 
> Processed 1007 edits across 0 regions cost 284 ms; edits skipped=76; 
> WAL=hdfs://nameservice1/hbase/genie/WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832,
>  size=128.5 M, length=134708720, corrupted=false, progress failed=true
>  2020-01-13 17:37:08,234 WARN 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker: log splitting of 
> WALs/hbasedn101.example.org,16020,1578934806382-splitting/hbasedn101.example.org%2C16020%2C1578934806382.1578937008832
>  failed, returning error
>  java.io.IOException: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:173)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.close(BoundedRecoveredHFilesOutputSink.java:140)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:339)
>  at 
> org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:181)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:105)
>  at 
> org.apache.hadoop.hbase.regionserver.SplitLogWorker.lambda$new$0(SplitLogWorker.java:84)
>  at 
> org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:70)
>  at 
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>  at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>  at java.base/java.lang.Thread.run(Thread.java:834)
>  Caused by: java.lang.NullPointerException
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.configContextForNonMetaWriter(BoundedRecoveredHFilesOutputSink.java:225)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:213)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:117)
>  at 
> org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:155)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>  at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> {code}
> It is a bit odd because log says there were zero regions. Not sure what that 
> was about.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache9 commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile enabled

2020-03-09 Thread GitBox

Apache9 commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile 
enabled
URL: https://github.com/apache/hbase/pull/1254#issuecomment-596914607
 
 
   Mind explaining what is the problem here?
   
   Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled

2020-03-09 Thread Guanghao Zhang (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055592#comment-17055592
 ] 

Guanghao Zhang commented on HBASE-23741:


Ping [~pankajkumar][~stack][~zhangduo] for reviewing.

> Data loss when WAL split to HFile enabled
> -
>
> Key: HBASE-23741
> URL: https://issues.apache.org/jira/browse/HBASE-23741
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Assignee: Guanghao Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
> Very simple steps as below,
> 1. Create table with 1 region
> 2. Insert 1 record 
> 3. Flush the table 
> 4. Scan table and observe timestamp of the inserted row
> 5. Insert same row key with same timestamp as previously inserted but with 
> different value
> 6. Kill -9 RS where table region is online
> 7. Start RS
> Scan the table and check the result, latest cell must be returned.
> Thanks [~sreenivasulureddy] for finding this issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HBASE-23741) Data loss when WAL split to HFile enabled

2020-03-09 Thread Guanghao Zhang (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang reassigned HBASE-23741:
--

Assignee: Guanghao Zhang

> Data loss when WAL split to HFile enabled
> -
>
> Key: HBASE-23741
> URL: https://issues.apache.org/jira/browse/HBASE-23741
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Assignee: Guanghao Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
> Very simple steps as below,
> 1. Create table with 1 region
> 2. Insert 1 record 
> 3. Flush the table 
> 4. Scan table and observe timestamp of the inserted row
> 5. Insert same row key with same timestamp as previously inserted but with 
> different value
> 6. Kill -9 RS where table region is online
> 7. Start RS
> Scan the table and check the result, latest cell must be returned.
> Thanks [~sreenivasulureddy] for finding this issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1262: HBASE-23953 SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1262: HBASE-23953 SimpleBalancer bug when 
second pass to fill up to min
URL: https://github.com/apache/hbase/pull/1262#issuecomment-596911968
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 33s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 24s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 14s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 36s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 43s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m  8s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 28s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 40s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  81m 44s |  hbase-server in the patch passed.  
|
   |  |   | 114m 24s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1262 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux cc8683b5615a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/testReport/
 |
   | Max. process+thread count | 5727 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile enabled

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1254: HBASE-23741 Data loss when WAL split to 
HFile enabled
URL: https://github.com/apache/hbase/pull/1254#issuecomment-596910920
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 19s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 36s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 16s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 48s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 44s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 59s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 17s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 17s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 34s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 42s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  77m  2s |  hbase-server in the patch passed.  
|
   |  |   | 110m  4s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1254 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux cfdb8259849a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/testReport/
 |
   | Max. process+thread count | 6131 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1262: HBASE-23953 SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1262: HBASE-23953 SimpleBalancer bug when 
second pass to fill up to min
URL: https://github.com/apache/hbase/pull/1262#issuecomment-596910992
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 27s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m  0s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  9s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 48s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  82m 20s |  hbase-server in the patch passed.  
|
   |  |   | 110m 20s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1262 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 42aff3f65f49 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/testReport/
 |
   | Max. process+thread count | 5847 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257#issuecomment-596909130
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 32s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 39s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  8s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 18s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   2m 59s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  4s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m  6s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  16m 13s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   2m 55s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 14s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  42m 33s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1257 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 515c20af1945 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 93 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile enabled

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1254: HBASE-23741 Data loss when WAL split to 
HFile enabled
URL: https://github.com/apache/hbase/pull/1254#issuecomment-596904750
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 34s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 30s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 57s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 22s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m  0s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  57m 56s |  hbase-server in the patch passed.  
|
   |  |   |  84m 23s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1254 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux b86bc6b647d0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/testReport/
 |
   | Max. process+thread count | 7845 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1262: HBASE-23953 SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1262: HBASE-23953 SimpleBalancer bug when 
second pass to fill up to min
URL: https://github.com/apache/hbase/pull/1262#issuecomment-596896506
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 10s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m  6s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 20s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 51s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m 21s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 41s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   1m 17s |  hbase-server: The patch 
generated 7 new + 154 unchanged - 0 fixed = 161 total (was 154)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m 20s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   3m 16s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 13s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  47m 57s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1262 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 330d5c69f805 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | checkstyle | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 83 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1262/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1254: HBASE-23741 Data loss when WAL split to HFile enabled

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1254: HBASE-23741 Data loss when WAL split to 
HFile enabled
URL: https://github.com/apache/hbase/pull/1254#issuecomment-596895276
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 16s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 15s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 27s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m  0s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  2s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  16m  5s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   2m 54s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 14s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  42m 11s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1254 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 163e63794017 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 93 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1254/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] nyl3532016 opened a new pull request #1262: HBASE-23953 SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread GitBox

nyl3532016 opened a new pull request #1262: HBASE-23953 SimpleBalancer bug when 
second pass to fill up to min
URL: https://github.com/apache/hbase/pull/1262
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23953) SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread niuyulin (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

niuyulin updated HBASE-23953:
-
Fix Version/s: 2.2.5

> SimpleBalancer bug when second pass to fill up to min
> -
>
> Key: HBASE-23953
> URL: https://issues.apache.org/jira/browse/HBASE-23953
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 2.2.0
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Major
> Fix For: 2.2.5
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23953) SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread niuyulin (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

niuyulin updated HBASE-23953:
-
Affects Version/s: 2.2.0

> SimpleBalancer bug when second pass to fill up to min
> -
>
> Key: HBASE-23953
> URL: https://issues.apache.org/jira/browse/HBASE-23953
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Affects Versions: 2.2.0
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23953) SimpleBalancer bug when second pass to fill up to min

2020-03-09 Thread niuyulin (Jira)

niuyulin created HBASE-23953:


 Summary: SimpleBalancer bug when second pass to fill up to min
 Key: HBASE-23953
 URL: https://issues.apache.org/jira/browse/HBASE-23953
 Project: HBase
  Issue Type: Bug
  Components: Balancer
Reporter: niuyulin
Assignee: niuyulin






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache9 commented on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

Apache9 commented on issue #1261: HBASE-23952: Address thread safety issue with 
Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596876547
 
 
   EventExecutor is a single thread executor...
   
   EventExecutorGroup is what you expected, a thread pool...
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] markrmiller edited a comment on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

markrmiller edited a comment on issue #1261: HBASE-23952: Address thread safety 
issue with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596869131
 
 
   I think the issue is that Netty can use different threads to access this 
data structure, ie:
   
   org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext
   ```
   static void invokeUserEventTriggered(final AbstractChannelHandlerContext 
next, final Object event) {
   ObjectUtil.checkNotNull(event, "event");
   EventExecutor executor = next.executor();
   if (executor.inEventLoop()) {
   next.invokeUserEventTriggered(event);
   } else {
   executor.execute(new Runnable() {
   @Override
   public void run() {
   next.invokeUserEventTriggered(event);
   }
   });
   }
   }
   ```
   
   
   If more than one thread is going to access a shared field, you have to 
publish it properly - either with memory barriers or it has to be final or 
effectively final. It seems very difficult to ensure that this is thread safe 
or will remain thread safe over time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on issue #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache9 commented on issue #1260: HBASE-23881 Ensure Netty client receives at 
least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596869496
 
 
   @joshelser If you do not mind, I plan to make a overall fix based on your 
patch and what's important, add more comments to say the rules here, to avoid 
further confusing of others.
   
   Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] markrmiller edited a comment on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

markrmiller edited a comment on issue #1261: HBASE-23952: Address thread safety 
issue with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596869131
 
 
   I think the issue is that Netty can use different threads to access this 
data structure, ie:
   
   ```
   static void invokeUserEventTriggered(final AbstractChannelHandlerContext 
next, final Object event) {
   ObjectUtil.checkNotNull(event, "event");
   EventExecutor executor = next.executor();
   if (executor.inEventLoop()) {
   next.invokeUserEventTriggered(event);
   } else {
   executor.execute(new Runnable() {
   @Override
   public void run() {
   next.invokeUserEventTriggered(event);
   }
   });
   }
   }
   ```
   
   
   If more than one thread is going to access a shared field, you have to 
publish it properly - either with memory barriers or it has to be final or 
effectively final. It seems very difficult to ensure that this is thread safe 
or will remain thread safe over time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] markrmiller commented on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

markrmiller commented on issue #1261: HBASE-23952: Address thread safety issue 
with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596869131
 
 
   I think the issue is that Netty can use different threads to access this 
data structure, ie:
   
   `
   static void invokeUserEventTriggered(final AbstractChannelHandlerContext 
next, final Object event) {
   ObjectUtil.checkNotNull(event, "event");
   EventExecutor executor = next.executor();
   if (executor.inEventLoop()) {
   next.invokeUserEventTriggered(event);
   } else {
   executor.execute(new Runnable() {
   @Override
   public void run() {
   next.invokeUserEventTriggered(event);
   }
   });
   }
   }
   `
   
   If more than one thread is going to access a shared field, you have to 
publish it properly - either with memory barriers or it has to be final or 
effectively final. It seems very difficult to ensure that this is thread safe 
or will remain thread safe over time.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1260: HBASE-23881 Ensure Netty client receives 
at least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596868693
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 32s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 35s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 31s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 28s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 51s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 31s |  hbase-client in master failed.  |
   | -0 :warning: |  javadoc  |   0m 22s |  hbase-examples in master failed.  |
   | -0 :warning: |  javadoc  |   0m 49s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 21s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   7m 16s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 25s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 25s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 26s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 42s |  hbase-client in the patch failed.  |
   | -0 :warning: |  javadoc  |   1m  5s |  hbase-server in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 34s |  hbase-examples in the patch failed. 
 |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 38s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  | 121m 39s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m 49s |  hbase-examples in the patch 
passed.  |
   |  |   | 165m 43s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1260 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 56d493d1085e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-examples.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-examples.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/testReport/
 |
   | Max. process+thread count | 5394 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-examples U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on issue #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache9 commented on issue #1260: HBASE-23881 Ensure Netty client receives at 
least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596867525
 
 
   Think again, the end condition here should be:
   
   1. SaslClient.isComplete returns true
   2. Once we have sent out a SaslToken, then we need to get one back.
   
   Let me check the code again.
   
   Found a bug through a roughly looking...
   
   ```
 public byte[] getInitialResponse() throws SaslException {
   if (saslClient.hasInitialResponse()) {
 return saslClient.evaluateChallenge(EMPTY_TOKEN);
   } else {
 return EMPTY_TOKEN;
   }
 }
   ```
   
   Where EMPTY_TOKEN is a byte[0], but in upper layer. we will check null to 
determine whether we have an initial response...


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1260: HBASE-23881 Ensure Netty client receives 
at least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596866986
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 32s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 36s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m 10s |  master passed  |
   | +1 :green_heart: |  compile  |   2m  0s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 29s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 20s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   6m  1s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m  0s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m  0s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 22s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 22s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 57s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  | 120m  6s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  unit  |   3m 17s |  hbase-examples in the patch 
passed.  |
   |  |   | 158m 57s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1260 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 48569bd7cf12 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/testReport/
 |
   | Max. process+thread count | 5372 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-examples U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on issue #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache9 commented on issue #1260: HBASE-23881 Ensure Netty client receives at 
least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596865027
 
 
   OK, I checked the PlainClient, so the assumption here is that, client should 
have get at least one sasl response from server, even if it has reported as 
complete? But waht if later sasl client reports complete? Do we still need to 
wait for the server response? I'm a bit confused.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache9 commented on a change in pull request #1260: HBASE-23881 Ensure Netty 
client receives at least one response before…
URL: https://github.com/apache/hbase/pull/1260#discussion_r390040132
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/security/NettyHBaseSaslRpcClientHandler.java
 ##
 @@ -114,7 +122,10 @@ public void handlerAdded(ChannelHandlerContext ctx) {
   if (initialResponse != null) {
 writeResponse(ctx, initialResponse);
   }
-  tryComplete(ctx);
+  // HBASE-23881 We do not want to check if the SaslClient thinks the 
handshake is
 
 Review comment:
   Is this correct? 
   
   The comment is
   ```
 * Determines whether the authentication exchange has completed.
 * This method may be called at any time, but typically, it
 * will not be called until the caller has received indication
 * from the server
 * (in a protocol-specific manner) that the exchange has completed.
   ```
   
   It does not say that we 'must' call it after we receive the reponse from 
server.
   
   And I'm curious that, what is the sasl client implementation? It just marks 
itself as complete succeeded, without any server response, and then after the 
server response it could revert itself as auth failed? I think this is a broken 
sasl client implementation?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache9 commented on a change in pull request #1260: HBASE-23881 Ensure Netty 
client receives at least one response before…
URL: https://github.com/apache/hbase/pull/1260#discussion_r390039053
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
 ##
 @@ -149,6 +149,10 @@ private void scheduleRelogin(Throwable error) {
 if (error instanceof FallbackDisallowedException) {
   return;
 }
+if (!provider.canRetry()) {
 
 Review comment:
   Why this change?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

Apache9 commented on issue #1261: HBASE-23952: Address thread safety issue with 
Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596846820
 
 
   In netty a handler will only be called in one thread, and we only access the 
id2Call map in the handler specific methods, so I do not think we need to 
change it to `ConcurrentHashMap`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23741) Data loss when WAL split to HFile enabled

2020-03-09 Thread Michael Stack (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055482#comment-17055482
 ] 

Michael Stack commented on HBASE-23741:
---

Good find [~pankajkumar]  (and [~sreenivasulureddy] )

> Data loss when WAL split to HFile enabled
> -
>
> Key: HBASE-23741
> URL: https://issues.apache.org/jira/browse/HBASE-23741
> Project: HBase
>  Issue Type: Bug
>  Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
> Very simple steps as below,
> 1. Create table with 1 region
> 2. Insert 1 record 
> 3. Flush the table 
> 4. Scan table and observe timestamp of the inserted row
> 5. Insert same row key with same timestamp as previously inserted but with 
> different value
> 6. Kill -9 RS where table region is online
> 7. Start RS
> Scan the table and check the result, latest cell must be returned.
> Thanks [~sreenivasulureddy] for finding this issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1261: HBASE-23952: Address thread safety issue 
with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596844099
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 34s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 30s |  master passed  |
   | +0 :ok: |  spotbugs  |   1m  3s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   1m  0s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  3s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 29s |  hbase-client: The patch 
generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  16m  7s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  findbugs  |   1m  7s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 14s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  37m 21s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1261 |
   | JIRA Issue | HBASE-23952 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 8fe714216573 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Max. process+thread count | 93 (vs. ulimit of 1) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1260: HBASE-23881 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1260: HBASE-23881 Ensure Netty client receives 
at least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596843946
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   4m 40s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 32s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 53s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   2m  0s |  master passed  |
   | +0 :ok: |  spotbugs  |   0m 42s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m 17s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 36s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 13s |  hbase-examples: The patch 
generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 40s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   3m 13s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 32s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  56m 44s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1260 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 74c38e11ee6f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | checkstyle | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-examples.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 83 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-examples U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1260/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1261: HBASE-23952: Address thread safety issue 
with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596843144
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 20s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 27s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 32s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 26s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 56s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 34s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 34s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 15s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m  5s |  hbase-client in the patch passed.  
|
   |  |   |  32m 46s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1261 |
   | JIRA Issue | HBASE-23952 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 05db727839ae 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/testReport/
 |
   | Max. process+thread count | 376 (vs. ulimit of 1) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1261: HBASE-23952: Address thread safety issue 
with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261#issuecomment-596841920
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 33s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 32s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  4s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 28s |  hbase-client in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 15s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 30s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 30s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 53s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 27s |  hbase-client in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m  0s |  hbase-client in the patch passed.  
|
   |  |   |  27m 24s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1261 |
   | JIRA Issue | HBASE-23952 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 827816f88e7e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/testReport/
 |
   | Max. process+thread count | 442 (vs. ulimit of 1) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1261/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] markrmiller opened a new pull request #1261: HBASE-23952: Address thread safety issue with Map used in BufferCallB…

2020-03-09 Thread GitBox

markrmiller opened a new pull request #1261: HBASE-23952: Address thread safety 
issue with Map used in BufferCallB…
URL: https://github.com/apache/hbase/pull/1261
 
 
   …eforeInitHandler.
   
   id2Call is a HashMap and has a call back method that accesses it that can be 
run via an executor as well as another method accessing it that can be run from 
a different thread.
   id2Call should likely be a ConcurrentHashMap to be shared like this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (HBASE-23952) Address thread safety issue with Map used in BufferCallBeforeInitHandler.

2020-03-09 Thread Mark Robert Miller (Jira)

Mark Robert Miller created HBASE-23952:
--

 Summary: Address thread safety issue with Map used in 
BufferCallBeforeInitHandler.
 Key: HBASE-23952
 URL: https://issues.apache.org/jira/browse/HBASE-23952
 Project: HBase
  Issue Type: Bug
Affects Versions: master
Reporter: Mark Robert Miller


id2Call is a HashMap and has a call back method that accesses it that can be 
run via an executor as well as another method accessing it that can be run from 
a different thread.

id2Call should likely be a ConcurrentHashMap to be shared like this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion 
trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#issuecomment-596829421
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 35s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 36s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   0m 34s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   1m  6s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   1m  5s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  1s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 31s |  hbase-client: The patch 
generated 0 new + 0 unchanged - 21 fixed = 0 total (was 21)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  15m 44s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  findbugs  |   1m  8s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 14s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  37m  7s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1259 |
   | JIRA Issue | HBASE-23951 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 70f04b541d95 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / dacceba0c7 |
   | Max. process+thread count | 93 (vs. ulimit of 1) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion 
trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#issuecomment-596828273
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   5m 20s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  5s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 37s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 32s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 42s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 30s |  hbase-client in branch-2 failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m  0s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 31s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 31s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 35s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 28s |  hbase-client in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   2m 11s |  hbase-client in the patch passed.  
|
   |  |   |  32m 39s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1259 |
   | JIRA Issue | HBASE-23951 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 91020c7032ca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / dacceba0c7 |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/testReport/
 |
   | Max. process+thread count | 468 (vs. ulimit of 1) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] joshelser commented on issue #1260: HBASE-23381 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

joshelser commented on issue #1260: HBASE-23381 Ensure Netty client receives at 
least one response before…
URL: https://github.com/apache/hbase/pull/1260#issuecomment-596827894
 
 
   This was a bear to figure out. This is only for Netty -- NIO works fine.
   
   This PR is also only valid for master -- due to the ZK-less MasterRegistry 
implementation. I think earlier 2.x branches will need custom patches. I'll 
pick those up after we're good on master.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] joshelser opened a new pull request #1260: HBASE-23381 Ensure Netty client receives at least one response before…

2020-03-09 Thread GitBox

joshelser opened a new pull request #1260: HBASE-23381 Ensure Netty client 
receives at least one response before…
URL: https://github.com/apache/hbase/pull/1260
 
 
   … considering SASL negotiation complete
   
   The PLAIN mechanism test added in the Shade authentication example has
   different semantics than GSSAPI mechanism -- the client reports that the
   handshake is done after the original challenge is computed. The javadoc
   on SaslClient, however, tells us that we need to wait for a response
   from the server before proceeding.
   
   The client, best as I can see, does not receive any data from HBase;
   however the application semantics (e.g. throw an exception on auth'n
   error) do not work as we intend as a result of this bug.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1259: HBASE-23951: Avoid high speed recursion 
trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259#issuecomment-596826184
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 35s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  5s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 30s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 26s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 20s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  2s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 27s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 27s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 17s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 25s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 58s |  hbase-client in the patch passed.  
|
   |  |   |  24m 43s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1259 |
   | JIRA Issue | HBASE-23951 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux a7f729cb3dd2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | branch-2 / dacceba0c7 |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/testReport/
 |
   | Max. process+thread count | 623 (vs. ulimit of 1) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1259/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1258: HBASE-23932 Minor improvements to Region 
Normalizer
URL: https://github.com/apache/hbase/pull/1258#issuecomment-596823332
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  8s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 24s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 32s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 43s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 56s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 13s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 13s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 30s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 42s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  77m 48s |  hbase-server in the patch passed.  
|
   |  |   | 110m  0s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1258 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux b46db2d14bad 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/testReport/
 |
   | Max. process+thread count | 5414 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated 
setStartRow from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596821396
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   4m 53s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 36s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  4s |  master passed  |
   | +1 :green_heart: |  compile  |   4m 25s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  0s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 58s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m  9s |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m 10s |  the patch passed  |
   | +1 :green_heart: |  javac  |   4m 10s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 47s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 55s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 54s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  |  49m  5s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  unit  |  10m  7s |  hbase-mapreduce in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   2m 25s |  hbase-thrift in the patch passed.  
|
   | +1 :green_heart: |  unit  |   3m 28s |  hbase-endpoint in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   7m 42s |  hbase-backup in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m  9s |  hbase-it in the patch passed.  |
   | +1 :green_heart: |  unit  |   2m 45s |  hbase-rest in the patch passed.  |
   |  |   | 122m 45s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1220 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 853796db7a5b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/testReport/
 |
   | Max. process+thread count | 9561 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-mapreduce hbase-thrift 
hbase-endpoint hbase-backup hbase-it hbase-rest U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1258: HBASE-23932 Minor improvements to Region 
Normalizer
URL: https://github.com/apache/hbase/pull/1258#issuecomment-596820271
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   8m 43s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 46s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  1s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 24s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 50s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  4s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  4s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 15s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |  76m 50s |  hbase-server in the patch failed.  |
   |  |   | 114m  2s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1258 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 6be02d1c4fb0 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/testReport/
 |
   | Max. process+thread count | 6046 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated 
setStartRow from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596819386
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |  15m 35s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 47s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  10m 16s |  master passed  |
   | +1 :green_heart: |  compile  |   7m  6s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   7m 35s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 44s |  hbase-client in master failed.  |
   | -0 :warning: |  javadoc  |   0m 28s |  hbase-backup in master failed.  |
   | -0 :warning: |  javadoc  |   0m 31s |  hbase-mapreduce in master failed.  |
   | -0 :warning: |  javadoc  |   0m 34s |  hbase-rest in master failed.  |
   | -0 :warning: |  javadoc  |   1m 11s |  hbase-server in master failed.  |
   | -0 :warning: |  javadoc  |   1m 23s |  hbase-thrift in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |  10m 10s |  the patch passed  |
   | +1 :green_heart: |  compile  |   5m 58s |  the patch passed  |
   | +1 :green_heart: |  javac  |   5m 58s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   7m 11s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 33s |  hbase-client in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 55s |  hbase-server in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 24s |  hbase-mapreduce in the patch 
failed.  |
   | -0 :warning: |  javadoc  |   1m 14s |  hbase-thrift in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 22s |  hbase-backup in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 29s |  hbase-rest in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 20s |  hbase-client in the patch passed.  
|
   | -0 :warning: |  unit  |   8m  4s |  hbase-server in the patch failed.  |
   | -0 :warning: |  unit  |  10m 10s |  hbase-mapreduce in the patch failed.  |
   | +1 :green_heart: |  unit  |   2m 50s |  hbase-thrift in the patch passed.  
|
   | +1 :green_heart: |  unit  |   3m 43s |  hbase-endpoint in the patch 
passed.  |
   | -0 :warning: |  unit  |   4m 50s |  hbase-backup in the patch failed.  |
   | +1 :green_heart: |  unit  |   1m  9s |  hbase-it in the patch passed.  |
   | +1 :green_heart: |  unit  |   3m 47s |  hbase-rest in the patch passed.  |
   |  |   | 114m 34s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1220 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 0d60caf88caf 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-backup.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-mapreduce.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-s

[GitHub] [hbase] markrmiller opened a new pull request #1259: HBASE-23951: Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread GitBox

markrmiller opened a new pull request #1259: HBASE-23951: Avoid high speed 
recursion trap in AsyncRequestFutureImpl.
URL: https://github.com/apache/hbase/pull/1259
 
 
   While working on branch-2, I ran into an issue where a retryable error kept 
occurring and code in AsyncRequestFutureImpl would reduce the backoff wait to 0 
and extremely rapidly eat up a of thread stack space with recursive retry 
calls. This little patch stops the backoff wait kill after 3 retries. Chosen 
kind of arbitrarily, perhaps 5 is the right number, but I find large retry 
counts tend to hide things and that has made me default to fairly conservative 
in all my arbitrary number picking.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] joshelser commented on a change in pull request #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #1258: HBASE-23932 Minor 
improvements to Region Normalizer
URL: https://github.com/apache/hbase/pull/1258#discussion_r39985
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 ##
 @@ -1868,52 +1889,50 @@ public RegionNormalizer getRegionNormalizer() {
*is globally disabled)
*/
   public boolean normalizeRegions() throws IOException {
-if (!isInitialized()) {
-  LOG.debug("Master has not been initialized, don't run region 
normalizer.");
-  return false;
-}
-if (this.getServerManager().isClusterShutdown()) {
-  LOG.info("Cluster is shutting down, don't run region normalizer.");
+if (regionNormalizerTracker == null || 
!regionNormalizerTracker.isNormalizerOn()) {
+  LOG.debug("Region normalization is disabled, don't run region 
normalizer.");
   return false;
 }
-if (isInMaintenanceMode()) {
-  LOG.info("Master is in maintenance mode, don't run region normalizer.");
+if (skipRegionManagementAction("region normalizer")) {
   return false;
 }
-if (!this.regionNormalizerTracker.isNormalizerOn()) {
-  LOG.debug("Region normalization is disabled, don't run region 
normalizer.");
+if (assignmentManager.hasRegionsInTransition()) {
   return false;
 }
 
 synchronized (this.normalizer) {
   // Don't run the normalizer concurrently
+
   List allEnabledTables = new ArrayList<>(
 this.tableStateManager.getTablesInStates(TableState.State.ENABLED));
 
   Collections.shuffle(allEnabledTables);
 
   for (TableName table : allEnabledTables) {
-if (isInMaintenanceMode()) {
-  LOG.debug("Master is in maintenance mode, stop running region 
normalizer.");
-  return false;
-}
-
 TableDescriptor tblDesc = getTableDescriptors().get(table);
 if (table.isSystemTable() || (tblDesc != null &&
 !tblDesc.isNormalizationEnabled())) {
   LOG.trace("Skipping normalization for {}, as it's either system"
   + " table or doesn't have auto normalization turned on", table);
   continue;
 }
-List plans = 
this.normalizer.computePlanForTable(table);
-if (plans != null) {
-  for (NormalizationPlan plan : plans) {
-plan.execute(asyncClusterConnection.toConnection().getAdmin());
-if (plan.getType() == PlanType.SPLIT) {
-  splitPlanCount++;
-} else if (plan.getType() == PlanType.MERGE) {
-  mergePlanCount++;
-}
+
+// make one last check that the cluster isn't shutting down before 
proceeding.
+if (skipRegionManagementAction("region normalizer")) {
+  return false;
+}
+
+final List plans = 
this.normalizer.computePlanForTable(table);
+if (CollectionUtils.isEmpty(plans)) {
+  return true;
+}
+
+for (NormalizationPlan plan : plans) {
+  plan.execute(asyncClusterConnection.toConnection().getAdmin());
 
 Review comment:
   `toConnection()` returns a cached `Connection` but we make a new `Admin` 
object every time (at least via `AsyncClusterConnectionImpl`. Should we just 
cache this `Admin` once for all `plans`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] joshelser commented on a change in pull request #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #1258: HBASE-23932 Minor 
improvements to Region Normalizer
URL: https://github.com/apache/hbase/pull/1258#discussion_r38814
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 ##
 @@ -1868,52 +1889,50 @@ public RegionNormalizer getRegionNormalizer() {
*is globally disabled)
*/
   public boolean normalizeRegions() throws IOException {
-if (!isInitialized()) {
-  LOG.debug("Master has not been initialized, don't run region 
normalizer.");
-  return false;
-}
-if (this.getServerManager().isClusterShutdown()) {
-  LOG.info("Cluster is shutting down, don't run region normalizer.");
+if (regionNormalizerTracker == null || 
!regionNormalizerTracker.isNormalizerOn()) {
+  LOG.debug("Region normalization is disabled, don't run region 
normalizer.");
   return false;
 }
-if (isInMaintenanceMode()) {
-  LOG.info("Master is in maintenance mode, don't run region normalizer.");
+if (skipRegionManagementAction("region normalizer")) {
   return false;
 }
-if (!this.regionNormalizerTracker.isNormalizerOn()) {
-  LOG.debug("Region normalization is disabled, don't run region 
normalizer.");
+if (assignmentManager.hasRegionsInTransition()) {
   return false;
 }
 
 synchronized (this.normalizer) {
   // Don't run the normalizer concurrently
+
   List allEnabledTables = new ArrayList<>(
 this.tableStateManager.getTablesInStates(TableState.State.ENABLED));
 
   Collections.shuffle(allEnabledTables);
 
   for (TableName table : allEnabledTables) {
-if (isInMaintenanceMode()) {
-  LOG.debug("Master is in maintenance mode, stop running region 
normalizer.");
-  return false;
-}
-
 TableDescriptor tblDesc = getTableDescriptors().get(table);
 if (table.isSystemTable() || (tblDesc != null &&
 !tblDesc.isNormalizationEnabled())) {
   LOG.trace("Skipping normalization for {}, as it's either system"
   + " table or doesn't have auto normalization turned on", table);
   continue;
 }
-List plans = 
this.normalizer.computePlanForTable(table);
-if (plans != null) {
-  for (NormalizationPlan plan : plans) {
-plan.execute(asyncClusterConnection.toConnection().getAdmin());
-if (plan.getType() == PlanType.SPLIT) {
-  splitPlanCount++;
-} else if (plan.getType() == PlanType.MERGE) {
-  mergePlanCount++;
-}
+
+// make one last check that the cluster isn't shutting down before 
proceeding.
+if (skipRegionManagementAction("region normalizer")) {
+  return false;
+}
+
+final List plans = 
this.normalizer.computePlanForTable(table);
+if (CollectionUtils.isEmpty(plans)) {
 
 Review comment:
   Nice -- wraps a null-check around `isEmpty()`. Had to look it up.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] joshelser commented on a change in pull request #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #1258: HBASE-23932 Minor 
improvements to Region Normalizer
URL: https://github.com/apache/hbase/pull/1258#discussion_r38243
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 ##
 @@ -1722,21 +1723,36 @@ public boolean balance() throws IOException {
 return balance(false);
   }
 
-  public boolean balance(boolean force) throws IOException {
-// if master not initialized, don't run balancer.
+  /**
+   * Checks master state before initiating action over region topology.
+   * @param action the name of the action under consideration, for logging.
+   * @return {@code true} when the caller should exit early, {@code false} 
otherwise.
+   */
+  private boolean skipRegionManagementAction(final String action) {
 
 Review comment:
   Nice!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257#issuecomment-596814512
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   5m 34s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 45s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 23s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 42s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 43s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   7m 13s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 22s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 22s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 54s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 46s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | -0 :warning: |  unit  |  92m 44s |  hbase-server in the patch failed.  |
   |  |   | 131m  0s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1257 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 5c7dde5b8e02 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/testReport/
 |
   | Max. process+thread count | 5491 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23951) Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread Mark Robert Miller (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Robert Miller updated HBASE-23951:
---
Description: While working on branch-2, I ran into an issue where a 
retryable error kept occurring and code in AsyncRequestFutureImpl would reduce 
the backoff wait to 0 and extremely rapidly eat up a of thread stack space with 
recursive retry calls. This little patch stops the backoff wait kill after 3 
retries. Chosen kind of arbitrarily, perhaps 5 is the right number, but I find 
large retry counts tend to hide things and that has made me default to fairly 
conservative in all my arbitrary number picking.  (was: While working on 
branch-2, I ran into an issue where a retryable error kept occurring and code 
in )

> Avoid high speed recursion trap in AsyncRequestFutureImpl.
> --
>
> Key: HBASE-23951
> URL: https://issues.apache.org/jira/browse/HBASE-23951
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: Mark Robert Miller
>Priority: Minor
>
> While working on branch-2, I ran into an issue where a retryable error kept 
> occurring and code in AsyncRequestFutureImpl would reduce the backoff wait to 
> 0 and extremely rapidly eat up a of thread stack space with recursive retry 
> calls. This little patch stops the backoff wait kill after 3 retries. Chosen 
> kind of arbitrarily, perhaps 5 is the right number, but I find large retry 
> counts tend to hide things and that has made me default to fairly 
> conservative in all my arbitrary number picking.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-18418) Remove apache_hbase_topology from dev-support

2020-03-09 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-18418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055440#comment-17055440
 ] 

Hudson commented on HBASE-18418:


Results for branch branch-2
[build #2541 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//JDK11_Nightly_Build_Report/]


(x) {color:red}-1 source release artifact{color}
-- See build output for details.


(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//console].


> Remove apache_hbase_topology from dev-support
> -
>
> Key: HBASE-18418
> URL: https://issues.apache.org/jira/browse/HBASE-18418
> Project: HBase
>  Issue Type: Task
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-18418_v1.patch
>
>
> As [~misty] noted on dev@, we still keep a folder containing the Apache HBase 
> topology for [clusterdock|https://github.com/clusterdock/framework] in 
> dev-support. Since the {{clusterdock}} org on GitHub will likely always have 
> a more up-to-date iteration of this code (and since I plan on showing it some 
> love there in the next few weeks), we'd probably be better off removing it 
> from HBase. Any objections?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23951) Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread Mark Robert Miller (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Robert Miller updated HBASE-23951:
---
Description: While working on branch-2, I ran into an issue where a 
retryable error kept occurring and code in 

> Avoid high speed recursion trap in AsyncRequestFutureImpl.
> --
>
> Key: HBASE-23951
> URL: https://issues.apache.org/jira/browse/HBASE-23951
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: Mark Robert Miller
>Priority: Minor
>
> While working on branch-2, I ran into an issue where a retryable error kept 
> occurring and code in 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23946) [JDK11] Yetus should vote -0 for known JDK11 issues

2020-03-09 Thread Hudson (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055439#comment-17055439
 ] 

Hudson commented on HBASE-23946:


Results for branch branch-2
[build #2541 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//JDK11_Nightly_Build_Report/]


(x) {color:red}-1 source release artifact{color}
-- See build output for details.


(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2541//console].


> [JDK11] Yetus should vote -0 for known JDK11 issues
> ---
>
> Key: HBASE-23946
> URL: https://issues.apache.org/jira/browse/HBASE-23946
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> So as t avoid everyone being depressed by failing builds, we should have 
> Yetus vote -0 instead of -1 for checks that are known to fail with JDK11. Off 
> the top of my head, these should be {{javadocs}} and {{unit}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23951) Avoid high speed recursion trap in AsyncRequestFutureImpl.

2020-03-09 Thread Mark Robert Miller (Jira)

Mark Robert Miller created HBASE-23951:
--

 Summary: Avoid high speed recursion trap in AsyncRequestFutureImpl.
 Key: HBASE-23951
 URL: https://issues.apache.org/jira/browse/HBASE-23951
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.3.0
Reporter: Mark Robert Miller






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389996084
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389993101
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389992344
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
 
 Review comment:
   This has the potential to bash the Namenode.
   
   To make space quotas work, each RegionServer will already report the sizes 
of regions to the Master. However, this isn't guaranteed to be up to date, nor 
is this exposed over RPC.
   
   I can't think of anything else which would (potentially) less impactful on 
the Namenode, so maybe we just watch this and make sure users are aware that 
they need to not run lots of these all at the same time.

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389997716
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389960883
 
 

 ##
 File path: hbase-tools/README.md
 ##
 @@ -0,0 +1,87 @@
+
+
+# Apache HBase Tool for merging regions
+
+_RegionsMerger_ is an utility tool for manually merging bunch of regions of
+a given table. It's mainly useful on situations when an HBase cluster has too
+many regions per RegionServers, and many of these regions are small enough that
+it can be merged together, reducing the total number of regions in the cluster
+and releasing RegionServers overall memory resources.
+
+This may happen for mistakenly pre-splits, or after a purge in table
+data, as regions would not be automatically merged.
+
+## Setup
+Make sure HBase tools jar is added to HBase classpath:
+
+```
+export HBASE_CLASSPATH=$HBASE_CLASSPATH:./hbase-tools-1.1.0-SNAPSHOT.jar
+```
+
+## Usage
+
+_RegionsMerger_ requires two arguments as parameters: 1) The name of the table
+to have regions merged; 2) The desired total number of regions for the informed
+table. For example, to merge all regions of table `my-table` until it gets to a
+total of 5 regions, assuming the _setup_ step above has been performed:
+
+```
+$ hbase org.apache.hbase.RegionsMerger my-table 5
+```
+
+## Implementation Details
+
+_RegionsMerger_ uses client API
+_org.apache.hadoop.hbase.client.Admin.getRegions_ to fetch the list of regions
+for the specified table, iterates through the resulting list, identifying pairs
+of adjacent regions. For each pair found, it submits a merge request using
+_org.apache.hadoop.hbase.client.Admin.mergeRegionsAsync_ client API method.
+This means multiple merge requests had been sent once the whole list has been
+iterated.
+
+Assuming all these requests complete, resulting total number of
+regions will be ceil((original total number regions)/2). This resulting total
 
 Review comment:
   nit: use english instead of pseudocode for a README.
   
   > Assuming that all merges issued by the RegionsMerger are successful, the 
resulting number of regions will be no more than half the original number of 
regions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389993901
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389993659
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389992585
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
 
 Rev

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389993225
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389994590
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389962346
 
 

 ##
 File path: hbase-tools/pom.xml
 ##
 @@ -0,0 +1,202 @@
+
+http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+4.0.0
+
+hbase-operator-tools
+org.apache.hbase.operator.tools
+1.1.0-SNAPSHOT
+..
+
+
+
+hbase-tools
+Apache HBase - HBase Tools
+Utility Maintenance tools for HBase 2+
+
+2.2.1
+2.11.1
+
+
+
+
+junit
+junit
+4.12
+test
+
+
+org.apache.logging.log4j
+log4j-slf4j-impl
+${log4j2.version}
+
+
+
+
+
+org.apache.hbase
+hbase-server
+${hbase.version}
+provided
+
+
+org.apache.hbase
+hbase-shaded-testing-util
+${hbase.version}
+
+
+org.apache.hbase
+hbase-zookeeper
+${hbase.version}
+provided
+test-jar
 
 Review comment:
   Does the reliance on test-jar's here indicate that we have more that should 
be in hbase-shaded-testing-util that isn't there today? Something more we can 
follow up on?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389993730
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389961596
 
 

 ##
 File path: hbase-tools/README.md
 ##
 @@ -0,0 +1,87 @@
+
+
+# Apache HBase Tool for merging regions
+
+_RegionsMerger_ is an utility tool for manually merging bunch of regions of
+a given table. It's mainly useful on situations when an HBase cluster has too
+many regions per RegionServers, and many of these regions are small enough that
+it can be merged together, reducing the total number of regions in the cluster
+and releasing RegionServers overall memory resources.
+
+This may happen for mistakenly pre-splits, or after a purge in table
+data, as regions would not be automatically merged.
+
+## Setup
+Make sure HBase tools jar is added to HBase classpath:
+
+```
+export HBASE_CLASSPATH=$HBASE_CLASSPATH:./hbase-tools-1.1.0-SNAPSHOT.jar
+```
+
+## Usage
+
+_RegionsMerger_ requires two arguments as parameters: 1) The name of the table
+to have regions merged; 2) The desired total number of regions for the informed
+table. For example, to merge all regions of table `my-table` until it gets to a
+total of 5 regions, assuming the _setup_ step above has been performed:
+
+```
+$ hbase org.apache.hbase.RegionsMerger my-table 5
+```
+
+## Implementation Details
+
+_RegionsMerger_ uses client API
+_org.apache.hadoop.hbase.client.Admin.getRegions_ to fetch the list of regions
+for the specified table, iterates through the resulting list, identifying pairs
+of adjacent regions. For each pair found, it submits a merge request using
+_org.apache.hadoop.hbase.client.Admin.mergeRegionsAsync_ client API method.
+This means multiple merge requests had been sent once the whole list has been
+iterated.
+
+Assuming all these requests complete, resulting total number of
+regions will be ceil((original total number regions)/2). This resulting total
+might not be equal to the target value passed as parameter, in which case
+_RegionsMerger_ will perform another round of merge requests, this time over
+the current existing regions (it fetches another list of regions from
+  _org.apache.hadoop.hbase.client.Admin.getRegions_).
+
+Merge requests are processed asynchronously. HBase may take a certain time to
+complete some merge requests, so _RegionsMerger_ may perform some sleep between
+rounds of regions iteration for sending requests. The specific amount of time 
is
+configured by `hbase.tools.merge.sleep` property, in milliseconds, and it
+defaults to `2000`(2 seconds).
+
+While iterating through the list of regions, once a pair of adjacent regions is
+detected, _RegionsMerger_ checks the current file system size of each region 
(excluding MOB data),
+before deciding to submit the merge request for the given regions. If the sum 
of
+both regions size exceeds a threshold, merge will not be attempted.
+This threshold is a configurable percentage of `hbase.hregion.max.filesize`
+value, and is applied to avoid merged regions from getting immediately split
+after the merge completes, which would happen automatically if the resulting
+region size reaches `hbase.hregion.max.filesize` value. The percentage of
+`hbase.hregion.max.filesize` is a double value configurable via
+`hbase.tools.merge.upper.mark` property and it defaults to `0.9`.
+
+Given this `hbase.hregion.max.filesize` restriction for merge results, it may 
be
+impossible to achieve the desired total number of regions, in certain 
scenarios.
 
 Review comment:
   Nit: drop ", in certain scenarios."
   
   You've already said "it may be impossible" -- that is sufficient to get your 
point across.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #46: HBASE-23562 [operator tools] Add a RegionsMerge tool that allows for …

2020-03-09 Thread GitBox

joshelser commented on a change in pull request #46: HBASE-23562 [operator 
tools] Add a RegionsMerge tool that allows for …
URL: 
https://github.com/apache/hbase-operator-tools/pull/46#discussion_r389998473
 
 

 ##
 File path: hbase-tools/src/main/java/org/apache/hbase/RegionsMerger.java
 ##
 @@ -0,0 +1,259 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import static org.apache.hadoop.hbase.HConstants.CATALOG_FAMILY;
+import static org.apache.hadoop.hbase.HConstants.REGIONINFO_QUALIFIER;
+import static org.apache.hadoop.hbase.HConstants.STATE_QUALIFIER;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.LongAdder;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.CompareOperator;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.filter.FilterList;
+import org.apache.hadoop.hbase.filter.RowFilter;
+import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.filter.SubstringComparator;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase maintenance tool for merging regions of a specific table, until a 
target number of regions
+ * for the table is reached, or no more merges can complete due to limit in 
resulting merged
+ * region size.
+ */
+public class RegionsMerger extends Configured implements 
org.apache.hadoop.util.Tool {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(RegionsMerger.class.getName());
+  public static final String RESULTING_REGION_UPPER_MARK = 
"hbase.tools.merge.upper.mark";
+  public static final String SLEEP = "hbase.tools.merge.sleep";
+  public static final String MAX_ROUNDS_IDDLE = 
"hbase.tools.max.iterations.blocked";
+
+  private final Configuration conf;
+  private final FileSystem fs;
+  private final double resultSizeThreshold;
+  private final int sleepBetweenCycles;
+  private final long maxRoundsStuck;
+
+  public RegionsMerger(Configuration conf) throws IOException {
+this.conf = conf;
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+fs = basePath.getFileSystem(conf);
+resultSizeThreshold = this.conf.getDouble(RESULTING_REGION_UPPER_MARK, 
0.9) *
+  this.conf.getLong(HConstants.HREGION_MAX_FILESIZE, 
HConstants.DEFAULT_MAX_FILE_SIZE);
+sleepBetweenCycles = this.conf.getInt(SLEEP, 2000);
+this.maxRoundsStuck = this.conf.getInt(MAX_ROUNDS_IDDLE, 10);
+  }
+
+  private Path getTablePath(TableName table){
+Path basePath = new Path(conf.get(HConstants.HBASE_DIR));
+basePath = new Path(basePath, "data");
+Path tablePath = new Path(basePath, table.getNamespaceAsString());
+return new Path(tablePath, table.getNameAsString());
+  }
+
+  private long sumSizeInFS(Path parentPath) throws IOException {
+long size = 0;
+FileStatus[] files = this.fs.listStatus(parentPath);
+for(FileStatus f : files) {
+  if(f.isFile()) {
+size += f.getLen();
+  } else if(f.isDirectory()) {
+size += sumSizeInFS(f.getPath());
+  }
+}
+return size;
+  }
+
+  private List getOpenRegions(Connection connection, TableName 
table) throws Exception {
+List regions = new ArrayList<>();
+Table metaTbl = connection.getTable(TableName.valueOf("hbase:meta"));
+S

[GitHub] [hbase] Apache-HBase commented on issue #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1258: HBASE-23932 Minor improvements to Region 
Normalizer
URL: https://github.com/apache/hbase/pull/1258#issuecomment-596805852
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |  11m 38s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 14s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 20s |  master passed  |
   | +0 :ok: |  spotbugs  |   5m  0s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m 31s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 42s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 18s |  hbase-server: The patch 
generated 0 new + 100 unchanged - 1 fixed = 100 total (was 101)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  22m 28s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   4m 39s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 22s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  66m 45s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1258 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 1f19d86eb3f3 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 84 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated 
setStartRow from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596804850
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 21s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 42s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 48s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   4m 27s |  master passed  |
   | +0 :ok: |  spotbugs  |   1m 28s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m  2s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   6m 11s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   4m  9s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  16m 16s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   2m 54s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   1m 37s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  68m 37s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1220 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux dcebfec45bc1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 94 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-mapreduce hbase-thrift 
hbase-endpoint hbase-backup hbase-it hbase-rest U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/4/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257#issuecomment-596798180
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   3m 16s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 46s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  1s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 22s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  1s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  1s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 55s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  52m 46s |  hbase-server in the patch passed.  
|
   |  |   |  82m 21s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1257 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 4343e3bb4659 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/testReport/
 |
   | Max. process+thread count | 6667 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257#issuecomment-596787147
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   6m 57s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m  0s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 14s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 42s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m 18s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 40s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 13s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 30s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   3m  6s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 12s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  52m 13s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1257 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 820cd50773df 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 83 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1252: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1252: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1252#issuecomment-596779017
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 35s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 25s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  9s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 29s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 20s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   3m 38s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 44s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 26s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 26s |  the patch passed  |
   | -1 :x: |  shadedjars  |   0m 10s |  patch has 7 errors when building our 
shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   3m 37s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 188m 10s |  root in the patch passed.  |
   |  |   | 224m 57s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1252 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 19cd5b5a68fe 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   | shadedjars | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/testReport/
 |
   | Max. process+thread count | 6513 (vs. ulimit of 1) |
   | modules | C: hbase-server . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on issue #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

ndimiduk commented on issue #1258: HBASE-23932 Minor improvements to Region 
Normalizer
URL: https://github.com/apache/hbase/pull/1258#issuecomment-596777804
 
 
   All tests in the normalizer package pass. Let's see what the build-bot says.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk opened a new pull request #1258: HBASE-23932 Minor improvements to Region Normalizer

2020-03-09 Thread GitBox

ndimiduk opened a new pull request #1258: HBASE-23932 Minor improvements to 
Region Normalizer
URL: https://github.com/apache/hbase/pull/1258
 
 
- consolidate checks made by master on behalf of balancer and
  normalizer: deciding if the master is in a healthy state for
  running any actions at all (skipRegionManagementAction). Normalizer
  now does as balancer did previously.
- both balancer and normalizer make one final check on above
  conditions between calculating an action plan and executing the
  plan. should make the process more responsive to shutdown
  requests.
- change normalizer to only consider acting on a region when it is in
  the OPEN state. previously we would normalizer attempt to merge a
  region that was already in a MERGING_NEW,MERGING,MERGED state.
- fix some typos in variable names.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23932) Minor improvements to Region Normalizer

2020-03-09 Thread Nick Dimiduk (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-23932:
-
Summary: Minor improvements to Region Normalizer  (was: Region Normalizer 
is disruptive while running hbck2 fix meta)

> Minor improvements to Region Normalizer
> ---
>
> Key: HBASE-23932
> URL: https://issues.apache.org/jira/browse/HBASE-23932
> Project: HBase
>  Issue Type: Improvement
>  Components: hbck2, master
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
>
> While hbck2 runs through fixes to meta, the Normalizer kicks in to "help". 
> Mostly this is fine, however, things get confused when {{fixMeta}} is action 
> on a region and then the normalizer decides to do something as well. Could be 
> we advise operators disable normalizer while running hbck2, but would be 
> better if the normalizer was less aggressive, so that there's fewer flags to 
> fiddle.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase-thirdparty] jojochuang commented on a change in pull request #13: HBASE-23811. [OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty.

2020-03-09 Thread GitBox

jojochuang commented on a change in pull request #13: HBASE-23811. 
[OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty.
URL: https://github.com/apache/hbase-thirdparty/pull/13#discussion_r389957220
 
 

 ##
 File path: hbase-shaded-jaeger/pom.xml
 ##
 @@ -0,0 +1,176 @@
+
+https://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="https://maven.apache.org/POM/4.0.0 
https://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+4.0.0
+
+hbase-thirdparty
+org.apache.hbase.thirdparty
+3.3.0-SNAPSHOT
+..
+
+hbase-shaded-jaeger
+Apache HBase Relocated (Shaded) JaegerTracing Libs
+The shaded uber jar of JaegerTracing library used by 
HBase.
+
+
+
+io.jaegertracing
+jaeger-client
+${jaegertracing.version}
+
+
+
+
+
+
+
+
+maven-assembly-plugin
+
+true
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+
aggregate-into-a-jar-with-relocated-third-parties
+package
+
+shade
+
+
+false
+
false
+false
+
+
+
+
org.apache.httpcomponents:httpclient  
+
commons-logging:commons-logging   
+
commons-codec:commons-codec   
+
org.apache.httpcomponents:httpcore
+
javax.annotation:javax.annotation-api 
+org.slf4j:slf4j-api 
  
+io.opentracing:*
  
+
+
+
+
+
 
 Review comment:
   Thanks @busbey for the suggest. Found a trick to:
   1.  exclude gson from the shaded jar, but
   2. update the reference to gson classes to the gson in the hbase-shaded-misc 
jar.
   
   Verified the gson classes are not in the hbase-shaded-jaeger jar and that 
the byte code references  org.apache.hbase.thirdparty.com.google.gson.*;


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated 
setStartRow from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596771908
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   3m 40s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 36s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 32s |  master passed  |
   | +1 :green_heart: |  compile  |   5m 34s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 45s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 28s |  hbase-client in master failed.  |
   | -0 :warning: |  javadoc  |   0m 19s |  hbase-backup in master failed.  |
   | -0 :warning: |  javadoc  |   0m 22s |  hbase-mapreduce in master failed.  |
   | -0 :warning: |  javadoc  |   0m 25s |  hbase-rest in master failed.  |
   | -0 :warning: |  javadoc  |   0m 44s |  hbase-server in master failed.  |
   | -0 :warning: |  javadoc  |   0m 57s |  hbase-thrift in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   7m 11s |  the patch passed  |
   | +1 :green_heart: |  compile  |   5m 24s |  the patch passed  |
   | +1 :green_heart: |  javac  |   5m 24s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 29s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 38s |  hbase-client in the patch failed.  |
   | -0 :warning: |  javadoc  |   1m  3s |  hbase-server in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 33s |  hbase-mapreduce in the patch 
failed.  |
   | -0 :warning: |  javadoc  |   1m 21s |  hbase-thrift in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 33s |  hbase-backup in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 38s |  hbase-rest in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 52s |  hbase-client in the patch passed.  
|
   | -0 :warning: |  unit  | 125m 10s |  hbase-server in the patch failed.  |
   | -0 :warning: |  unit  |  11m 21s |  hbase-mapreduce in the patch failed.  |
   | +1 :green_heart: |  unit  |   3m 11s |  hbase-thrift in the patch passed.  
|
   | +1 :green_heart: |  unit  |   3m 15s |  hbase-endpoint in the patch 
passed.  |
   | -0 :warning: |  unit  |   4m 39s |  hbase-backup in the patch failed.  |
   | +1 :green_heart: |  unit  |   1m  5s |  hbase-it in the patch passed.  |
   | +1 :green_heart: |  unit  |   3m 16s |  hbase-rest in the patch passed.  |
   |  |   | 208m 28s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1220 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 4733368cf076 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-backup.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-mapreduce.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-s

[GitHub] [hbase] Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated 
setStartRow from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596769454
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   3m 40s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 36s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  0s |  master passed  |
   | +1 :green_heart: |  compile  |   4m 36s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 23s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   2m 50s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 50s |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m 20s |  the patch passed  |
   | +1 :green_heart: |  javac  |   4m 20s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 26s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   3m  0s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 59s |  hbase-client in the patch passed.  
|
   | -1 :x: |  unit  | 116m 24s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  unit  |  18m 48s |  hbase-mapreduce in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   2m 38s |  hbase-thrift in the patch passed.  
|
   | +1 :green_heart: |  unit  |   4m  3s |  hbase-endpoint in the patch 
passed.  |
   | +1 :green_heart: |  unit  |  10m 44s |  hbase-backup in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m  2s |  hbase-it in the patch passed.  |
   | +1 :green_heart: |  unit  |   3m 23s |  hbase-rest in the patch passed.  |
   |  |   | 203m  2s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1220 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 487b9eb67d74 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/testReport/
 |
   | Max. process+thread count | 6474 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-mapreduce hbase-thrift 
hbase-endpoint hbase-backup hbase-it hbase-rest U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1252: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1252: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1252#issuecomment-596769203
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 35s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 43s |  master passed  |
   | +1 :green_heart: |  compile  |   4m 28s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 52s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 45s |  hbase-server in master failed.  |
   | -0 :warning: |  javadoc  |   0m 20s |  root in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   7m  5s |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m 39s |  the patch passed  |
   | +1 :green_heart: |  javac  |   4m 39s |  the patch passed  |
   | -1 :x: |  shadedjars  |   0m 13s |  patch has 7 errors when building our 
shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 44s |  hbase-server in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 20s |  root in the patch failed.  |
   ||| _ Other Tests _ |
   | -0 :warning: |  unit  | 164m 39s |  root in the patch failed.  |
   |  |   | 201m 53s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1252 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux f79cc08e8dbb 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
 |
   | shadedjars | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/testReport/
 |
   | Max. process+thread count | 5608 (vs. ulimit of 1) |
   | modules | C: hbase-server . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] pustota2009 opened a new pull request #1257: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

pustota2009 opened a new pull request #1257: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1257
 
 
   When data much more than BlockCache we can save CPU cycles and increase 
performance to 3 times.
   PS Sorry, had some problems with build previuos PR, trying again.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] pustota2009 closed pull request #1252: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

pustota2009 closed pull request #1252: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1252
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of 
hbase-it /test for direct API use of chaos.
URL: https://github.com/apache/hbase/pull/1201#issuecomment-596727535
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 30s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  2s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 44s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 16s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 45s |  master passed  |
   | +0 :ok: |  spotbugs  |   0m 44s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   0m 42s |  master passed  |
   | -0 :warning: |  patch  |   0m 58s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m  4s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 20s |  hbase-it: The patch generated 3 
new + 128 unchanged - 1 fixed = 131 total (was 129)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  1s |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  hadoopcheck  |  16m  6s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  findbugs  |   0m 48s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 25s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  38m 40s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1201 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle xml |
   | uname | Linux 66aadad86dcb 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | checkstyle | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-it.txt
 |
   | Max. process+thread count | 93 (vs. ulimit of 1) |
   | modules | C: hbase-common hbase-it U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of 
hbase-it /test for direct API use of chaos.
URL: https://github.com/apache/hbase/pull/1201#issuecomment-596726555
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 17s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 34s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 19s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 46s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 20s |  hbase-common in master failed.  |
   | -0 :warning: |  patch  |   6m 46s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   7m 55s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  8s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  8s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 47s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 17s |  hbase-common in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 21s |  hbase-it in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 26s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  |   0m 55s |  hbase-it in the patch passed.  |
   |  |   |  36m 18s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1201 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 96f4b4c83f00 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 2020-01-14 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt
 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-it.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/testReport/
 |
   | Max. process+thread count | 417 (vs. ulimit of 1) |
   | modules | C: hbase-common hbase-it U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of 
hbase-it /test for direct API use of chaos.
URL: https://github.com/apache/hbase/pull/1201#issuecomment-596724432
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 40s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 44s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m 11s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 46s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  master passed  |
   | -0 :warning: |  patch  |   6m 43s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 50s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 53s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 53s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 41s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 19s |  hbase-it generated 3 new + 0 
unchanged - 0 fixed = 3 total (was 0)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 12s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  |   0m 56s |  hbase-it in the patch passed.  |
   |  |   |  32m 10s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1201 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 7e6a50159281 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | Default Java | 1.8.0_232 |
   | javadoc | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-hbase-it.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/testReport/
 |
   | Max. process+thread count | 610 (vs. ulimit of 1) |
   | modules | C: hbase-common hbase-it U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/4/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] HorizonNet commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

HorizonNet commented on issue #1220: HBASE-23847 Removed deprecated setStartRow 
from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596716033
 
 
   Triggered a new build as the build failure is not related to the actual 
changes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated setStartRow from Scan

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1220: HBASE-23847 Removed deprecated 
setStartRow from Scan
URL: https://github.com/apache/hbase/pull/1220#issuecomment-596710060
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   5m 33s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 34s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  3s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   4m 51s |  master passed  |
   | +0 :ok: |  spotbugs  |   1m 31s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m 17s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 56s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   4m  4s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  18m  0s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   3m 24s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   1m 25s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  73m 59s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1220 |
   | Optional Tests | dupname asflicense spotbugs findbugs hadoopcheck 
hbaseanti checkstyle |
   | uname | Linux 0f7d103a5731 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | Max. process+thread count | 83 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-server hbase-mapreduce hbase-thrift 
hbase-endpoint hbase-backup hbase-it hbase-rest U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1220/3/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1252: [HBASE 23887] Up to 3x increase BlockCache performance

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1252: [HBASE 23887] Up to 3x increase 
BlockCache performance
URL: https://github.com/apache/hbase/pull/1252#issuecomment-596703989
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  6s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +0 :ok: |  yamllint  |   0m  0s |  yamllint was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 41s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   2m 41s |  master passed  |
   | +0 :ok: |  spotbugs  |  15m 29s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   3m  6s |  hbase-server in master failed.  |
   | -1 :x: |  findbugs  |   9m 32s |  root in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 33s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   2m 40s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  0s |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 39s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | -1 :x: |  findbugs  |   3m 11s |  hbase-server in the patch failed.  |
   | -1 :x: |  findbugs  |   9m 34s |  root in the patch failed.  |
   ||| _ Other Tests _ |
   | -1 :x: |  asflicense  |   0m 25s |  The patch generated 1 ASF License 
warnings.  |
   |  |   |  68m 50s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1252 |
   | Optional Tests | dupname asflicense yamllint spotbugs findbugs hadoopcheck 
hbaseanti checkstyle xml |
   | uname | Linux e61d73dba2b5 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 31484f007f |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-general-check/output/branch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-general-check/output/branch-findbugs-root.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-general-check/output/patch-findbugs-hbase-server.txt
 |
   | findbugs | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-general-check/output/patch-findbugs-root.txt
 |
   | asflicense | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/artifact/yetus-general-check/output/patch-asflicense-problems.txt
 |
   | Max. process+thread count | 83 (vs. ulimit of 1) |
   | modules | C: hbase-server . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1252/9/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23562) [operator tools] Add a RegionsMerge tool that allows for merging multiple adjacent regions until a desired number of regions is reached.

2020-03-09 Thread Josh Elser (Jira)



[ 
https://issues.apache.org/jira/browse/HBASE-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055290#comment-17055290
 ] 

Josh Elser commented on HBASE-23562:


{quote}Any further ideas/suggestions in relation to the current PR?
{quote}
Nothing that has to go int this PR, I think. Maybe a follow-on to make sure the 
Book draws the proper distinction between the two tools :)

> [operator tools] Add a RegionsMerge tool that allows for merging multiple 
> adjacent regions until a desired number of regions is reached.
> 
>
> Key: HBASE-23562
> URL: https://issues.apache.org/jira/browse/HBASE-23562
> Project: HBase
>  Issue Type: New Feature
>  Components: hbase-operator-tools
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
>
> There's been a few occasions where different customers had faced the need to 
> reduce the number of regions for specific tables, either due to mistakenly 
> pre-split or after a purge in table data. This jira is for adding a simple 
> merge tool that takes the table name and the desired number of regions, then 
> perform region merges until the total number of regions for the given table 
> reached the passed valued. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] saintstack commented on a change in pull request #1250: HBASE-23945 Dockerfiles showing hadolint check failures

2020-03-09 Thread GitBox

saintstack commented on a change in pull request #1250: HBASE-23945 Dockerfiles 
showing hadolint check failures
URL: https://github.com/apache/hbase/pull/1250#discussion_r389873407
 
 

 ##
 File path: dev-support/Dockerfile
 ##
 @@ -22,8 +22,14 @@
 # dev-support/flaky-tests/flaky-reporting.Jenkinsfile
 FROM ubuntu:18.04
 
-ADD . /hbase/dev-support
+COPY . /hbase/dev-support
 
-RUN apt-get -y update \
-&& apt-get -y install curl python-pip \
-&& pip install -r /hbase/dev-support/python-requirements.txt
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq -y update \
+&& DEBIAN_FRONTEND=noninteractive apt-get -qq -y install 
--no-install-recommends \
+  curl=7.58.0-2ubuntu3.8 \
+  python2.7=2.7.17-1~18.04 \
+  python-pip=9.0.1-2.3~ubuntu1.18.04.1 \
+  python-setuptools=39.0.1-2 \
 
 Review comment:
   nod


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] saintstack commented on a change in pull request #1250: HBASE-23945 Dockerfiles showing hadolint check failures

2020-03-09 Thread GitBox

saintstack commented on a change in pull request #1250: HBASE-23945 Dockerfiles 
showing hadolint check failures
URL: https://github.com/apache/hbase/pull/1250#discussion_r389874834
 
 

 ##
 File path: dev-support/hbase_docker/Dockerfile
 ##
 @@ -14,37 +14,73 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM ubuntu:14.04
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 
-# Install Git, which is missing from the Ubuntu base images.
-RUN apt-get update && apt-get install -y git
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+  DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends 
-y \
+ca-certificates=20180409 \
+curl=7.58.0-2ubuntu3.8 \
+locales=2.27-3ubuntu1
 
-# Add the dependencies from the hbase_docker folder and delete ones we don't 
need.
-WORKDIR /root
-ADD . /root
-RUN find . -not -name "*tar.gz" -delete
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+FROM BASE_IMAGE AS MAVEN_DOWNLOAD_IMAGE
+ENV MAVEN_VERSION='3.5.4'
+ENV MAVEN_URL 
"https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz";
+ENV MAVEN_SHA256 
'ce50b1c91364cb77efe3776f756a6d92b76d9038b0a0782f7d53acf1e997a14d'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl --location --fail --silent --show-error --output /tmp/maven.tar.gz 
"${MAVEN_URL}" && \
+  echo "${MAVEN_SHA256} */tmp/maven.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK8_DOWNLOAD_IMAGE
+ENV OPENJDK8_URL 
'https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u232-b09/OpenJDK8U-jdk_x64_linux_hotspot_8u232b09.tar.gz'
+ENV OPENJDK8_SHA256 
'7b7884f2eb2ba2d47f4c0bf3bb1a2a95b73a3a7734bd47ebf9798483a7bcc423'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl --location --fail --silent --show-error --output 
/tmp/adoptopenjdk8.tar.gz "${OPENJDK8_URL}" && \
+  echo "${OPENJDK8_SHA256} */tmp/adoptopenjdk8.tar.gz" | sha256sum -c -
 
-# Install Java.
-RUN mkdir -p /usr/java
-RUN tar xzf *jdk* --strip-components 1 -C /usr/java
-ENV JAVA_HOME /usr/java
+FROM BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 
-# Install Maven.
-RUN mkdir -p /usr/local/apache-maven
-RUN tar xzf *maven* --strip-components 1 -C /usr/local/apache-maven
-ENV MAVEN_HOME /usr/local/apache-maven
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends 
-y \
+  git=1:2.17.1-1ubuntu0.5 \
+  && \
+  apt-get clean && \
+  rm -rf /var/lib/apt/lists/*
 
-# Add Java and Maven to the path.
-ENV PATH 
/usr/java/bin:/usr/local/apache-maven/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+# hadolint ignore=DL3010
+COPY --from=MAVEN_DOWNLOAD_IMAGE /tmp/maven.tar.gz /tmp/maven.tar.gz
 
 Review comment:
   Add this in as a comment into the DOckerfile?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1256: HBASE-23631 : Backport HBASE-23350 (Make compaction files cacheonWrit…

2020-03-09 Thread GitBox

Apache-HBase commented on issue #1256: HBASE-23631 : Backport HBASE-23350 (Make 
compaction files cacheonWrit…
URL: https://github.com/apache/hbase/pull/1256#issuecomment-596688985
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   7m  4s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
6 new or modified test files.  |
   ||| _ branch-1 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   8m 43s |  branch-1 passed  |
   | +1 :green_heart: |  compile  |   0m 40s |  branch-1 passed with JDK 
v1.8.0_242  |
   | +1 :green_heart: |  compile  |   0m 45s |  branch-1 passed with JDK 
v1.7.0_252  |
   | +1 :green_heart: |  checkstyle  |   1m 48s |  branch-1 passed  |
   | +1 :green_heart: |  shadedjars  |   2m 55s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  branch-1 passed with JDK 
v1.8.0_242  |
   | +1 :green_heart: |  javadoc  |   0m 40s |  branch-1 passed with JDK 
v1.7.0_252  |
   | +0 :ok: |  spotbugs  |   2m 51s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   2m 49s |  branch-1 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 54s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 41s |  the patch passed with JDK 
v1.8.0_242  |
   | +1 :green_heart: |  javac  |   0m 41s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 45s |  the patch passed with JDK 
v1.7.0_252  |
   | +1 :green_heart: |  javac  |   0m 45s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 39s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   2m 48s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |   4m 50s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2.  |
   | +1 :green_heart: |  javadoc  |   0m 31s |  the patch passed with JDK 
v1.8.0_242  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  the patch passed with JDK 
v1.7.0_252  |
   | +1 :green_heart: |  findbugs  |   2m 50s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 127m 22s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 35s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 174m 19s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.7 Server=19.03.7 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1256/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1256 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 27431eb4665b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1256/out/precommit/personality/provided.sh
 |
   | git revision | branch-1 / 1931714 |
   | Default Java | 1.7.0_252 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 
/usr/lib/jvm/zulu-7-amd64:1.7.0_252 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1256/1/testReport/
 |
   | Max. process+thread count | 4215 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1256/1/console |
   | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Work started] (HBASE-23932) Region Normalizer is disruptive while running hbck2 fix meta

2020-03-09 Thread Nick Dimiduk (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-23932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-23932 started by Nick Dimiduk.

> Region Normalizer is disruptive while running hbck2 fix meta
> 
>
> Key: HBASE-23932
> URL: https://issues.apache.org/jira/browse/HBASE-23932
> Project: HBase
>  Issue Type: Improvement
>  Components: hbck2, master
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
>
> While hbck2 runs through fixes to meta, the Normalizer kicks in to "help". 
> Mostly this is fine, however, things get confused when {{fixMeta}} is action 
> on a region and then the normalizer decides to do something as well. Could be 
> we advise operators disable normalizer while running hbck2, but would be 
> better if the normalizer was less aggressive, so that there's fewer flags to 
> fiddle.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-18418) Remove apache_hbase_topology from dev-support

2020-03-09 Thread Nick Dimiduk (Jira)



 [ 
https://issues.apache.org/jira/browse/HBASE-18418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-18418.
--
Resolution: Fixed

> Remove apache_hbase_topology from dev-support
> -
>
> Key: HBASE-18418
> URL: https://issues.apache.org/jira/browse/HBASE-18418
> Project: HBase
>  Issue Type: Task
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-18418_v1.patch
>
>
> As [~misty] noted on dev@, we still keep a folder containing the Apache HBase 
> topology for [clusterdock|https://github.com/clusterdock/framework] in 
> dev-support. Since the {{clusterdock}} org on GitHub will likely always have 
> a more up-to-date iteration of this code (and since I plan on showing it some 
> love there in the next few weeks), we'd probably be better off removing it 
> from HBase. Any objections?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

1 2 >

1 - 100 of 150 matches

Mail list logo