[GitHub] [hbase] Apache-HBase commented on pull request #4718: HBASE-27314 Make index block be customized and configured

[GitBox]

Apache-HBase commented on PR #4718:
URL: https://github.com/apache/hbase/pull/4718#issuecomment-1226825562

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 38s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 23s |  master passed  |
   | +1 :green_heart: |  compile  |   1m  5s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m  2s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 12s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  4s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  4s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m  2s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 28s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m  8s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  | 208m  1s |  hbase-server in the patch passed.  
|
   |  |   | 230m  3s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4718 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux b0cb37b67487 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/testReport/
 |
   | Max. process+thread count | 2820 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-client hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4718: HBASE-27314 Make index block be customized and configured

[GitBox]

Apache-HBase commented on PR #4718:
URL: https://github.com/apache/hbase/pull/4718#issuecomment-1226816683

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 37s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 21s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 50s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 49s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 12s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 49s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 49s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 48s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m 19s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  | 194m 17s |  hbase-server in the patch passed.  
|
   |  |   | 217m  6s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4718 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 9d6d4d606936 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 
28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/testReport/
 |
   | Max. process+thread count | 2633 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-client hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-27320) hide some sensitive configuration information in the UI

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584630#comment-17584630
 ] 

Hudson commented on HBASE-27320:


Results for branch branch-2
[build #626 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-20904) Prometheus /metrics http endpoint for monitoring integration

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-20904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584629#comment-17584629
 ] 

Hudson commented on HBASE-20904:


Results for branch branch-2
[build #626 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Prometheus /metrics http endpoint for monitoring integration
> 
>
> Key: HBASE-20904
> URL: https://issues.apache.org/jira/browse/HBASE-20904
> Project: HBase
>  Issue Type: New Feature
>  Components: metrics, monitoring
>Reporter: Hari Sekhon
>Assignee: Luca Kovacs
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> Feature Request to add Prometheus /metrics http endpoint for monitoring 
> integration:
> [https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cscrape_config%3E]
> Prometheus metrics format for that endpoint:
> [https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584631#comment-17584631
 ] 

Hudson commented on HBASE-27246:


Results for branch branch-2
[build #626 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/626/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] rda3mon commented on pull request #4698: HBASE-27238 backport backup restore to 2.x

[GitBox]

rda3mon commented on PR #4698:
URL: https://github.com/apache/hbase/pull/4698#issuecomment-1226772078

   @bbeaudreault Please allow me time till this weekend. Tied up with few work
   related stuff.
   
   ---
   Mallikarjun
   
   
   On Wed, Aug 24, 2022 at 11:33 PM Bryan Beaudreault ***@***.***>
   wrote:
   
   > @rda3mon  is this ready for review?
   >
   > —
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584601#comment-17584601
 ] 

Hudson commented on HBASE-27246:


Results for branch branch-2.4
[build #416 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/]:
 (/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27320) hide some sensitive configuration information in the UI

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584600#comment-17584600
 ] 

Hudson commented on HBASE-27320:


Results for branch branch-2.4
[build #416 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/]:
 (/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/416/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (HBASE-27323) Support to take the initiative to compact cold large files and compression diff hfiles after changing storefile compression algrithm

[Xiaolin Ha (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584572#comment-17584572
 ] 

Xiaolin Ha edited comment on HBASE-27323 at 8/25/22 3:20 AM:
-

Yes, they are definitely different things.

The attribute "COMPRESSION_COMPACT, COMPRESSION_COMPACT_MAJOR, 
COMPRESSION_COMPACT_MINOR" aims to support different compression algorithms for 
one column family from the COMPRESSION type. I think it can use different 
compressions for recently flushed and colder data, since in some circumstances 
hot data are more recently and can use faster algorithms, while colder can use 
higher ratio algorithms. 

In this issue, it supports to quickly apply the changed compression algorithm 
to ALL the store files through the regular MINOR compactions, though there are 
some cold large files that rare participate in minor compactions. When we only 
change the "COMPRESSION" attribute, all the different compression algorithm 
store files whatever is large or small, should be rewrite to the new 
compression algorithm.


was (Author: xiaolin ha):
Yes, they are definitely different things.

The attribute "COMPRESSION_COMPACT, COMPRESSION_COMPACT_MAJOR, 
COMPRESSION_COMPACT_MINOR" aims to support different compression algorithms for 
one column family from the COMPRESSION type. I think it can use different 
compressions for recently flushed and colder data, since in some circumstances 
hot data are more recently and can used faster algorithms, while colder can use 
higher ratio algorithms. 

In this issue, it can support to quickly apply the changed compression 
algorithm to ALL the store files through the regular MINOR compactions, though 
there are some cold large files that rare participate in minor compactions. 
When we only change the "COMPRESSION" attribute, all the different compression 
algorithm store files whatever is large or small, should be rewrite to the new 
compression algorithm.

> Support to take the initiative to compact cold large files and compression 
> diff hfiles after changing storefile compression algrithm 
> -
>
> Key: HBASE-27323
> URL: https://issues.apache.org/jira/browse/HBASE-27323
> Project: HBase
>  Issue Type: Improvement
>  Components: Compaction
>Affects Versions: 3.0.0-alpha-3, 2.4.13
>Reporter: Xiaolin Ha
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> We can set switch to enable this feature, to make the compression algrithm 
> changes be applied to all the existing store files, especially when we make a 
> lower space usage compression algrithm change, e.g. from LZO to ZSTD can save 
> more than 30% spaces.
> All the compaction policies should support this feature.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27323) Support to take the initiative to compact cold large files and compression diff hfiles after changing storefile compression algrithm

[Xiaolin Ha (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584572#comment-17584572
 ] 

Xiaolin Ha commented on HBASE-27323:


Yes, they are definitely different things.

The attribute "COMPRESSION_COMPACT, COMPRESSION_COMPACT_MAJOR, 
COMPRESSION_COMPACT_MINOR" aims to support different compression algorithms for 
one column family from the COMPRESSION type. I think it can use different 
compressions for recently flushed and colder data, since in some circumstances 
hot data are more recently and can used faster algorithms, while colder can use 
higher ratio algorithms. 

In this issue, it can support to quickly apply the changed compression 
algorithm to ALL the store files through the regular MINOR compactions, though 
there are some cold large files that rare participate in minor compactions. 
When we only change the "COMPRESSION" attribute, all the different compression 
algorithm store files whatever is large or small, should be rewrite to the new 
compression algorithm.

> Support to take the initiative to compact cold large files and compression 
> diff hfiles after changing storefile compression algrithm 
> -
>
> Key: HBASE-27323
> URL: https://issues.apache.org/jira/browse/HBASE-27323
> Project: HBase
>  Issue Type: Improvement
>  Components: Compaction
>Affects Versions: 3.0.0-alpha-3, 2.4.13
>Reporter: Xiaolin Ha
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> We can set switch to enable this feature, to make the compression algrithm 
> changes be applied to all the existing store files, especially when we make a 
> lower space usage compression algrithm change, e.g. from LZO to ZSTD can save 
> more than 30% spaces.
> All the compaction policies should support this feature.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4718: HBASE-27314 Make index block be customized and configured

[GitBox]

Apache-HBase commented on PR #4718:
URL: https://github.com/apache/hbase/pull/4718#issuecomment-1226715116

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 35s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 44s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 10s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 18s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 51s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 36s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   2m 13s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 10s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m  7s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 20s |  the patch passed  |
   | -0 :warning: |  javac  |   0m 28s |  hbase-common generated 2 new + 0 
unchanged - 0 fixed = 2 total (was 0)  |
   | -0 :warning: |  checkstyle  |   0m 27s |  hbase-server: The patch 
generated 6 new + 18 unchanged - 0 fixed = 24 total (was 18)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   8m  1s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  spotless  |   0m 35s |  patch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   2m 30s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 21s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  34m  7s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4718 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux 88f660e1365c 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | javac | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-common.txt
 |
   | checkstyle | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-client hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4718/3/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (HBASE-27325) the bulkload max call queue size can be update to a wrong value

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-27325.
---
Fix Version/s: 2.6.0
 Hadoop Flags: Reviewed
   Resolution: Fixed

Pushed to master and branch-2.

Thanks [~frostruan] for contributing!

> the bulkload max call queue size can be update to a wrong value
> ---
>
> Key: HBASE-27325
> URL: https://issues.apache.org/jira/browse/HBASE-27325
> Project: HBase
>  Issue Type: Bug
>  Components: IPC/RPC
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> The configKey can be wrong, because 
> name.toLowerCase(Locale.ROOT).contains("bulkLoad") is always false.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache9 merged pull request #4728: HBASE-27325 the bulkload max call queue size can be update to a wrong value

[GitBox]

Apache9 merged PR #4728:
URL: https://github.com/apache/hbase/pull/4728


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (HBASE-27327) Class missing at runtime

[fengxianjing (Jira)]

fengxianjing created HBASE-27327:


 Summary: Class missing at runtime
 Key: HBASE-27327
 URL: https://issues.apache.org/jira/browse/HBASE-27327
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 2.3.4
Reporter: fengxianjing


We found that some class cannot be found after the regionserver has been 
running for a period of time(more than a month). And more than half of the 
machines in our cluster have this problem. Some are still running normally, but 
_/rs-status_ cannot be opened，and others have various problems(such as rit, 
replication failed, abort failed)

Some exceptions are as follows:
{code:java}
java.lang.NoClassDefFoundError: 
org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl
at 
org.apache.hadoop.hbase.regionserver.RSStatusServlet.doGet(RSStatusServlet.java:49)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at 
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
at 
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at 
org.apache.hadoop.hbase.http.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:66)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at 
org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:52)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at 
org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1495)
{code}
{code:java}
2022-08-24 19:22:52,536 ERROR [RS_CLOSE_REGION-regionserver/:26020-1] 
regionserver.HRegionServer: * ABORTING region server 
10.x.x.x,26020,1659357427101: Replay of WAL required. Forcing server shutdown 
*
org.apache.hadoop.hbase.DroppedSnapshotException: region: 
:,xx,1661298619920.943104cbcf4a74db9896fd4abd051411.
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2906)
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2575)
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2547)
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2538)
at 
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1652)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1591)
at 
org.apache.hadoop.hbase.regionserver.handler.UnassignRegionHandler.process(UnassignRegionHandler.java:114)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: 
org/apache/hadoop/hbase/regionserver/querymatcher/DeleteTracker$DeleteResult
at 
org.apache.hadoop.hbase.regionserver.querymatcher.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:108)
at 
org.apache.hadoop.hbase.regionserver.querymatcher.ScanQueryMatcher.checkDeleted(ScanQueryMatcher.java:209)
at 
org.apache.hadoop.hbase.regionserver.querymatcher.MinorCompactionScanQueryMatcher.match(MinorCompactionScanQueryMatcher.java:54)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:627)
at 
org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:127)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:69)
at 
org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1067)
at 
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2442)
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2842)
... 10 more
Caused by: java.lang.ClassNotFoundException: 
org.apache.hadoop.hbase.regionserver.querymatcher.DeleteTracker$DeleteResult
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 19 more

{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584526#comment-17584526
 ] 

Hudson commented on HBASE-27246:


Results for branch master
[build #665 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27320) hide some sensitive configuration information in the UI

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584525#comment-17584525
 ] 

Hudson commented on HBASE-27320:


Results for branch master
[build #665 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-20904) Prometheus /metrics http endpoint for monitoring integration

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-20904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584524#comment-17584524
 ] 

Hudson commented on HBASE-20904:


Results for branch master
[build #665 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Prometheus /metrics http endpoint for monitoring integration
> 
>
> Key: HBASE-20904
> URL: https://issues.apache.org/jira/browse/HBASE-20904
> Project: HBase
>  Issue Type: New Feature
>  Components: metrics, monitoring
>Reporter: Hari Sekhon
>Assignee: Luca Kovacs
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> Feature Request to add Prometheus /metrics http endpoint for monitoring 
> integration:
> [https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cscrape_config%3E]
> Prometheus metrics format for that endpoint:
> [https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27303) Unnecessary replication to secondary region replicas should avoid when WAL.sync throws Exception

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584523#comment-17584523
 ] 

Hudson commented on HBASE-27303:


Results for branch master
[build #665 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/665/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Unnecessary replication to secondary region replicas should avoid when 
> WAL.sync throws Exception
> 
>
> Key: HBASE-27303
> URL: https://issues.apache.org/jira/browse/HBASE-27303
> Project: HBase
>  Issue Type: Improvement
>  Components: read replicas
>Affects Versions: 3.0.0-alpha-4
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> As HBASE-27230 and HBASE-27223 said, if {{WAL.sync}} throws an exception, we 
> should abort the region server to avoid the data inconsistent between the 
> primary region and secondary region replicas, so we have no need to replicate 
> to secondary region replicas when  {{WAL.sync}} throws an exception.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (HBASE-23340) hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldW

[Bryan Beaudreault (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Beaudreault resolved HBASE-23340.
---
Release Note: Previously the LogCleaner chores had their own ZK client. If 
they encounter Session expired error, the LogCleaner chore will never succeed 
again despite the HMaster continuing to run. With this change, the LogCleaner 
chores now share the underlying ZK of the HMaster (similar to HFileCleaner 
chores). So now, if an unrecoverable session expiration occurs, the hmaster 
will abort and cleaner chores will not be left as zombies.
  Resolution: Fixed

> hmaster  /hbase/replication/rs  session expired (hbase replication default 
> value is true, we don't use ) causes logcleaner can not clean oldWALs, which 
> resulits in oldWALs too large (more than 2TB)
> -
>
> Key: HBASE-23340
> URL: https://issues.apache.org/jira/browse/HBASE-23340
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0-alpha-1, 2.2.3
>Reporter: jackylau
>Assignee: Bo Cui
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-1
>
> Attachments: Snipaste_2019-11-21_10-39-25.png, 
> Snipaste_2019-11-21_14-10-36.png
>
>
> hmaster /hbase/replication/rs session expired (hbase replication default 
> value is true, we don't use ) causes logcleaner can not clean oldWALs, which 
> resulits in oldWALs too large (more than 2TB).
> !Snipaste_2019-11-21_10-39-25.png!
>  
> !Snipaste_2019-11-21_14-10-36.png!
>  
> we can solve it by following :
> 1) increase the session timeout(but i think it is not a good idea. because we 
> do not know how long to set is suitable)
> 2) close the hbase replication. It is not a good idea too, when our user uses 
> this feature
> 3) we need add retry times, for example when it has already happened three 
> times, we set the ReplicationLogCleaner and SnapShotCleaner stop
> that is all my ideas, i do not konw it is suitable, If it is suitable, could 
> i commit a PR?
> Does anynode have a good idea.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Reopened] (HBASE-23340) hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldW

[Bryan Beaudreault (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Beaudreault reopened HBASE-23340:
---

Reopening to add a release note. I stumbled across this issue in my environment 
and the fix here is very unclear, burried in one of the closed PRs. I'd like to 
add a release note so that others can have an easier time figuring out what 
happened here.

> hmaster  /hbase/replication/rs  session expired (hbase replication default 
> value is true, we don't use ) causes logcleaner can not clean oldWALs, which 
> resulits in oldWALs too large (more than 2TB)
> -
>
> Key: HBASE-23340
> URL: https://issues.apache.org/jira/browse/HBASE-23340
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Affects Versions: 3.0.0-alpha-1, 2.2.3
>Reporter: jackylau
>Assignee: Bo Cui
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: Snipaste_2019-11-21_10-39-25.png, 
> Snipaste_2019-11-21_14-10-36.png
>
>
> hmaster /hbase/replication/rs session expired (hbase replication default 
> value is true, we don't use ) causes logcleaner can not clean oldWALs, which 
> resulits in oldWALs too large (more than 2TB).
> !Snipaste_2019-11-21_10-39-25.png!
>  
> !Snipaste_2019-11-21_14-10-36.png!
>  
> we can solve it by following :
> 1) increase the session timeout(but i think it is not a good idea. because we 
> do not know how long to set is suitable)
> 2) close the hbase replication. It is not a good idea too, when our user uses 
> this feature
> 3) we need add retry times, for example when it has already happened three 
> times, we set the ReplicationLogCleaner and SnapShotCleaner stop
> that is all my ideas, i do not konw it is suitable, If it is suitable, could 
> i commit a PR?
> Does anynode have a good idea.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

Apache-HBase commented on PR #4724:
URL: https://github.com/apache/hbase/pull/4724#issuecomment-1226334405

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 36s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 40s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m  5s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 49s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m  5s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 13s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 48s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m  2s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 30s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 204m 32s |  hbase-server in the patch passed.  
|
   |  |   | 224m 49s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4724 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux abff7af1c2f9 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/testReport/
 |
   | Max. process+thread count | 2786 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

Apache-HBase commented on PR #4724:
URL: https://github.com/apache/hbase/pull/4724#issuecomment-1226318413

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 52s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 29s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 55s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 25s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 56s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 56s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 56s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 53s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 195m 59s |  hbase-server in the patch passed.  
|
   |  |   | 216m 58s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4724 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux eb478b3f396e 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 
28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/testReport/
 |
   | Max. process+thread count | 2667 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-26810) Add dynamic configuration support for system coprocessors

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-26810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584441#comment-17584441
 ] 

Hudson commented on HBASE-26810:


Results for branch branch-2.5
[build #194 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Add dynamic configuration support for system coprocessors
> -
>
> Key: HBASE-26810
> URL: https://issues.apache.org/jira/browse/HBASE-26810
> Project: HBase
>  Issue Type: Bug
>  Components: conf, Coprocessors
>Affects Versions: 3.0.0-alpha-3
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> [Dynamic Configuraiton|https://hbase.apache.org/book.html#dyn_config] is very 
> helpful that the operator can keep the JVM for regionserver or master running 
> without restarting it. 
> With this feature, this task aims to extend the scope of on demend 
> configuration to system coprocessors including REGION, USER REGION, 
> REGIONSERVER, MASTER such that we could save time on rolling restart with a 
> quicker update. 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584443#comment-17584443
 ] 

Hudson commented on HBASE-27246:


Results for branch branch-2.5
[build #194 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27320) hide some sensitive configuration information in the UI

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584442#comment-17584442
 ] 

Hudson commented on HBASE-27320:


Results for branch branch-2.5
[build #194 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-26882) Backport "HBASE-26810 Add dynamic configuration support for system coprocessors" to branch-2

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584440#comment-17584440
 ] 

Hudson commented on HBASE-26882:


Results for branch branch-2.5
[build #194 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/General_20Nightly_20Build_20Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/194/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Backport "HBASE-26810 Add dynamic configuration support for system 
> coprocessors" to branch-2
> 
>
> Key: HBASE-26882
> URL: https://issues.apache.org/jira/browse/HBASE-26882
> Project: HBase
>  Issue Type: Task
>  Components: Coprocessors, master, regionserver
>Affects Versions: 2.5.0, 2.6.0, 2.4.12
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
> Fix For: 2.5.0, 2.4.12
>
>
> Backport HBASE-26810 to streams of branch-2



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

Apache-HBase commented on PR #4729:
URL: https://github.com/apache/hbase/pull/4729#issuecomment-1226160722

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  2s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 17s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 37s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 15s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 39s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   0m 38s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 13s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 38s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 38s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 14s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   7m 56s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  spotless  |   0m 39s |  patch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   0m 45s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m  9s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  23m 10s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4729 |
   | JIRA Issue | HBASE-26809 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux 29b5335ae6dc 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

Apache-HBase commented on PR #4729:
URL: https://github.com/apache/hbase/pull/4729#issuecomment-1226156678

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  7s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 52s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 20s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 43s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 16s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 49s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 23s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 23s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 18s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 18s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 39s |  hbase-client in the patch passed.  
|
   |  |   |  20m 59s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4729 |
   | JIRA Issue | HBASE-26809 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 2e816b168013 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 
28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/testReport/
 |
   | Max. process+thread count | 161 (vs. ulimit of 3) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4728: HBASE-27325 the bulkload max call queue size can be update to a wrong value

[GitBox]

Apache-HBase commented on PR #4728:
URL: https://github.com/apache/hbase/pull/4728#issuecomment-1226153484

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 22s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 48s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 42s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m  9s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 55s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 51s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 51s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 35s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 236m 53s |  hbase-server in the patch failed.  |
   |  |   | 255m 56s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4728 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux e49693ba63b7 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/testReport/
 |
   | Max. process+thread count | 2245 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

Apache-HBase commented on PR #4729:
URL: https://github.com/apache/hbase/pull/4729#issuecomment-1226152692

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  8s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 34s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 21s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 49s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 18s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 40s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 21s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 21s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 54s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 17s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 17s |  hbase-client in the patch passed.  
|
   |  |   |  17m 56s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4729 |
   | JIRA Issue | HBASE-26809 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux c2a25ae0c225 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/testReport/
 |
   | Max. process+thread count | 186 (vs. ulimit of 3) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/2/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

Apache-HBase commented on PR #4729:
URL: https://github.com/apache/hbase/pull/4729#issuecomment-1226130065

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  3s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 32s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 38s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 16s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 42s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   0m 41s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 12s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 37s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 37s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 14s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   7m 58s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | -1 :x: |  spotless  |   0m 17s |  patch has 63 errors when running 
spotless:check, run spotless:apply to fix.  |
   | +1 :green_heart: |  spotbugs  |   0m 45s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 11s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  23m 21s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4729 |
   | JIRA Issue | HBASE-26809 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux eed1dfe4f983 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | spotless | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/artifact/yetus-general-check/output/patch-spotless.txt
 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

Apache-HBase commented on PR #4729:
URL: https://github.com/apache/hbase/pull/4729#issuecomment-1226130127

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 43s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 23s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 21s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  8s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 19s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m  3s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 20s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 20s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 57s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 20s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 37s |  hbase-client in the patch passed.  
|
   |  |   |  22m 53s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4729 |
   | JIRA Issue | HBASE-26809 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux c8d2466a9282 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 
28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/testReport/
 |
   | Max. process+thread count | 164 (vs. ulimit of 3) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

Apache-HBase commented on PR #4729:
URL: https://github.com/apache/hbase/pull/4729#issuecomment-1226126383

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   2m 56s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 46s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 20s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 53s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 20s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 36s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 20s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 20s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 48s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 17s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 17s |  hbase-client in the patch passed.  
|
   |  |   |  19m 38s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4729 |
   | JIRA Issue | HBASE-26809 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux be2094ddca86 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/testReport/
 |
   | Max. process+thread count | 199 (vs. ulimit of 3) |
   | modules | C: hbase-client U: hbase-client |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4729/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4728: HBASE-27325 the bulkload max call queue size can be update to a wrong value

[GitBox]

Apache-HBase commented on PR #4728:
URL: https://github.com/apache/hbase/pull/4728#issuecomment-1226125844

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 12s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 54s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 53s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 42s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 46s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 46s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 47s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 27s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 210m 30s |  hbase-server in the patch passed.  
|
   |  |   | 229m  9s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4728 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux c01b382a49eb 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/testReport/
 |
   | Max. process+thread count | 2428 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] briaugenreich opened a new pull request, #4729: HBASE-26809: Report client backoff time for server overloaded

[GitBox]

briaugenreich opened a new pull request, #4729:
URL: https://github.com/apache/hbase/pull/4729

   Enable metric reporting on backoff time when a server is overloaded. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Comment Edited] (HBASE-26548) Investigate mTLS in RPC layer

[Bryan Beaudreault (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584411#comment-17584411
 ] 

Bryan Beaudreault edited comment on HBASE-26548 at 8/24/22 6:09 PM:


HBASE-2 has landed in master and branch-2, bringing TLS encryption and 
one-way authentication of servers to the 2.6.0+. As a follow-up, HBASE-27280 
has been filed to implement mTLS (mutual/two-way, where server also 
authenticates client) . A patch has been submitted, so I'm resolving this issue 
which was a placeholder for the investigation piece.


was (Author: bbeaudreault):
HBASE-2 has landed in master and branch-2. As a follow-up, HBASE-27280 has 
been filed to implement mTLS. A patch has been submitted, so I'm resolving this 
issue which was a placeholder for the investigation piece.

> Investigate mTLS in RPC layer
> -
>
> Key: HBASE-26548
> URL: https://issues.apache.org/jira/browse/HBASE-26548
> Project: HBase
>  Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Priority: Major
> Attachments: 0001-One-way-TLS-on-Netty-RPC-Implementation.patch
>
>
> Current authentication options are heavily based on SASL and Kerberos. For 
> organizations that don't already deploy Kerberos or other token provider, 
> this is a heavy lift. Another very common way of authenticating in the 
> industry is mTLS, which makes use of SSL certifications and can solve both 
> wire encryption and auth. For those already deploying trusted certificates in 
> their infra, mTLS may be much easier to integrate.
> It isn't necessarily easy to implement this, but I do think we could use 
> existing Netty SSL support in the NettyRpcClient and NettyRpcServer. I know 
> it's easy to add SSL to non-blocking IO through a 
> hadoop.rpc.socket.factory.class.default which returns SSLSockets, but that 
> doesn't touch on the certification verification at all.
> Much more investigation is needed, but logging this due to some interest 
> encountered on slack.
> Slack thread: 
> https://apache-hbase.slack.com/archives/C13K8NVAM/p1638980520110600



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (HBASE-26548) Investigate mTLS in RPC layer

[Bryan Beaudreault (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Beaudreault resolved HBASE-26548.
---
Resolution: Done

HBASE-2 has landed in master and branch-2. As a follow-up, HBASE-27280 has 
been filed to implement mTLS. A patch has been submitted, so I'm resolving this 
issue which was a placeholder for the investigation piece.

> Investigate mTLS in RPC layer
> -
>
> Key: HBASE-26548
> URL: https://issues.apache.org/jira/browse/HBASE-26548
> Project: HBase
>  Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Priority: Major
> Attachments: 0001-One-way-TLS-on-Netty-RPC-Implementation.patch
>
>
> Current authentication options are heavily based on SASL and Kerberos. For 
> organizations that don't already deploy Kerberos or other token provider, 
> this is a heavy lift. Another very common way of authenticating in the 
> industry is mTLS, which makes use of SSL certifications and can solve both 
> wire encryption and auth. For those already deploying trusted certificates in 
> their infra, mTLS may be much easier to integrate.
> It isn't necessarily easy to implement this, but I do think we could use 
> existing Netty SSL support in the NettyRpcClient and NettyRpcServer. I know 
> it's easy to add SSL to non-blocking IO through a 
> hadoop.rpc.socket.factory.class.default which returns SSLSockets, but that 
> doesn't touch on the certification verification at all.
> Much more investigation is needed, but logging this due to some interest 
> encountered on slack.
> Slack thread: 
> https://apache-hbase.slack.com/archives/C13K8NVAM/p1638980520110600



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27326) Add validation of request user and groups from TLS certificate

[Bryan Beaudreault (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Beaudreault updated HBASE-27326:
--
Fix Version/s: 2.6.0

> Add validation of request user and groups from TLS certificate
> --
>
> Key: HBASE-27326
> URL: https://issues.apache.org/jira/browse/HBASE-27326
> Project: HBase
>  Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0
>
>
> When using mTLS for client authentication, we can allow the user to configure 
> certain certificate fields as a means for validating the passed username on 
> the ConnectionHeader. We can further look to inject groups for the user into 
> the request context, which can be used for downstream authz in (for example) 
> AuthManager/AccessChecker/etc.
> I would propose two new configs:
> {code:java}
> 
>   hbase.rpc.tls.certificate.username.oid
>   
>   When specified and TLS enabled, the client's SSL certificate 
> will be inspected for an OID of this value. A value must be found and the 
> value must match the username passed in the ConnectionHeader. For example, 
> can be set to "CN" and we will use the CommonName of the certificate to 
> validate the username.
> 
> 
>   hbase.rpc.tls.certificate.group.oid
>   
>   When specified and TLS enabled, the client's SSL certificate 
> will be inspected for OIDs of this value. If one or more values are found, 
> they will be used as the user's groups for use in hbase authz.
> {code}
> I think this would only apply when AuthenticationMethod is SIMPLE (no 
> kerberos).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] bbeaudreault commented on pull request #4698: HBASE-27238 backport backup restore to 2.x

[GitBox]

bbeaudreault commented on PR #4698:
URL: https://github.com/apache/hbase/pull/4698#issuecomment-1226059422

   @rda3mon is this ready for review?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

Apache-HBase commented on PR #4724:
URL: https://github.com/apache/hbase/pull/4724#issuecomment-1226050782

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   6m 36s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 27s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 48s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 46s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 41s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 47s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 12s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 46s |  the patch passed  |
   | -0 :warning: |  javac  |   0m 32s |  hbase-common generated 9 new + 0 
unchanged - 0 fixed = 9 total (was 0)  |
   | -0 :warning: |  javac  |   2m 14s |  hbase-server generated 1 new + 192 
unchanged - 1 fixed = 193 total (was 193)  |
   | -0 :warning: |  checkstyle  |   0m 13s |  hbase-common: The patch 
generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)  |
   | -0 :warning: |  checkstyle  |   0m 31s |  hbase-server: The patch 
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   8m  3s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  spotless  |   0m 39s |  patch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 58s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 19s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  37m 36s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4724 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux c8f3da05dee7 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | javac | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-common.txt
 |
   | javac | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt
 |
   | checkstyle | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt
 |
   | checkstyle | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4724/2/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (HBASE-27326) Add validation of request user and groups from TLS certificate

[Bryan Beaudreault (Jira)]

Bryan Beaudreault created HBASE-27326:
-

 Summary: Add validation of request user and groups from TLS 
certificate
 Key: HBASE-27326
 URL: https://issues.apache.org/jira/browse/HBASE-27326
 Project: HBase
  Issue Type: Improvement
Reporter: Bryan Beaudreault
Assignee: Bryan Beaudreault


When using mTLS for client authentication, we can allow the user to configure 
certain certificate fields as a means for validating the passed username on the 
ConnectionHeader. We can further look to inject groups for the user into the 
request context, which can be used for downstream authz in (for example) 
AuthManager/AccessChecker/etc.

I would propose two new configs:
{code:java}

  hbase.rpc.tls.certificate.username.oid
  
  When specified and TLS enabled, the client's SSL certificate 
will be inspected for an OID of this value. A value must be found and the value 
must match the username passed in the ConnectionHeader. For example, can be set 
to "CN" and we will use the CommonName of the certificate to validate the 
username.


  hbase.rpc.tls.certificate.group.oid
  
  When specified and TLS enabled, the client's SSL certificate 
will be inspected for OIDs of this value. If one or more values are found, they 
will be used as the user's groups for use in hbase authz.
{code}
I think this would only apply when AuthenticationMethod is SIMPLE (no kerberos).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] bbeaudreault commented on pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on PR #4724:
URL: https://github.com/apache/hbase/pull/4724#issuecomment-1226030954

   @anmolnar I can't assign you as a reviewer, but would appreciate a look when 
you get a chance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (HBASE-27280) Add mutual authentication support to TLS

[Bryan Beaudreault (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bryan Beaudreault updated HBASE-27280:
--
Labels: patch-available  (was: )
Status: Patch Available  (was: Open)

> Add mutual authentication support to TLS
> 
>
> Key: HBASE-27280
> URL: https://issues.apache.org/jira/browse/HBASE-27280
> Project: HBase
>  Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>  Labels: patch-available
> Fix For: 2.6.0
>
>
> With HBASE-2 we now have native TLS on server and client. By default 
> clients validate server certificate on handshake. This issue adds server 
> authentication of clients. We can also add support for custom rules, such as 
> cert CommonName validation.
> I've already got a POC running of this, so assigning to me



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] bbeaudreault commented on pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on PR #4724:
URL: https://github.com/apache/hbase/pull/4724#issuecomment-1226019986

   I've added comprehensive tests (about 250 test cases covering all 
permutations). I also marked with comments the files which are copied from ZK 
unchanged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] bbeaudreault commented on a diff in pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on code in PR #4724:
URL: https://github.com/apache/hbase/pull/4724#discussion_r954070605


##
hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/tls/TestX509Util.java:
##
@@ -142,6 +142,38 @@ public void cleanUp() {
 Security.setProperty("com.sun.security.enableCRLDP", 
Boolean.FALSE.toString());
   }
 
+  @Test
+  public void testCreateSSLContextWithClientAuthDefault() throws Exception {

Review Comment:
   I add tests for ClientAuth passthrough here, but it's not really possible 
(as far as i can tell) to access the TrustManager of the underlying 
SslContext/SSLEngine. So I can't easily add tests here for the TrustManager 
hostname verification arg passthrough. So instead we have the comprehensive 
end-to-end tests in TestMutualTlsClientSide/ServerSide



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] bbeaudreault commented on a diff in pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on code in PR #4724:
URL: https://github.com/apache/hbase/pull/4724#discussion_r954069133


##
hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/tls/TestHBaseTrustManager.java:
##
@@ -0,0 +1,290 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.crypto.tls;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.math.BigInteger;
+import java.net.InetAddress;
+import java.net.Socket;
+import java.security.KeyPair;
+import java.security.KeyPairGenerator;
+import java.security.Security;
+import java.security.cert.X509Certificate;
+import java.util.ArrayList;
+import java.util.Calendar;
+import java.util.Date;
+import java.util.List;
+import java.util.Random;
+import javax.net.ssl.X509ExtendedTrustManager;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.bouncycastle.asn1.x500.X500NameBuilder;
+import org.bouncycastle.asn1.x500.style.BCStyle;
+import org.bouncycastle.asn1.x509.BasicConstraints;
+import org.bouncycastle.asn1.x509.Extension;
+import org.bouncycastle.asn1.x509.GeneralName;
+import org.bouncycastle.asn1.x509.GeneralNames;
+import org.bouncycastle.asn1.x509.KeyUsage;
+import org.bouncycastle.cert.X509v3CertificateBuilder;
+import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
+import org.bouncycastle.cert.jcajce.JcaX509v3CertificateBuilder;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.operator.ContentSigner;
+import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.mockito.invocation.InvocationOnMock;
+import org.mockito.stubbing.Answer;
+
+//
+/**
+ * Test cases taken and adapted from Apache ZooKeeper Project. We can only 
test calls to
+ * HBaseTrustManager using Sockets (not SSLEngines). This can be fine since 
the logic is the same.
+ * @see https://github.com/apache/zookeeper/blob/c74658d398cdc1d207aa296cb6e20de00faec03e/zookeeper-server/src/test/java/org/apache/zookeeper/common/HBaseTrustManagerTest.java";>Base
+ *  revision
+ */
+@Category({ MiscTests.class, SmallTests.class })
+public class TestHBaseTrustManager {

Review Comment:
   this class was copied verbatim



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] bbeaudreault commented on a diff in pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on code in PR #4724:
URL: https://github.com/apache/hbase/pull/4724#discussion_r954068720


##
hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/tls/TestHBaseHostnameVerifier.java:
##
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.crypto.tls;
+
+import static org.junit.Assert.fail;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+import java.lang.invoke.MethodHandles;
+import java.security.KeyPair;
+import java.security.Security;
+import java.security.cert.CertificateFactory;
+import java.security.cert.X509Certificate;
+import javax.net.ssl.SSLException;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.apache.hadoop.hbase.testclassification.SmallTests;
+import org.bouncycastle.asn1.x500.X500Name;
+import org.bouncycastle.asn1.x500.X500NameBuilder;
+import org.bouncycastle.asn1.x500.style.BCStyle;
+import org.bouncycastle.asn1.x509.GeneralName;
+import org.bouncycastle.asn1.x509.GeneralNames;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+/**
+ * Test cases taken and adapted from Apache ZooKeeper Project
+ * @see https://github.com/apache/zookeeper/blob/5820d10d9dc58c8e12d2e25386fdf92acb360359/zookeeper-server/src/test/java/org/apache/zookeeper/common/ZKHostnameVerifierTest.java";>Base
+ *  revision
+ */
+@Category({ MiscTests.class, SmallTests.class })

Review Comment:
   the test cases in this class were copied from zk, but there they use 
pre-generated certificate strings. Here I decided to use our existing 
X509TestHelpers to generate the certs at test time. I verified that the 
generated certs here match the expectation of the pre-generated ones in zk.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] bbeaudreault commented on a diff in pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on code in PR #4724:
URL: https://github.com/apache/hbase/pull/4724#discussion_r954066160


##
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/tls/HBaseTrustManager.java:
##
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.crypto.tls;
+
+import java.net.InetAddress;
+import java.net.Socket;
+import java.net.UnknownHostException;
+import java.security.cert.CertificateException;
+import java.security.cert.X509Certificate;
+import javax.net.ssl.SSLEngine;
+import javax.net.ssl.SSLException;
+import javax.net.ssl.X509ExtendedTrustManager;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A custom TrustManager that supports hostname verification We attempt to 
perform verification
+ * using just the IP address first and if that fails will attempt to perform a 
reverse DNS lookup
+ * and verify using the hostname. This file has been copied from the Apache 
ZooKeeper project.
+ * @see https://github.com/apache/zookeeper/blob/c74658d398cdc1d207aa296cb6e20de00faec03e/zookeeper-server/src/main/java/org/apache/zookeeper/common/ZKTrustManager.java";>Base
+ *  revision
+ */
+@InterfaceAudience.Private
+public class HBaseTrustManager extends X509ExtendedTrustManager {

Review Comment:
   this file was copied verbatim



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] bbeaudreault commented on a diff in pull request #4724: HBASE-27280 Add mutual authentication support to TLS

[GitBox]

bbeaudreault commented on code in PR #4724:
URL: https://github.com/apache/hbase/pull/4724#discussion_r954065571


##
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/tls/HBaseHostnameVerifier.java:
##
@@ -0,0 +1,367 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.crypto.tls;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.security.cert.Certificate;
+import java.security.cert.CertificateParsingException;
+import java.security.cert.X509Certificate;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import javax.naming.InvalidNameException;
+import javax.naming.NamingException;
+import javax.naming.directory.Attribute;
+import javax.naming.directory.Attributes;
+import javax.naming.ldap.LdapName;
+import javax.naming.ldap.Rdn;
+import javax.net.ssl.HostnameVerifier;
+import javax.net.ssl.SSLException;
+import javax.net.ssl.SSLPeerUnverifiedException;
+import javax.net.ssl.SSLSession;
+import javax.security.auth.x500.X500Principal;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * When enabled in {@link X509Util}, handles verifying that the hostname of a 
peer matches the
+ * certificate it presents.
+ * 
+ * This file has been copied from the Apache ZooKeeper project.
+ * @see https://github.com/apache/zookeeper/blob/5820d10d9dc58c8e12d2e25386fdf92acb360359/zookeeper-server/src/main/java/org/apache/zookeeper/common/ZKHostnameVerifier.java";>Base
+ *  revision
+ */
+@InterfaceAudience.Private
+final class HBaseHostnameVerifier implements HostnameVerifier {

Review Comment:
   this file was copied verbatim



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Huaxiang Sun (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584328#comment-17584328
 ] 

Huaxiang Sun commented on HBASE-27246:
--

Move to 2.4.15 as 2.4.14RC1 has been built.

> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Huaxiang Sun (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun updated HBASE-27246:
-
Fix Version/s: 2.4.15
   (was: 2.4.14)

> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4721: HBASE-27231 FSHLog should retry writing WAL entries when syncs to HDF…

[GitBox]

Apache-HBase commented on PR #4721:
URL: https://github.com/apache/hbase/pull/4721#issuecomment-1225905017

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 38s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 52s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 16s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 49s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 48s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 17s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 206m 27s |  hbase-server in the patch passed.  
|
   |  |   | 226m  1s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4721 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 80232c8f3837 5.4.0-1083-aws #90~18.04.1-Ubuntu SMP Fri Aug 5 
08:12:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/6/testReport/
 |
   | Max. process+thread count | 2822 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/6/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4728: HBASE-27325 the bulkload max call queue size can be update to a wrong value

[GitBox]

Apache-HBase commented on PR #4728:
URL: https://github.com/apache/hbase/pull/4728#issuecomment-1225902598

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 11s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 29s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 17s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 32s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 41s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 21s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 13s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 16s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 16s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 31s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   8m  3s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  spotless  |   0m 39s |  patch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 26s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 11s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  29m  6s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4728 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux 7360c3aceb03 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 
15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4728/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (HBASE-27325) the bulkload max call queue size can be update to a wrong value

[ruanhui (Jira)]

ruanhui created HBASE-27325:
---

 Summary: the bulkload max call queue size can be update to a wrong 
value
 Key: HBASE-27325
 URL: https://issues.apache.org/jira/browse/HBASE-27325
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Affects Versions: 3.0.0-alpha-3
Reporter: ruanhui
Assignee: ruanhui
 Fix For: 3.0.0-alpha-4


The configKey can be wrong, because 

name.toLowerCase(Locale.ROOT).contains("bulkLoad") is always false.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4727: HBASE-27317 : Storefiletracking : Rectifying the option for columnfamily as mandatory

[GitBox]

Apache-HBase commented on PR #4727:
URL: https://github.com/apache/hbase/pull/4727#issuecomment-1225790253

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 18s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   3m 33s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 11s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  9s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 53s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 50s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 50s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 13s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 254m  2s |  hbase-server in the patch failed.  |
   |  |   | 277m 42s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4727 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 1ad662f53d26 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/testReport/
 |
   | Max. process+thread count | 2283 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-25983) javadoc generation fails on openjdk-11.0.11+9

[Duo Zhang (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-25983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584263#comment-17584263
 ] 

Duo Zhang commented on HBASE-25983:
---

OK, with YETUS-1197 in place I could successfully build javadoc with jdk11.

> javadoc generation fails on openjdk-11.0.11+9
> -
>
> Key: HBASE-25983
> URL: https://issues.apache.org/jira/browse/HBASE-25983
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.4.3
> Environment: maven - 3.5.4 and 3.6.2
> java - openjdk 11.0.11+9
> centos6
> hbase - 2.4.3
>Reporter: Bryan Beaudreault
>Assignee: Duo Zhang
>Priority: Major
>
> I'm trying to build javadoc for HBase 2.4.3 on jdk11. The command I'm running 
> is as follows:
> {code:java}
> JAVA_HOME=/usr/lib/jvm/java-11-openjdk-11.0.11.0.9-2.el8_4.x86_64/  mvn 
> -Phadoop-3.0 -Phadoop.profile=3.0 -Dhadoop-three.version=3.2.2 
> -Dhadoop.guava.version=27.0-jre -Dslf4j.version=1.7.25 
> -Djetty.version=9.3.29.v20201019 -Dzookeeper.version=3.5.7 -DskipTests 
> -Dcheckstyle.skip=true site{code}
> I've tried this with maven 3.5.4 and 3.6.2. Based on JAVA_HOME above, 
> jdk11.0.11+9.
> {{The error is as follows:}}
> {code:java}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-site-plugin:3.7.1:site (default-site) on 
> project hbase: Error generating maven-javadoc-plugin:3.2.0:aggregate-no-fork 
> report:
>  [ERROR] Exit code: 1 - javadoc: warning - The old Doclet and Taglet APIs in 
> the packages
>  [ERROR] com.sun.javadoc, com.sun.tools.doclets and their implementations
>  [ERROR] are planned to be removed in a future JDK release. These
>  [ERROR] components have been superseded by the new APIs in 
> jdk.javadoc.doclet.
>  [ERROR] Users are strongly recommended to migrate to the new APIs.
>  [ERROR] javadoc: error - invalid flag: -author
>  [ERROR]
>  [ERROR] Command line was: 
> /usr/lib/jvm/java-11-openjdk-11.0.11.0.9-2.el8_4.x86_64/bin/javadoc -J-Xmx2G 
> @options @packages
>  [ERROR]
>  [ERROR] Refer to the generated Javadoc files in 
> '/hbase/rpm/build/BUILD/hbase-2.4.3/target/site/apidocs' dir.
>  [ERROR] -> [Help 1]
>  [ERROR]
>  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
>  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>  [ERROR]
>  [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
>  [ERROR] [Help 1]{code}
> I believe this is due to the yetus doclet 
> org.apache.yetus.audience.tools.IncludePublicAnnotationsStandardDoclet. 
> Commenting this doclet out from the userapi and testuserapi reportSets in 
> pom.xml fixes the build.
>  
>  I noticed hbase 2.4.3 depends on audience-annotations 0.5.0, which is very 
> old. I tried updating to 0.13.0, but that did not help. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27321) The ReplicationLogCleaner is not thread safe but can be called from different thread at the same time

[Wellington Chevreuil (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584259#comment-17584259
 ] 

Wellington Chevreuil commented on HBASE-27321:
--

{quote}

So I prefer the latter approach, to disable multiple cleaners run at the same 
time.

{quote}

+1 for this approach.

> The ReplicationLogCleaner is not thread safe but can be called from different 
> thread at the same time
> -
>
> Key: HBASE-27321
> URL: https://issues.apache.org/jira/browse/HBASE-27321
> Project: HBase
>  Issue Type: Bug
>  Components: master, Replication
>Reporter: Duo Zhang
>Priority: Critical
>
> In preClean method we will update the class fields and then use them in the 
> getDeletableFiles method. This implies that we will have only one cleaner run 
> at the same time.
> But actually, in MasterRpcServices.runCleanerChore, we will call execute the 
> HFileCleaner and LogCleaner directly, not in the cleanerChore thread.
> This could cause data loss for replication, which is a very serious problem.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4727: HBASE-27317 : Storefiletracking : Rectifying the option for columnfamily as mandatory

[GitBox]

Apache-HBase commented on PR #4727:
URL: https://github.com/apache/hbase/pull/4727#issuecomment-1225729404

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  2s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 48s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 49s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 47s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 47s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 210m 55s |  hbase-server in the patch passed.  
|
   |  |   | 229m 13s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4727 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 28330b3f4c38 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 
15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/testReport/
 |
   | Max. process+thread count | 2553 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4726: HBASE-27313: Persist list of Hfiles names for which prefetch is done

[GitBox]

Apache-HBase commented on PR #4726:
URL: https://github.com/apache/hbase/pull/4726#issuecomment-1225714060

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 41s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 22s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m  1s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 10s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 11s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m  1s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 25s |  hbase-protocol-shaded in the patch 
passed.  |
   | -1 :x: |  unit  | 256m 57s |  hbase-server in the patch failed.  |
   |  |   | 276m 58s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4726 |
   | JIRA Issue | HBASE-27313 |
   | Optional Tests | unit javac javadoc shadedjars compile |
   | uname | Linux 9f9632b7e7ed 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/testReport/
 |
   | Max. process+thread count | 2820 (vs. ulimit of 3) |
   | modules | C: hbase-protocol-shaded hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-27323) Support to take the initiative to compact cold large files and compression diff hfiles after changing storefile compression algrithm

[Bryan Beaudreault (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584242#comment-17584242
 ] 

Bryan Beaudreault commented on HBASE-27323:
---

Is this different from setting one of COMPRESSION_COMPACT, 
COMPRESSION_COMPACT_MAJOR, COMPRESSION_COMPACT_MINOR?

> Support to take the initiative to compact cold large files and compression 
> diff hfiles after changing storefile compression algrithm 
> -
>
> Key: HBASE-27323
> URL: https://issues.apache.org/jira/browse/HBASE-27323
> Project: HBase
>  Issue Type: Improvement
>  Components: Compaction
>Affects Versions: 3.0.0-alpha-3, 2.4.13
>Reporter: Xiaolin Ha
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> We can set switch to enable this feature, to make the compression algrithm 
> changes be applied to all the existing store files, especially when we make a 
> lower space usage compression algrithm change, e.g. from LZO to ZSTD can save 
> more than 30% spaces.
> All the compaction policies should support this feature.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache-HBase commented on pull request #4721: HBASE-27231 FSHLog should retry writing WAL entries when syncs to HDF…

[GitBox]

Apache-HBase commented on PR #4721:
URL: https://github.com/apache/hbase/pull/4721#issuecomment-1225655061

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  3s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 10s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 13s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 30s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 38s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 15s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 11s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 13s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 13s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 30s |  hbase-server: The patch 
generated 0 new + 21 unchanged - 2 fixed = 21 total (was 23)  |
   | +1 :green_heart: |  whitespace  |   0m  1s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   7m 47s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  spotless  |   0m 40s |  patch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 24s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 10s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  27m 44s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/6/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4721 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux 2e07e1a9bc96 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/6/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4726: HBASE-27313: Persist list of Hfiles names for which prefetch is done

[GitBox]

Apache-HBase commented on PR #4726:
URL: https://github.com/apache/hbase/pull/4726#issuecomment-1225652740

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 54s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 25s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 46s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 31s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 32s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 23s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 40s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 40s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 47s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 40s |  hbase-protocol-shaded in the patch 
passed.  |
   | -1 :x: |  unit  | 200m 43s |  hbase-server in the patch failed.  |
   |  |   | 225m 39s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4726 |
   | JIRA Issue | HBASE-27313 |
   | Optional Tests | unit javac javadoc shadedjars compile |
   | uname | Linux 6a4668a1b737 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   | unit | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/testReport/
 |
   | Max. process+thread count | 2414 (vs. ulimit of 3) |
   | modules | C: hbase-protocol-shaded hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-27320) hide some sensitive configuration information in the UI

[Bryan Beaudreault (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584221#comment-17584221
 ] 

Bryan Beaudreault commented on HBASE-27320:
---

Nice, thanks [~frostruan]!

> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] sunhelly commented on pull request #4536: HBASE-25709 Close region may stuck when region is compacting and skip…

[GitBox]

sunhelly commented on PR #4536:
URL: https://github.com/apache/hbase/pull/4536#issuecomment-1225609176

   Ping @virajjasani , please take a look in your convenience, thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-24950) Another solution for Splittable Meta

[Duo Zhang (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-24950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584183#comment-17584183
 ] 

Duo Zhang commented on HBASE-24950:
---

The development work is current on hold. Please see the last comment in 
HBASE-25382.

{quote}
This is a note for folks watching this jira or generally interested in the 
"split meta" feature work. The work is currently on indefinite hold because 
attempts to reach consensus have broken down. Committers if you believe you 
have consensus to commit something related to this work please touch base with 
private@hbase prior to sending the changes into the master branch or any 
release train branch.
{quote}

> Another solution for Splittable Meta
> 
>
> Key: HBASE-24950
> URL: https://issues.apache.org/jira/browse/HBASE-24950
> Project: HBase
>  Issue Type: Umbrella
>  Components: meta, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> For implementing splittable meta with a master local region to store the 
> 'root table'.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache9 commented on a diff in pull request #4691: HBASE-20904 Prometheus /metrics http endpoint for monitoring

[GitBox]

Apache9 commented on code in PR #4691:
URL: https://github.com/apache/hbase/pull/4691#discussion_r953651328


##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -154,6 +154,18 @@ public class HttpServer implements FilterContainer {
   public static final String NO_CACHE_FILTER = "NoCacheFilter";
   public static final String APP_DIR = "webapps";
 
+  public static final String METRIC_SERVLETS_CONF_KEY = 
"hbase.http.metrics.servlets";
+  public static final String[] METRICS_SERVLETS_DEFAULT = { "jmx", "metrics", 
"prometheus" };
+  private static final ImmutableMap METRIC_SERVLETS = new ImmutableMap.Builder()
+  .put("jmx",
+new ServletConfig("jmx", "/jmx", 
"org.apache.hadoop.hbase.http.jmx.JMXJsonServlet"))
+  .put("metrics",
+new ServletConfig("metrics", "/metrics", 
"org.apache.hadoop.metrics.MetricsServlet"))
+  .put("prometheus", new ServletConfig("prometheus", "/prometheus",

Review Comment:
   Ah, this is a problem. I just read it as 'Prometheus metrics http endpoint 
for monitoring'... Missed the '/'



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a diff in pull request #4691: HBASE-20904 Prometheus /metrics http endpoint for monitoring

[GitBox]

Apache9 commented on code in PR #4691:
URL: https://github.com/apache/hbase/pull/4691#discussion_r953649099


##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.http.prometheus;
+
+import com.google.errorprone.annotations.RestrictedApi;
+import java.io.IOException;
+import java.io.Writer;
+import java.util.Collection;
+import java.util.regex.Pattern;
+import javax.servlet.http.HttpServlet;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.metrics2.AbstractMetric;
+import org.apache.hadoop.metrics2.MetricType;
+import org.apache.hadoop.metrics2.MetricsRecord;
+import org.apache.hadoop.metrics2.MetricsTag;
+import org.apache.hadoop.metrics2.impl.MetricsExportHelper;
+import org.apache.yetus.audience.InterfaceAudience;
+
+@InterfaceAudience.Private
+public class PrometheusHadoopServlet extends HttpServlet {

Review Comment:
   I do not see any advantages here to use jersey? The servlet is really easy 
to implement...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a diff in pull request #4691: HBASE-20904 Prometheus /metrics http endpoint for monitoring

[GitBox]

Apache9 commented on code in PR #4691:
URL: https://github.com/apache/hbase/pull/4691#discussion_r953648382


##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.http.prometheus;
+
+import com.google.errorprone.annotations.RestrictedApi;
+import java.io.IOException;
+import java.io.Writer;
+import java.util.Collection;
+import java.util.regex.Pattern;
+import javax.servlet.http.HttpServlet;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.metrics2.AbstractMetric;
+import org.apache.hadoop.metrics2.MetricType;
+import org.apache.hadoop.metrics2.MetricsRecord;
+import org.apache.hadoop.metrics2.MetricsTag;
+import org.apache.hadoop.metrics2.impl.MetricsExportHelper;
+import org.apache.yetus.audience.InterfaceAudience;
+
+@InterfaceAudience.Private
+public class PrometheusHadoopServlet extends HttpServlet {

Review Comment:
   Because it exports the metrics we registered to hadoop metrics system?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache9 commented on a diff in pull request #4691: HBASE-20904 Prometheus /metrics http endpoint for monitoring

[GitBox]

Apache9 commented on code in PR #4691:
URL: https://github.com/apache/hbase/pull/4691#discussion_r953647493


##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -784,6 +787,22 @@ protected void addDefaultServlets(ContextHandlerCollection 
contexts, Configurati
   LOG.info("ASYNC_PROFILER_HOME environment variable and 
async.profiler.home system property "
 + "not specified. Disabling /prof endpoint.");
 }
+
+/* register metrics servlets */
+String[] enabledServlets = conf.getStrings(METRIC_SERVLETS_CONF_KEY, 
METRICS_SERVLETS_DEFAULT);
+for (String enabledServlet : enabledServlets) {
+  try {
+ServletConfig servletConfig = METRIC_SERVLETS.get(enabledServlet);
+if (servletConfig != null) {
+  Class clz = Class.forName(servletConfig.getClazz());
+  addPrivilegedServlet(servletConfig.getName(), 
servletConfig.getPathSpec(),
+clz.asSubclass(HttpServlet.class));
+}
+  } catch (Exception e) {
+/* shouldn't be fatal, so warn the user about it */

Review Comment:
   For me, I think the current implementation is enough. We will warn if a 
registered endpoint fails to load. And the default configuration is to load all 
metrics endpoints, if users configured it to none, they must have a reason, so 
I do not think here we need to warn it. And I also do not think we should abort 
the process if none metrics can be loaded, as it does not effect the normal 
read/write.



##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -784,6 +787,22 @@ protected void addDefaultServlets(ContextHandlerCollection 
contexts, Configurati
   LOG.info("ASYNC_PROFILER_HOME environment variable and 
async.profiler.home system property "
 + "not specified. Disabling /prof endpoint.");
 }
+
+/* register metrics servlets */
+String[] enabledServlets = conf.getStrings(METRIC_SERVLETS_CONF_KEY, 
METRICS_SERVLETS_DEFAULT);
+for (String enabledServlet : enabledServlets) {
+  try {
+ServletConfig servletConfig = METRIC_SERVLETS.get(enabledServlet);
+if (servletConfig != null) {
+  Class clz = Class.forName(servletConfig.getClazz());
+  addPrivilegedServlet(servletConfig.getName(), 
servletConfig.getPathSpec(),
+clz.asSubclass(HttpServlet.class));
+}
+  } catch (Exception e) {
+/* shouldn't be fatal, so warn the user about it */

Review Comment:
   For me, I think the current implementation is enough. We will warn if a 
registered endpoint fails to load. And the default configuration is to load all 
metrics endpoints, if users configured it to none, they must have a reason, so 
I do not think here we need to warn it. And I also do not think we should abort 
the process if none metrics can be loaded, as it does not affect the normal 
read/write.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4721: HBASE-27231 FSHLog should retry writing WAL entries when syncs to HDF…

[GitBox]

Apache-HBase commented on PR #4721:
URL: https://github.com/apache/hbase/pull/4721#issuecomment-1225520108

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 41s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  5s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 24s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 36s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m  4s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 22s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 12s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 35s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 35s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m  5s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 22s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 218m 17s |  hbase-server in the patch failed.  |
   |  |   | 235m  1s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4721 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 75519dd2e7f3 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b4e5875dd9 |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | unit | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/testReport/
 |
   | Max. process+thread count | 2426 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] ndimiduk commented on a diff in pull request #4691: HBASE-20904 Prometheus /metrics http endpoint for monitoring

[GitBox]

ndimiduk commented on code in PR #4691:
URL: https://github.com/apache/hbase/pull/4691#discussion_r953611489


##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -154,6 +154,18 @@ public class HttpServer implements FilterContainer {
   public static final String NO_CACHE_FILTER = "NoCacheFilter";
   public static final String APP_DIR = "webapps";
 
+  public static final String METRIC_SERVLETS_CONF_KEY = 
"hbase.http.metrics.servlets";
+  public static final String[] METRICS_SERVLETS_DEFAULT = { "jmx", "metrics", 
"prometheus" };
+  private static final ImmutableMap METRIC_SERVLETS = new ImmutableMap.Builder()
+  .put("jmx",
+new ServletConfig("jmx", "/jmx", 
"org.apache.hadoop.hbase.http.jmx.JMXJsonServlet"))
+  .put("metrics",
+new ServletConfig("metrics", "/metrics", 
"org.apache.hadoop.metrics.MetricsServlet"))
+  .put("prometheus", new ServletConfig("prometheus", "/prometheus",

Review Comment:
   We should have changed the title of the jira to match the actual endpoint 
this feature uses. People will be confused that `/metrics` does not emit the 
prometheus formatted output.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4721: HBASE-27231 FSHLog should retry writing WAL entries when syncs to HDF…

[GitBox]

Apache-HBase commented on PR #4721:
URL: https://github.com/apache/hbase/pull/4721#issuecomment-1225512980

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 18s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 34s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 45s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 46s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 34s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 47s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 44s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 211m 54s |  hbase-server in the patch passed.  
|
   |  |   | 228m 30s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4721 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux bbc17cc31201 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b4e5875dd9 |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/testReport/
 |
   | Max. process+thread count | 2497 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4721/5/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4727: HBASE-27317 : Storefiletracking : Rectifying the option for columnfamily as mandatory

[GitBox]

Apache-HBase commented on PR #4727:
URL: https://github.com/apache/hbase/pull/4727#issuecomment-1225510528

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  5s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m 27s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 13s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 33s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 41s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 19s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   2m  8s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 14s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 14s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 30s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |   8m  3s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  spotless  |   0m 38s |  patch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   1m 24s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 12s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  28m 44s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4727 |
   | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti 
spotless checkstyle compile |
   | uname | Linux 3edf4348c242 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4727/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] ndimiduk commented on a diff in pull request #4691: HBASE-20904 Prometheus /metrics http endpoint for monitoring

[GitBox]

ndimiduk commented on code in PR #4691:
URL: https://github.com/apache/hbase/pull/4691#discussion_r953603702


##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.http.prometheus;
+
+import com.google.errorprone.annotations.RestrictedApi;
+import java.io.IOException;
+import java.io.Writer;
+import java.util.Collection;
+import java.util.regex.Pattern;
+import javax.servlet.http.HttpServlet;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.metrics2.AbstractMetric;
+import org.apache.hadoop.metrics2.MetricType;
+import org.apache.hadoop.metrics2.MetricsRecord;
+import org.apache.hadoop.metrics2.MetricsTag;
+import org.apache.hadoop.metrics2.impl.MetricsExportHelper;
+import org.apache.yetus.audience.InterfaceAudience;
+
+@InterfaceAudience.Private
+public class PrometheusHadoopServlet extends HttpServlet {

Review Comment:
   Why is this called a "Hadoop" servlet?



##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.http.prometheus;
+
+import com.google.errorprone.annotations.RestrictedApi;
+import java.io.IOException;
+import java.io.Writer;
+import java.util.Collection;
+import java.util.regex.Pattern;
+import javax.servlet.http.HttpServlet;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.metrics2.AbstractMetric;
+import org.apache.hadoop.metrics2.MetricType;
+import org.apache.hadoop.metrics2.MetricsRecord;
+import org.apache.hadoop.metrics2.MetricsTag;
+import org.apache.hadoop.metrics2.impl.MetricsExportHelper;
+import org.apache.yetus.audience.InterfaceAudience;
+
+@InterfaceAudience.Private
+public class PrometheusHadoopServlet extends HttpServlet {

Review Comment:
   Oh no, why was this implemented as a raw servlet :'(
   
   We have Jersey. There is absolutely no reason to implement a servlet by hand.



##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -784,6 +787,22 @@ protected void addDefaultServlets(ContextHandlerCollection 
contexts, Configurati
   LOG.info("ASYNC_PROFILER_HOME environment variable and 
async.profiler.home system property "
 + "not specified. Disabling /prof endpoint.");
 }
+
+/* register metrics servlets */
+String[] enabledServlets = conf.getStrings(METRIC_SERVLETS_CONF_KEY, 
METRICS_SERVLETS_DEFAULT);
+for (String enabledServlet : enabledServlets) {
+  try {
+ServletConfig servletConfig = METRIC_SERVLETS.get(enabledServlet);
+if (servletConfig != null) {
+  Class clz = Class.forName(servletConfig.getClazz());
+  addPrivilegedServlet(servletConfig.getName(), 
servletConfig.getPathSpec(),
+clz.asSubclass(HttpServlet.class));
+}
+  } catch (Exception e) {
+/* shouldn't be fatal, so warn the user about it */

Review Comment:
   I'm surprised that having metrics be optional was accepted by review. Before 
this change, it was impossible to run an HBase processes 

[jira] [Commented] (HBASE-24950) Another solution for Splittable Meta

[Xiaolin Ha (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-24950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584159#comment-17584159
 ] 

Xiaolin Ha commented on HBASE-24950:


Hi, [~zhangduo] , will this issue be going on? Is there any plan to merge to 
master branch?

> Another solution for Splittable Meta
> 
>
> Key: HBASE-24950
> URL: https://issues.apache.org/jira/browse/HBASE-24950
> Project: HBase
>  Issue Type: Umbrella
>  Components: meta, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> For implementing splittable meta with a master local region to store the 
> 'root table'.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27324) Auto merging small stripes in StripeCompactionPolicy

[Xiaolin Ha (Jira)]

Xiaolin Ha created HBASE-27324:
--

 Summary: Auto merging small stripes in StripeCompactionPolicy
 Key: HBASE-27324
 URL: https://issues.apache.org/jira/browse/HBASE-27324
 Project: HBase
  Issue Type: Improvement
  Components: Compaction
Affects Versions: 2.4.13, 3.0.0-alpha-3
Reporter: Xiaolin Ha
 Fix For: 2.6.0, 3.0.0-alpha-4


Currently, stripes merge only happens when two stripes are empty, but that is 
not enough. With the data appending the stripes splits by splits, but with the 
data deleting, stripes might become smaller size while the stripes count is 
large. 

Auto merging small stripes can help to reduce the file count in the store and 
improve read effiecency. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27323) Support to take the initiative to compact cold large files and compression diff hfiles after changing storefile compression algrithm

[Xiaolin Ha (Jira)]

Xiaolin Ha created HBASE-27323:
--

 Summary: Support to take the initiative to compact cold large 
files and compression diff hfiles after changing storefile compression algrithm 
 Key: HBASE-27323
 URL: https://issues.apache.org/jira/browse/HBASE-27323
 Project: HBase
  Issue Type: Improvement
  Components: Compaction
Affects Versions: 2.4.13, 3.0.0-alpha-3
Reporter: Xiaolin Ha
 Fix For: 2.6.0, 3.0.0-alpha-4


We can set switch to enable this feature, to make the compression algrithm 
changes be applied to all the existing store files, especially when we make a 
lower space usage compression algrithm change, e.g. from LZO to ZSTD can save 
more than 30% spaces.

All the compaction policies should support this feature.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] abhradeepkundu opened a new pull request, #4727: HBASE-27317 : Storefiletracking : Rectifying the option for columnfamily as mandatory

[GitBox]

abhradeepkundu opened a new pull request, #4727:
URL: https://github.com/apache/hbase/pull/4727

   This is to rectify the columnfamily argument from optional to mandatory. In 
the sysout message it is wrongly put as optional parameter but it is actually a 
mandatory parameter.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] frostruan commented on pull request #4723: HBASE-27320 hide some sensitive configuration information in the UI

[GitBox]

frostruan commented on PR #4723:
URL: https://github.com/apache/hbase/pull/4723#issuecomment-1225475960

   > The list of MASK_PROPERTIES may be configurable?
   
   Thanks for your advice. Will start this work later.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4726: HBASE-27313: Persist list of Hfiles names for which prefetch is done

[GitBox]

Apache-HBase commented on PR #4726:
URL: https://github.com/apache/hbase/pull/4726#issuecomment-1225471999

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 10s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +0 :ok: |  prototool  |   0m  1s |  prototool was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   3m 18s |  master passed  |
   | +1 :green_heart: |  compile  |   3m 47s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 37s |  master passed  |
   | +1 :green_heart: |  spotless  |   0m 44s |  branch has no errors when 
running spotless:check.  |
   | +1 :green_heart: |  spotbugs  |   5m 26s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 10s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 51s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 32s |  the patch passed  |
   | +1 :green_heart: |  cc  |   3m 32s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 32s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 36s |  hbase-server: The patch 
generated 13 new + 5 unchanged - 0 fixed = 18 total (was 5)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  hadoopcheck  |  13m 14s |  Patch does not cause any 
errors with Hadoop 3.2.4 3.3.4.  |
   | +1 :green_heart: |  hbaseprotoc  |   1m 46s |  the patch passed  |
   | -1 :x: |  spotless  |   0m 54s |  patch has 68 errors when running 
spotless:check, run spotless:apply to fix.  |
   | -1 :x: |  spotbugs  |   1m 41s |  hbase-server generated 2 new + 0 
unchanged - 0 fixed = 2 total (was 0)  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 16s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  52m  1s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | FindBugs | module:hbase-server |
   |  |  
org.apache.hadoop.hbase.io.hfile.CacheConfig.PREFETCH_PERSISTENCE_PATH_KEY 
isn't final but should be  At CacheConfig.java:be  At CacheConfig.java:[line 
96] |
   |  |  org.apache.hadoop.hbase.io.hfile.PrefetchExecutor.prefetchCompleted 
isn't final but should be  At PrefetchExecutor.java:be  At 
PrefetchExecutor.java:[line 54] |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-general-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4726 |
   | JIRA Issue | HBASE-27313 |
   | Optional Tests | dupname asflicense cc hbaseprotoc spotless prototool 
javac spotbugs hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 808d0edae963 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / b44bfc52cc |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   | checkstyle | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
 |
   | spotless | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-general-check/output/patch-spotless.txt
 |
   | spotbugs | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html
 |
   | Max. process+thread count | 64 (vs. ulimit of 3) |
   | modules | C: hbase-protocol-shaded hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4726/1/console 
|
   | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (HBASE-27246) RSGroupMappingScript#getRSGroup has thread safety problem

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-27246.
---
Fix Version/s: 2.6.0
   2.5.1
   3.0.0-alpha-4
   2.4.14
 Hadoop Flags: Reviewed
   Resolution: Fixed

Pushed to branch-2.4+.

Thanks [~xytss123] for contributing!

> RSGroupMappingScript#getRSGroup has thread safety problem
> -
>
> Key: HBASE-27246
> URL: https://issues.apache.org/jira/browse/HBASE-27246
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.14
>
> Attachments: Test.java, result.png
>
>
> We are using version 1.4.12 and met a problem in table creation phase some 
> time. The master branch also has this problem. The error message is:
> {code:java}
> 2022-07-26 19:26:20.122 [http-nio-8078-exec-24,d2ad4b13b542b6fb] ERROR 
> HBaseServiceImpl - hbase create table: xxx: failed. 
> (HBaseServiceImpl.java:116)
> java.lang.RuntimeException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: 
> org.apache.hadoop.hbase.constraint.ConstraintException: Default RSGroup 
> (default
> default) for this table's namespace does not exist.
> {code}
> The rsgroup here should be one 'default' but not two consecutive 'default'.  
> The code to get RSGroup from a mapping script is:
> {code:java}
> String getRSGroup(String namespace, String tablename) {
>   if (rsgroupMappingScript == null) {
> return null;
>   }
>   String[] exec = rsgroupMappingScript.getExecString();
>   exec[1] = namespace;
>   exec[2] = tablename;
>   try {
> rsgroupMappingScript.execute();
>   } catch (IOException e) {
> // This exception may happen, like process doesn't have permission to 
> run this script.
> LOG.error("{}, placing {} back to default rsgroup", e.getMessage(),
>   TableName.valueOf(namespace, tablename));
> return RSGroupInfo.DEFAULT_GROUP;
>   }
>   return rsgroupMappingScript.getOutput().trim();
> }
> {code}
> here the rsgourpMappingScript could be executed by multi-threads.
> To test it is a multi-thread issue, I ran a piece of code locally and found 
> that the hadoop ShellCommandExecutor is not thread-safe (I run the code with 
> hadoop 2.10.0 and 3.3.2). So that we should make this method synchronized. 
> Besides, I found that this issue is retained in master branch also.
> The test code is attached and my rsgroup mapping script is very simple:
> {code:java}
> #!/bin/bash
> namespace=$1
> tablename=$2
> echo default
> {code}
> The reproduced screenshot is also attached.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27313) Persist list of Hfiles names for which prefetch is done

[Shanmukha Haripriya Kota (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shanmukha Haripriya Kota updated HBASE-27313:
-
Status: Patch Available  (was: In Progress)

> Persist list of Hfiles names for which prefetch is done
> ---
>
> Key: HBASE-27313
> URL: https://issues.apache.org/jira/browse/HBASE-27313
> Project: HBase
>  Issue Type: Improvement
>  Components: BucketCache
>Reporter: Shanmukha Haripriya Kota
>Assignee: Shanmukha Haripriya Kota
>Priority: Major
>
> Persist list of Hfiles names for which prefetch is completed when persist for 
> in-memory bucket cache mapping state is enabled.
> We need to persist the Set, which contains a list of all the HFiles for which 
> the prefetch has completed along with the In-memory state of the bucket cache 
> so that when we restart and if any prefetch is scheduled again for the same 
> HFile and we already have the in-memory state retrieved, we can skip the 
> prefetch for it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Kota-SH opened a new pull request, #4726: HBASE-27313: Persist list of Hfiles names for which prefetch is done

[GitBox]

Kota-SH opened a new pull request, #4726:
URL: https://github.com/apache/hbase/pull/4726

   Jira link: https://issues.apache.org/jira/browse/HBASE-27313 
   
   Summary:
   Persist list of Hfiles names for which prefetch is completed when persist 
for in-memory bucket cache mapping state is enabled
   
   We need to persist the Set, which contains a list of all the HFiles for 
which the prefetch has completed along with the In-memory state of the bucket 
cache so that when we restart and if any prefetch is scheduled again for the 
same HFile and we already have the in-memory state retrieved, we can skip the 
prefetch for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] fan1emon commented on pull request #4719: HBASE-27316 Time based metrics will be reset after any get request

[GitBox]

fan1emon commented on PR #4719:
URL: https://github.com/apache/hbase/pull/4719#issuecomment-1225376011

   First query jmx
   ```
 "FlushTime_num_ops": 11142,
 "FlushTime_min": 1201,
 "FlushTime_max": 4614,
 "FlushTime_mean": 2437,
 "FlushTime_25th_percentile": 1344,
 "FlushTime_median": 2383,
 "FlushTime_75th_percentile": 2647,
 "FlushTime_90th_percentile": 4607,
 "FlushTime_95th_percentile": 4610,
 "FlushTime_98th_percentile": 4612,
 "FlushTime_99th_percentile": 4613,
 "FlushTime_99.9th_percentile": 4613,
 "FlushTime_TimeRangeCount_1000-3000": 4,
 "FlushTime_TimeRangeCount_3000-1": 1,
   ```
   Next
   ```
 "FlushTime_num_ops": 11191,
 "FlushTime_min": 534,
 "FlushTime_max": 3932,
 "FlushTime_mean": 2835,
 "FlushTime_25th_percentile": 2676,
 "FlushTime_median": 3104,
 "FlushTime_75th_percentile": 3531,
 "FlushTime_90th_percentile": 3913,
 "FlushTime_95th_percentile": 3922,
 "FlushTime_98th_percentile": 3928,
 "FlushTime_99th_percentile": 3930,
 "FlushTime_99.9th_percentile": 3931,
 "FlushTime_TimeRangeCount_300-1000": 1,
 "FlushTime_TimeRangeCount_1000-3000": 2,
 "FlushTime_TimeRangeCount_3000-1": 4,
   ```
   Some metrics like `FlushTime_max` , `FlushTime_TimeRangeCount_1000-3000` are 
reset


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] fan1emon commented on pull request #4719: HBASE-27316 Time based metrics will be reset after any get request

[GitBox]

fan1emon commented on PR #4719:
URL: https://github.com/apache/hbase/pull/4719#issuecomment-1225369787

   I can't figure out why snapshot will be reset after any get request.
   The count of ops will keeps growing while the quantiles are reset.
   
   I'll fix the test error if this patch needed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (HBASE-27109) Move replication queue storage from zookeeper to a separated HBase table

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584109#comment-17584109
 ] 

Hudson commented on HBASE-27109:


Results for branch HBASE-27109/table_based_rqs
[build #7 on 
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/7/]:
 (/) *{color:green}+1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/7/General_20Nightly_20Build_20Report/]




(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/7/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/7/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Move replication queue storage from zookeeper to a separated HBase table
> 
>
> Key: HBASE-27109
> URL: https://issues.apache.org/jira/browse/HBASE-27109
> Project: HBase
>  Issue Type: New Feature
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> This is a more specific issue based on the works which are already done in 
> HBASE-15867.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] fanyunbojerry commented on pull request #4723: HBASE-27320 hide some sensitive configuration information in the UI

[GitBox]

fanyunbojerry commented on PR #4723:
URL: https://github.com/apache/hbase/pull/4723#issuecomment-1225347681

   The list of MASK_PROPERTIES may be configurable?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4713: HBASE-27304 Support using IP to expose master/rs servers for some special scenarios

[GitBox]

Apache-HBase commented on PR #4713:
URL: https://github.com/apache/hbase/pull/4713#issuecomment-1225334855

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 11s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  2s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 16s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   3m 47s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 19s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 57s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   3m 46s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 40s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 44s |  hbase-common in the patch passed.  
|
   | +1 :green_heart: |  unit  | 214m 31s |  hbase-server in the patch passed.  
|
   |  |   | 234m 43s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4713 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 517f43e52634 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 
02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / f9ea7ee0d6 |
   | Default Java | AdoptOpenJDK-1.8.0_282-b08 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/testReport/
 |
   | Max. process+thread count | 2400 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hbase] Apache-HBase commented on pull request #4713: HBASE-27304 Support using IP to expose master/rs servers for some special scenarios

[GitBox]

Apache-HBase commented on PR #4713:
URL: https://github.com/apache/hbase/pull/4713#issuecomment-1225324794

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 38s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): 
--brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list 
--whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   2m 27s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 56s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 12s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 11s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 43s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m  0s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   4m 35s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 47s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   2m  1s |  hbase-common in the patch passed.  
|
   | -1 :x: |  unit  | 202m 51s |  hbase-server in the patch failed.  |
   |  |   | 224m 51s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/4713 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 1d6c5e654440 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 
23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / f9ea7ee0d6 |
   | Default Java | AdoptOpenJDK-11.0.10+9 |
   | unit | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/testReport/
 |
   | Max. process+thread count | 2617 (vs. ulimit of 3) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4713/5/console 
|
   | versions | git=2.17.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (HBASE-27320) hide some sensitive configuration information in the UI

[ruanhui (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ruanhui updated HBASE-27320:

Release Note: hide superuser and password related settings in the 
configuration UI

> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27320) hide some sensitive configuration information in the UI

[ruanhui (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584088#comment-17584088
 ] 

ruanhui commented on HBASE-27320:
-

ok. Thanks for reviewing [~zhangduo] [~taklwu] .

> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [hbase] Apache9 merged pull request #4657: HBASE-27246 RSGroupMappingScript#getRSGroup has thread safety problem

[GitBox]

Apache9 merged PR #4657:
URL: https://github.com/apache/hbase/pull/4657


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (HBASE-27320) hide some sensitive configuration information in the UI

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-27320.
---
Fix Version/s: 2.6.0
   2.5.1
   2.4.15
 Hadoop Flags: Reviewed
   Resolution: Fixed

Pushed to branch-2.4+.

Thanks [~frostruan] for contributing!

Please fill the release note to mention the behavior change.

> hide some sensitive configuration information in the UI
> ---
>
> Key: HBASE-27320
> URL: https://issues.apache.org/jira/browse/HBASE-27320
> Project: HBase
>  Issue Type: Improvement
>  Components: security, UI
>Affects Versions: 3.0.0-alpha-3
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> In the discussion about how to store keystore/truststore password securely, 
> [~bbeaudreault]  mentioned and I quote here
> "I agree that it seems insecure to put it directly into the hbase-site.xml. 
> Another reason is due to the RS UI which (helpfully) can print the entire 
> site configuration. We’d need to make sure the password is excluded from 
> that, but better to remove it from site xml altogether".
> I also felt that some sensitive information was exposed in the UI, for 
> example, if we set superuser in the hbase-site.xml, the non-admin users can 
> obtain superuser information and simulate superuser to perform some 
> non-permitted operations on the cluster. So I think maybe we should hide 
> these sensitive information in the UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)