[jira] [Updated] (HBASE-24696) Include JVM information on Web UI under "Software Attributes"
[ https://issues.apache.org/jira/browse/HBASE-24696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HBASE-24696: -- Attachment: (was: Screen Shot 2020-07-17 at 10.54.09 PM.png) > Include JVM information on Web UI under "Software Attributes" > - > > Key: HBASE-24696 > URL: https://issues.apache.org/jira/browse/HBASE-24696 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Nick Dimiduk >Priority: Minor > Attachments: Screen Shot 2020-07-17 at 10.55.56 PM.png > > > It's a small thing, but seems like an omission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24696) Include JVM information on Web UI under "Software Attributes"
[ https://issues.apache.org/jira/browse/HBASE-24696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HBASE-24696: -- Attachment: Screen Shot 2020-07-17 at 10.55.56 PM.png > Include JVM information on Web UI under "Software Attributes" > - > > Key: HBASE-24696 > URL: https://issues.apache.org/jira/browse/HBASE-24696 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Nick Dimiduk >Priority: Minor > Attachments: Screen Shot 2020-07-17 at 10.55.56 PM.png > > > It's a small thing, but seems like an omission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24696) Include JVM information on Web UI under "Software Attributes"
[ https://issues.apache.org/jira/browse/HBASE-24696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160339#comment-17160339 ] Mingliang Liu commented on HBASE-24696: --- I filed a simple PR. Not sure if that is towards the right direction. Thanks [~ndimiduk] > Include JVM information on Web UI under "Software Attributes" > - > > Key: HBASE-24696 > URL: https://issues.apache.org/jira/browse/HBASE-24696 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Nick Dimiduk >Priority: Minor > Attachments: Screen Shot 2020-07-17 at 10.54.09 PM.png > > > It's a small thing, but seems like an omission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24696) Include JVM information on Web UI under "Software Attributes"
[ https://issues.apache.org/jira/browse/HBASE-24696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HBASE-24696: -- Attachment: Screen Shot 2020-07-17 at 10.54.09 PM.png > Include JVM information on Web UI under "Software Attributes" > - > > Key: HBASE-24696 > URL: https://issues.apache.org/jira/browse/HBASE-24696 > Project: HBase > Issue Type: Improvement > Components: UI >Reporter: Nick Dimiduk >Priority: Minor > Attachments: Screen Shot 2020-07-17 at 10.54.09 PM.png > > > It's a small thing, but seems like an omission. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] liuml07 opened a new pull request #2087: HBASE-24696 Include JVM information on Web UI under "Software Attributes"
liuml07 opened a new pull request #2087: URL: https://github.com/apache/hbase/pull/2087 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
liuml07 commented on pull request #2044: URL: https://github.com/apache/hbase/pull/2044#issuecomment-660431436 > Here need conf also: OffPeakHours.getInstance(conf).isOffPeakHour() Right. Unless we also save `OffPeakHours` instance in constructor...so the value of removing `conf` need much effort. We can stay with current implementation I guess then. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2086: HBASE-24664 Some changing of split region by overall region size rath…
Apache-HBase commented on pull request #2086: URL: https://github.com/apache/hbase/pull/2086#issuecomment-660429646 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 38s | branch-2.3 passed | | +1 :green_heart: | compile | 2m 12s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 11s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | compile | 2m 12s | the patch passed | | +1 :green_heart: | javac | 2m 12s | the patch passed | | +1 :green_heart: | shadedjars | 4m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 10s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 23m 41s | root in the patch failed. | | | | 54m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2086 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7288eb6ff3ca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c582fb5d11 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/testReport/ | | Max. process+thread count | 844 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
bsglz commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456751970
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##
@@ -827,26 +828,34 @@ protected double scale(double min, double max, double
value) {
*/
static class MoveCostFunction extends CostFunction {
private static final String MOVE_COST_KEY =
"hbase.master.balancer.stochastic.moveCost";
+private static final String MOVE_COST_OFFPEAK_KEY =
"hbase.master.balancer.stochastic.moveCost.offpeak";
private static final String MAX_MOVES_PERCENT_KEY =
"hbase.master.balancer.stochastic.maxMovePercent";
-private static final float DEFAULT_MOVE_COST = 7;
+@VisibleForTesting
+protected static final float DEFAULT_MOVE_COST = 7;
+protected static final float DEFAULT_MOVE_COST_OFFPEAK = 3;
Review comment:
IMO, the main different of two ways is whether we need to update test
code or not when we change the default value.
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
bsglz commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456751823
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##
@@ -827,26 +828,34 @@ protected double scale(double min, double max, double
value) {
*/
static class MoveCostFunction extends CostFunction {
private static final String MOVE_COST_KEY =
"hbase.master.balancer.stochastic.moveCost";
+private static final String MOVE_COST_OFFPEAK_KEY =
"hbase.master.balancer.stochastic.moveCost.offpeak";
private static final String MAX_MOVES_PERCENT_KEY =
"hbase.master.balancer.stochastic.maxMovePercent";
-private static final float DEFAULT_MOVE_COST = 7;
+@VisibleForTesting
+protected static final float DEFAULT_MOVE_COST = 7;
+protected static final float DEFAULT_MOVE_COST_OFFPEAK = 3;
Review comment:
Seems the test will not pass when we update the default value?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2086: HBASE-24664 Some changing of split region by overall region size rath…
Apache-HBase commented on pull request #2086: URL: https://github.com/apache/hbase/pull/2086#issuecomment-660428860 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 24s | branch-2.3 passed | | +1 :green_heart: | compile | 2m 47s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 13s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | root in branch-2.3 failed. | | -0 :warning: | javadoc | 0m 17s | hbase-common in branch-2.3 failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 2m 48s | the patch passed | | +1 :green_heart: | javac | 2m 48s | the patch passed | | +1 :green_heart: | shadedjars | 6m 14s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 49s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 16s | root in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 15m 26s | root in the patch failed. | | | | 47m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2086 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 48b3723c0107 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c582fb5d11 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/testReport/ | | Max. process+thread count | 736 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2086/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
bsglz commented on pull request #2044: URL: https://github.com/apache/hbase/pull/2044#issuecomment-660427923 > In my last comment, > > > That said, to remove this.conf = conf dependency, we can get two multipliers from config, and call setMultiplier() accordingly here in cost() by checking isOffPeakHour(). > > I meant in constructor, we can get two multipliers from config and remove the `this.conf` field. In `cost()` we call `setMultiplier()` and choose the right parameter by checking isOffPeakHour(). Here need conf also: OffPeakHours.getInstance(conf).isOffPeakHour() This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
liuml07 commented on pull request #2044: URL: https://github.com/apache/hbase/pull/2044#issuecomment-660427619 In my last comment, > That said, to remove this.conf = conf dependency, we can get two multipliers from config, and call setMultiplier() accordingly here in cost() by checking isOffPeakHour(). I meant in constructor, we can get two multipliers from config and remove the `this.conf` field. In `cost()` we call `setMultiplier()` and choose the right parameter by checking isOffPeakHour(). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
liuml07 commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456750615
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##
@@ -827,26 +828,34 @@ protected double scale(double min, double max, double
value) {
*/
static class MoveCostFunction extends CostFunction {
private static final String MOVE_COST_KEY =
"hbase.master.balancer.stochastic.moveCost";
+private static final String MOVE_COST_OFFPEAK_KEY =
"hbase.master.balancer.stochastic.moveCost.offpeak";
private static final String MAX_MOVES_PERCENT_KEY =
"hbase.master.balancer.stochastic.maxMovePercent";
-private static final float DEFAULT_MOVE_COST = 7;
+@VisibleForTesting
+protected static final float DEFAULT_MOVE_COST = 7;
+protected static final float DEFAULT_MOVE_COST_OFFPEAK = 3;
Review comment:
@bsglz and @virajjasani I do not have preference here if you agree on
that. Either way works obviously!
I asked again (see [later
comment](https://github.com/apache/hbase/pull/2044#discussion_r456748462))
because I'm not sure this is entirely true:
> If default values change here and test is not updated, it is anyways going
to fail.
Suppose there is a bug in code (not today but future). The bug will be
hidden if test still pass (possible) when we update the default value and code,
but not the test.
Another pro of using variable `DEFAULT_MOVE_COST` instead of literal value
`7` is about readability. Clearer to know where the test expected value comes
from. The main con of using `@VisibleForTesting` is about exposing private
values, which I personally always avoid... I can see the value of either way
here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1994: HBASE-24654 Allow unset table's rsgroup
Apache-HBase commented on pull request #1994: URL: https://github.com/apache/hbase/pull/1994#issuecomment-660427160 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 20s | master passed | | +1 :green_heart: | checkstyle | 1m 52s | master passed | | +1 :green_heart: | spotbugs | 3m 38s | master passed | | -0 :warning: | patch | 2m 35s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | checkstyle | 1m 44s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 43m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1994/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1994 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 80c156d4eda2 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9b02a26a1d | | Max. process+thread count | 85 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1994/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
bsglz commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456748565
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
##
@@ -284,6 +284,25 @@ public void testLocalityCost() throws Exception {
}
}
+ @Test
+ public void testMoveCostMultiplier() throws Exception {
+Configuration conf = HBaseConfiguration.create();
+StochasticLoadBalancer.CostFunction
+costFunction = new StochasticLoadBalancer.MoveCostFunction(conf);
+BaseLoadBalancer.Cluster cluster = mockCluster(clusterStateMocks[0]);
+costFunction.init(cluster);
+costFunction.cost();
+assertEquals(7, costFunction.getMultiplier(), 0.01);
Review comment:
@liuml07 Yeah, you are right. Thanks for the comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
bsglz commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456748462
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
##
@@ -284,6 +284,25 @@ public void testLocalityCost() throws Exception {
}
}
+ @Test
+ public void testMoveCostMultiplier() throws Exception {
+Configuration conf = HBaseConfiguration.create();
+StochasticLoadBalancer.CostFunction
+costFunction = new StochasticLoadBalancer.MoveCostFunction(conf);
+BaseLoadBalancer.Cluster cluster = mockCluster(clusterStateMocks[0]);
+costFunction.init(cluster);
+costFunction.cost();
+assertEquals(7, costFunction.getMultiplier(), 0.01);
Review comment:
> Can we please avoid @VisibleForTesting and make them private? On the
other hand, use default value directly in test cases. If default values change
here and test is not updated, it is anyways going to fail.
Discussed about it above with viraj.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-24708) Flaky Test TestRegionReplicas#testVerifySecondaryAbilityToReadWithOnFiles
[ https://issues.apache.org/jira/browse/HBASE-24708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160330#comment-17160330 ] Mingliang Liu commented on HBASE-24708: --- Do you have links to builds / stack trace? > Flaky Test TestRegionReplicas#testVerifySecondaryAbilityToReadWithOnFiles > - > > Key: HBASE-24708 > URL: https://issues.apache.org/jira/browse/HBASE-24708 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 2.3.0 >Reporter: Huaxiang Sun >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] liuml07 commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
liuml07 commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456747846
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
##
@@ -284,6 +284,25 @@ public void testLocalityCost() throws Exception {
}
}
+ @Test
+ public void testMoveCostMultiplier() throws Exception {
+Configuration conf = HBaseConfiguration.create();
+StochasticLoadBalancer.CostFunction
+costFunction = new StochasticLoadBalancer.MoveCostFunction(conf);
+BaseLoadBalancer.Cluster cluster = mockCluster(clusterStateMocks[0]);
+costFunction.init(cluster);
+costFunction.cost();
+assertEquals(7, costFunction.getMultiplier(), 0.01);
Review comment:
Not sure if we want to replace 7 with `DEFAULT_MOVE_COST`, which
currently is private...same to `DEFAULT_MOVE_COST_OFFPEAK`
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##
@@ -827,26 +828,34 @@ protected double scale(double min, double max, double
value) {
*/
static class MoveCostFunction extends CostFunction {
private static final String MOVE_COST_KEY =
"hbase.master.balancer.stochastic.moveCost";
+private static final String MOVE_COST_OFFPEAK_KEY =
"hbase.master.balancer.stochastic.moveCost.offpeak";
private static final String MAX_MOVES_PERCENT_KEY =
"hbase.master.balancer.stochastic.maxMovePercent";
-private static final float DEFAULT_MOVE_COST = 7;
+@VisibleForTesting
+protected static final float DEFAULT_MOVE_COST = 7;
+protected static final float DEFAULT_MOVE_COST_OFFPEAK = 3;
private static final int DEFAULT_MAX_MOVES = 600;
private static final float DEFAULT_MAX_MOVE_PERCENT = 0.25f;
private final float maxMovesPercent;
+private final Configuration conf;
MoveCostFunction(Configuration conf) {
super(conf);
-
- // Move cost multiplier should be the same cost or higher than the rest
of the costs to ensure
- // that large benefits are need to overcome the cost of a move.
- this.setMultiplier(conf.getFloat(MOVE_COST_KEY, DEFAULT_MOVE_COST));
+ this.conf = conf;
// What percent of the number of regions a single run of the balancer
can move.
maxMovesPercent = conf.getFloat(MAX_MOVES_PERCENT_KEY,
DEFAULT_MAX_MOVE_PERCENT);
}
@Override
protected double cost() {
+ // Move cost multiplier should be the same cost or higher than the rest
of the costs to ensure
+ // that large benefits are need to overcome the cost of a move.
+ if (OffPeakHours.getInstance(conf).isOffPeakHour()) {
+this.setMultiplier(conf.getFloat(MOVE_COST_OFFPEAK_KEY,
DEFAULT_MOVE_COST_OFFPEAK));
+ } else {
+this.setMultiplier(conf.getFloat(MOVE_COST_KEY, DEFAULT_MOVE_COST));
+ }
Review comment:
@virajjasani Good question.
I glimpsed and I guess what @bsglz meant is every time calling the
`MoveCostFunction::cost()` function against the same `MoveCostFunction` object
should get a different value per the config and the current time. Since this
`MoveCostFunction` object is constructed once, the multiplier should be updated
here. We can not move the whole if condition to constructor. Updating the
`config` object dynamically is another use case (which should also work).
Correct me if I'm wrong @bsglz .
That said, to remove `this.conf = conf` dependency, we can get two
multipliers from config, and call `setMultiplier()` accordingly here in
`cost()` by checking `isOffPeakHour()`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bsglz opened a new pull request #2086: HBASE-24664 Some changing of split region by overall region size rath…
bsglz opened a new pull request #2086: URL: https://github.com/apache/hbase/pull/2086 …er than only one store size (cherry picked from commit b434ccaaeaeb08b3088a6a437e5d49ef1fcbb0cb) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-23634) Enable "Split WAL to HFile" by default
[ https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160327#comment-17160327 ] Bo Cui edited comment on HBASE-23634 at 7/18/20, 4:01 AM: -- 1、before compaction, large number of small hfiles affect read and write performance of region 2、a hfile needs 3 NN RPCs to bulkload during openRegion(validate、rename、createReader) if bulkLoadService ThreadNum is 3, and hfiles is 20(because wal number is 20), and RS is 100, region is 2K*100, and openRegion thread is 75 so hbase needs 3*3*75*100 concurrent NN RPCs and needs 3*20*2K*100 total NN RPCs was (Author: bo cui): 1、before compaction, large number of small hfiles affect read and write performance of region 2、a hfile needs 3 NN RPCs to bulkload during openRegion(validate、rename、createReader) if bulkLoadService ThreadNum is 3, and hfiles is 20(because wal number is 20), and RS is 100, region is 2K*100, and openRegion thread is 75 so hbase needs 3*3*75*100 concurrent NN RPCs and needs 3*20*2K*100 NN total NN RPCs > Enable "Split WAL to HFile" by default > -- > > Key: HBASE-23634 > URL: https://issues.apache.org/jira/browse/HBASE-23634 > Project: HBase > Issue Type: Task >Affects Versions: 3.0.0-alpha-1, 2.3.0 >Reporter: Guanghao Zhang >Priority: Blocker > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23634) Enable "Split WAL to HFile" by default
[ https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160327#comment-17160327 ] Bo Cui commented on HBASE-23634: 1、before compaction, large number of small hfiles affect read and write performance of region 2、a hfile needs 3 NN RPCs to bulkload during openRegion(validate、rename、createReader) if bulkLoadService ThreadNum is 3, and hfiles is 20(because wal number is 20), and RS is 100, region is 2K*100, and openRegion thread is 75 so hbase needs 3*3*75*100 concurrent NN RPCs and needs 3*20*2K*100 NN total NN RPCs > Enable "Split WAL to HFile" by default > -- > > Key: HBASE-23634 > URL: https://issues.apache.org/jira/browse/HBASE-23634 > Project: HBase > Issue Type: Task >Affects Versions: 3.0.0-alpha-1, 2.3.0 >Reporter: Guanghao Zhang >Priority: Blocker > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bsglz opened a new pull request #2085: HBASE-24664 Some changing of split region by overall region size rath…
bsglz opened a new pull request #2085: URL: https://github.com/apache/hbase/pull/2085 …er than only one store size (cherry picked from commit b434ccaaeaeb08b3088a6a437e5d49ef1fcbb0cb) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2081: HBASE-24742 Improve performance of SKIP vs SEEK logic.
Apache-HBase commented on pull request #2081: URL: https://github.com/apache/hbase/pull/2081#issuecomment-660418361 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 8s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 27s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 20s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 6m 20s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 191m 10s | hbase-server in the patch failed. | | | | 220m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2081 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 279ca8c9cacb 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c582fb5d11 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/testReport/ | | Max. process+thread count | 2635 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2082: HBASE-24570 connection#close throws NPE
Apache-HBase commented on pull request #2082: URL: https://github.com/apache/hbase/pull/2082#issuecomment-660418106 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 22s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 28s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 27s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 0m 40s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | branch-2.2 passed | | +0 :ok: | spotbugs | 1m 30s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 26s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 20s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | checkstyle | 0m 38s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 27m 57s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | | +1 :green_heart: | findbugs | 1m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 39s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 68m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2082/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2082 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux b7b88b4b04a3 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-2082/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 73043507e9 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2082/1/testReport/ | | Max. process+thread count | 269 (vs. ulimit of 1) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2082/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 opened a new pull request #2084: HBASE-22263 Master creates duplicate ServerCrashProcedure on initiali…
cuibo01 opened a new pull request #2084: URL: https://github.com/apache/hbase/pull/2084 …zation, leading to assignment hanging in region-dense clusters This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2083: HBASE-24395 ServerName#getHostname() is case sensitive
Apache-HBase commented on pull request #2083: URL: https://github.com/apache/hbase/pull/2083#issuecomment-660413700 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 4m 50s | Docker failed to build yetus/hbase:de2952b78c. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/2083 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2083/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 opened a new pull request #2083: HBASE-24395 ServerName#getHostname() is case sensitive
cuibo01 opened a new pull request #2083: URL: https://github.com/apache/hbase/pull/2083 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2081: HBASE-24742 Improve performance of SKIP vs SEEK logic.
Apache-HBase commented on pull request #2081: URL: https://github.com/apache/hbase/pull/2081#issuecomment-660411619 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 53s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 4m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 11s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 4m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 134m 26s | hbase-server in the patch passed. | | | | 158m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2081 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ad112d1a7932 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c582fb5d11 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/testReport/ | | Max. process+thread count | 4057 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24746) The sort icons overlap the col name in master UI
[ https://issues.apache.org/jira/browse/HBASE-24746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160313#comment-17160313 ] Zheng Wang commented on HBASE-24746: Thanks. [~stack] > The sort icons overlap the col name in master UI > - > > Key: HBASE-24746 > URL: https://issues.apache.org/jira/browse/HBASE-24746 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > Attachments: HBASE-24746-afterpatch.png, HBASE-24746-beforepatch.png > > > In tables section, when description not few, the sort icons and col name are > overlap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24570) connection#close throws NPE
[
https://issues.apache.org/jira/browse/HBASE-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bo Cui updated HBASE-24570:
---
Status: Patch Available (was: Open)
> connection#close throws NPE
> ---
>
> Key: HBASE-24570
> URL: https://issues.apache.org/jira/browse/HBASE-24570
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.3
>Reporter: Bo Cui
>Assignee: Bo Cui
>Priority: Minor
>
> In the ConnectionImplementation construction method, if getRegistry throws
> exception， registry will be null，and then close will throw NPE
> {code:java}
> try {
> this.registry = AsyncRegistryFactory.getRegistry(conf);
> ...
> } catch (Throwable e) {
> // avoid leaks: registry, rpcClient, ...
> LOG.debug("connection construction failed", e);
> close();
> throw e;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bsglz commented on pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
bsglz commented on pull request #2044: URL: https://github.com/apache/hbase/pull/2044#issuecomment-660410619 The prob is that here we want change the multiplier everyday when the current hour switch between peak and offpeak, there will not a config changed event. Thanks. @virajjasani This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 opened a new pull request #2082: HBASE-24570 connection#close throws NPE
cuibo01 opened a new pull request #2082: URL: https://github.com/apache/hbase/pull/2082 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24664) Some changing of split region by overall region size rather than only one store size
[
https://issues.apache.org/jira/browse/HBASE-24664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160305#comment-17160305
]
Hudson commented on HBASE-24664:
Results for branch branch-2
[build #2749 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Some changing of split region by overall region size rather than only one
> store size
>
>
> Key: HBASE-24664
> URL: https://issues.apache.org/jira/browse/HBASE-24664
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 3.0.0-alpha-1, 2.4.0
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> As a distributed cluster, HBase distribute loads in unit of region, so if
> region grows too big,
> it will bring some negative effects, such as:
> 1. Harder to homogenize disk usage(consider locality)
> 2. Might cost more time on region opening
> 3. After split, the daughter region might lead to more io cost on compaction
> in a short time(if write evenly)
> I tried to introduce a new SteppingAllStoresSizeSplitPolicy in HBASE-24530,
> but after discussed in comments and related
> [thread|https://lists.apache.org/thread.html/r08a8103e2532eb667a0fcb4efa8a4117b3f82e6251bc4bd0bc157c26%40%3Cdev.hbase.apache.org%3E],
> finally we decide to change the existing split policy with a new option that
> if it should count all store files, and for master it would be true, else
> false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24718) Generic NamedQueue framework for recent in-memory history (refactor slowlog)
[
https://issues.apache.org/jira/browse/HBASE-24718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160306#comment-17160306
]
Hudson commented on HBASE-24718:
Results for branch branch-2
[build #2749 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Generic NamedQueue framework for recent in-memory history (refactor slowlog)
>
>
> Key: HBASE-24718
> URL: https://issues.apache.org/jira/browse/HBASE-24718
> Project: HBase
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> As per the discussion on parent jira, we should come up with named queue
> (online ring buffer) to serve recent history for multiple use-cases like
> slowlog, balancer decision, other region activities e.g flush, compaction,
> split, merge etc.
> Since we already have slow/large rpc logs in ring buffer (HBASE-22978), as
> part of this Jira, the proposal is to refactor slowlog provider to get
> generic payload for ring buffer and based on event type (slow_log is the only
> one for now), we can have separate internal in-memory queues.
> After this refactor, it should be relatively simpler to use the same
> framework and create more cases like parent Jira (balancer decision in ring
> buffer).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24746) The sort icons overlap the col name in master UI
[
https://issues.apache.org/jira/browse/HBASE-24746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160307#comment-17160307
]
Hudson commented on HBASE-24746:
Results for branch branch-2
[build #2749 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The sort icons overlap the col name in master UI
> -
>
> Key: HBASE-24746
> URL: https://issues.apache.org/jira/browse/HBASE-24746
> Project: HBase
> Issue Type: Bug
> Components: UI
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0
>
> Attachments: HBASE-24746-afterpatch.png, HBASE-24746-beforepatch.png
>
>
> In tables section, when description not few, the sort icons and col name are
> overlap.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24675) On Master restart all servers are assigned to default rsgroup.
[
https://issues.apache.org/jira/browse/HBASE-24675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160308#comment-17160308
]
Hudson commented on HBASE-24675:
Results for branch branch-2
[build #2749 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> On Master restart all servers are assigned to default rsgroup.
> --
>
> Key: HBASE-24675
> URL: https://issues.apache.org/jira/browse/HBASE-24675
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.3
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0
>
>
> Steps to reproduce:
> # Install a HBase cluster with three RS(rs1,rs2 and rs3) and one Master
> # Create two rsgroups r1 and r2 and move rs1 to r1 and rs2 to r2
> {code:java}
> add_rsgroup 'r1';add_rsgroup 'r2';move_servers_rsgroup
> 'r1',['host1:16020'];move_servers_rsgroup 'r2',['host2:16020']
> {code}
> # Restart Master
> # Run list_rsgroups for hbase shell, all region servers are assigned to
> default regroup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24382) Flush partial stores of region filtered by seqId when archive wal due to too many wals
[
https://issues.apache.org/jira/browse/HBASE-24382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160304#comment-17160304
]
Hudson commented on HBASE-24382:
Results for branch branch-2
[build #2749 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2749/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Flush partial stores of region filtered by seqId when archive wal due to too
> many wals
> --
>
> Key: HBASE-24382
> URL: https://issues.apache.org/jira/browse/HBASE-24382
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-1, 2.4.0
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> When the logRoller archive the oldest wal due to too many wals, if a region
> should be flushed, we flush all stores of it, but it is not necessary, maybe
> we can use unflushedSeqId of store to filter them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160292#comment-17160292 ] Mingliang Liu commented on HBASE-24692: --- Yes, that problem is also seen in my Chrome and Safari on macOS...Have no clue so far how that fix could be since it comes from the bootstrap CSS. > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png, > 24692-ex4.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2081: HBASE-24742 Improve performance of SKIP vs SEEK logic.
Apache-HBase commented on pull request #2081: URL: https://github.com/apache/hbase/pull/2081#issuecomment-660395593 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 15s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 4s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | the patch passed | | -0 :warning: | checkstyle | 1m 13s | hbase-server: The patch generated 1 new + 21 unchanged - 0 fixed = 22 total (was 21) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 31s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 42m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2081 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux be47be3cbaa4 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / c582fb5d11 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2081/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2014: HBASE-24673 TransitionRegionStateProcedure of non-meta regions should…
ndimiduk commented on a change in pull request #2014:
URL: https://github.com/apache/hbase/pull/2014#discussion_r456725385
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
##
@@ -200,14 +200,21 @@ private void queueAssign(MasterProcedureEnv env,
RegionStateNode regionNode)
}
}
- private void openRegion(MasterProcedureEnv env, RegionStateNode regionNode)
throws IOException {
+ private void openRegion(MasterProcedureEnv env, RegionStateNode regionNode)
+throws IOException, ProcedureSuspendedException {
ServerName loc = regionNode.getRegionLocation();
if (loc == null) {
LOG.warn("No location specified for {}, jump back to state {} to get
one", getRegion(),
RegionStateTransitionState.REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE);
setNextState(RegionStateTransitionState.REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE);
return;
}
+final boolean isMeta = regionNode.getRegionInfo().isMetaRegion();
+final boolean isMetaAvailable =
!env.getAssignmentManager().isMetaRegionInTransition();
+if (!isMeta && !isMetaAvailable) {
+ // meta is not assigned yet, so yield
+ throw new ProcedureSuspendedException();
Review comment:
I agree, neither this code nor the proposed change provide a
satisfactory solution. However, I think if we can avoid turning the TRSP over
to the worker thread when we know meta is unavailable, we reduce the window of
risk. Another change be to alter the configuration of the client that attempts
the meta update: instead of retrying for the full loop, it might try a single
RPC and if that fails yield the procedure. That way the PEWorker isn't occupied
by the default client retry logic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2014: HBASE-24673 TransitionRegionStateProcedure of non-meta regions should…
ndimiduk commented on a change in pull request #2014:
URL: https://github.com/apache/hbase/pull/2014#discussion_r456725385
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
##
@@ -200,14 +200,21 @@ private void queueAssign(MasterProcedureEnv env,
RegionStateNode regionNode)
}
}
- private void openRegion(MasterProcedureEnv env, RegionStateNode regionNode)
throws IOException {
+ private void openRegion(MasterProcedureEnv env, RegionStateNode regionNode)
+throws IOException, ProcedureSuspendedException {
ServerName loc = regionNode.getRegionLocation();
if (loc == null) {
LOG.warn("No location specified for {}, jump back to state {} to get
one", getRegion(),
RegionStateTransitionState.REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE);
setNextState(RegionStateTransitionState.REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE);
return;
}
+final boolean isMeta = regionNode.getRegionInfo().isMetaRegion();
+final boolean isMetaAvailable =
!env.getAssignmentManager().isMetaRegionInTransition();
+if (!isMeta && !isMetaAvailable) {
+ // meta is not assigned yet, so yield
+ throw new ProcedureSuspendedException();
Review comment:
I agree, neither this code nor the proposed change provide a
satisfactory solution. However, I think if we can avoid turning the TRSP over
to the worker thread when we know meta is unavailable, we reduce the window of
risk. Another change might alter the configuration of the client that attempts
the meta update: instead of retrying for the full loop, it would try a single
RPC and if that fails yield the procedure. That way the PEWorker isn't occupied
by the default client retry logic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2014: HBASE-24673 TransitionRegionStateProcedure of non-meta regions should…
ndimiduk commented on a change in pull request #2014:
URL: https://github.com/apache/hbase/pull/2014#discussion_r456725385
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
##
@@ -200,14 +200,21 @@ private void queueAssign(MasterProcedureEnv env,
RegionStateNode regionNode)
}
}
- private void openRegion(MasterProcedureEnv env, RegionStateNode regionNode)
throws IOException {
+ private void openRegion(MasterProcedureEnv env, RegionStateNode regionNode)
+throws IOException, ProcedureSuspendedException {
ServerName loc = regionNode.getRegionLocation();
if (loc == null) {
LOG.warn("No location specified for {}, jump back to state {} to get
one", getRegion(),
RegionStateTransitionState.REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE);
setNextState(RegionStateTransitionState.REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE);
return;
}
+final boolean isMeta = regionNode.getRegionInfo().isMetaRegion();
+final boolean isMetaAvailable =
!env.getAssignmentManager().isMetaRegionInTransition();
+if (!isMeta && !isMetaAvailable) {
+ // meta is not assigned yet, so yield
+ throw new ProcedureSuspendedException();
Review comment:
I agree, neither this code nor the proposed change provide a
satisfactory solution. However, I think if we can avoid turning the SRSP over
to the worker thread when we know meta is unavailable, we reduce the window of
risk. Another change be to alter the configuration of the client that attempts
the meta update: instead of retrying for the full loop, it might try a single
RPC and if that fails yield the procedure. That way the PEWorker isn't occupied
by the default client retry logic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-20904) Prometheus /metrics http endpoint for monitoring integration
[ https://issues.apache.org/jira/browse/HBASE-20904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160287#comment-17160287 ] Nick Dimiduk commented on HBASE-20904: -- I'm not keen on copying more metrics code from Hadoop in to HBase. I'm also not keen on expanding our codebase when perfectly good alternatives exist that are supported by the project with which we're trying to integrate. I'm not saying -0, I'm just asking for strong justification. It looks like there's someone [actively working|https://github.com/prometheus/jmx_exporter/pull/459] on making Prometheus/HBase integration work better. Why not team up with them to make the agent more robust for our users? Also, isn't the Slider project [retired|https://incubator.apache.org/projects/slider.html]? bq. we have to dynamically decide the port on which agent listens on as two server processes (M / RS) can be deployed on a single host (by slider/yarn). This becomes a burden for the slider user as agent doesn't provide this funcationality and one cannot choose some random number <64k. Having the agent support binding to an arbitrary port and advertising it back to an operator seems like a feature the agent should have, file a feature request? I haven't used slider so I don't know what limitations are present on that platform, but any host running multiple JVMs with this agent installed will have a similar need, so seems like an upstream problem they want to solve anyway. bq. Well we can use agent+pushgateways combination but the metrics are published all the time which may be become unnecessary given the size of metrics published (prometheus expo format is verbose). I don't quite follow you here... I see the agent supports allow/deny lists for metrics, running as an agent or the scraping gateway. Does this not work to limit the data transferred? To implement this as a servlet, we would be generating all the metrics (as we do now, with JSON), which ends up transferring all the data as well. bq. these dynamically chosen ports have to be (discovered by) / (told to) prometheus scraper. This is again a burden for the slider user as the HBase can be run as a Again, I'm not sure about Slider, but service discovery seems like a problem that Slider needs to solve in general, so some interface between that mechanism in Slider and the dynamic port binding improvement in the agent would be required. bq. Some customers are skeptical about opening new ports for an agent. I believe the intended deployment model of the agent to bind to localhost. A prometheus scraper agent, also running on localhost, can query the data. In that case, there's no opening a port to the network, just a local socket between processes. > Prometheus /metrics http endpoint for monitoring integration > > > Key: HBASE-20904 > URL: https://issues.apache.org/jira/browse/HBASE-20904 > Project: HBase > Issue Type: New Feature > Components: metrics, monitoring >Reporter: Hari Sekhon >Assignee: Madhusoodan >Priority: Major > > Feature Request to add Prometheus /metrics http endpoint for monitoring > integration: > [https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cscrape_config%3E] > Prometheus metrics format for that endpoint: > [https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160277#comment-17160277 ] Nick Dimiduk commented on HBASE-24692: -- Oh I see. Did you try it with a width that's too narrow for the full navigation bar, but too wide to have the drop-down icon show up, like in ex-1? I wonder if this is a Chrome vs Safari rendering difference. Thanks for looking [~liuml07]. > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png, > 24692-ex4.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24742) Improve performance of SKIP vs SEEK logic
[ https://issues.apache.org/jira/browse/HBASE-24742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-24742: -- Reopen for 2.3 backport, https://github.com/apache/hbase/pull/2081 > Improve performance of SKIP vs SEEK logic > - > > Key: HBASE-24742 > URL: https://issues.apache.org/jira/browse/HBASE-24742 > Project: HBase > Issue Type: Bug > Components: Performance, regionserver >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.4.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0, 2.1.10, 2.2.6 > > Attachments: 24742-master.txt, hbase-1.6-regression-flame-graph.png, > hbase-24742-branch-1.txt > > > In our testing of HBase 1.3 against the current tip of branch-1 we saw a 30% > slowdown in scanning scenarios. > We tracked it back to HBASE-17958 and HBASE-19863. > Both add comparisons to one of the tightest HBase has. > [~bharathv] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2081: HBASE-24742 Improve performance of SKIP vs SEEK logic.
ndimiduk opened a new pull request #2081: URL: https://github.com/apache/hbase/pull/2081 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160274#comment-17160274 ] Mingliang Liu edited comment on HBASE-24692 at 7/17/20, 11:55 PM: -- [~ndimiduk] I had a look at this, but only find that the navigation bar (or header bar as you referred to) collapses. So if you click the icon on top right, you will see the navigation bar shows up. I guess this was designed for narrow screens. was (Author: liuml07): [~ndimiduk] I had a look at this, but only find that the navigation bar (or header bar as you referred to) collapses. So if you click the icon on top right, you will see the navigation bar shows up. I guess this was designed for narrow screens. !24692-ex4.png! > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png, > 24692-ex4.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HBASE-24692: -- Attachment: 24692-ex4.png > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png, > 24692-ex4.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160274#comment-17160274 ] Mingliang Liu edited comment on HBASE-24692 at 7/17/20, 11:54 PM: -- [~ndimiduk] I had a look at this, but only find that the navigation bar (or header bar as you referred to) collapses. So if you click the icon on top right, you will see the navigation bar shows up. I guess this was designed for narrow screens. !24692-ex4.png! was (Author: liuml07): [~ndimiduk] I had a look at this, but only find that the navigation bar (or header bar as you referred to) collapses. So if you click the icon on top right, you will see the navigation bar shows up. I guess this was designed for narrow screens. > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png, > 24692-ex4.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160274#comment-17160274 ] Mingliang Liu commented on HBASE-24692: --- [~ndimiduk] I had a look at this, but only find that the navigation bar (or header bar as you referred to) collapses. So if you click the icon on top right, you will see the navigation bar shows up. I guess this was designed for narrow screens. > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24692) WebUI header bar overlaps page content when window is too narrow
[ https://issues.apache.org/jira/browse/HBASE-24692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HBASE-24692: - Assignee: Mingliang Liu > WebUI header bar overlaps page content when window is too narrow > > > Key: HBASE-24692 > URL: https://issues.apache.org/jira/browse/HBASE-24692 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Mingliang Liu >Priority: Minor > Attachments: 24692-ex1.png, 24692-ex2.png, 24692-ex3.png > > > It seems the CSS on our WebUI is such that the header will expand down > vertically as the content wraps dynamically. However, the page content does > not shift down along with it, resulting in the header overlapping the page > content. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24698) Turn OFF Canary WebUI as default
[ https://issues.apache.org/jira/browse/HBASE-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24698: -- Description: See parent issue. There is a CLASSPATH issue when running against hadoop3 that needs resolving. Meantime, the canary fails to run with a cryptic message (mentioned in the parent in full but it is 'Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.shaded.org.eclipse.jetty.server.HttpConfiguration$Customizer' in short). This will surprise operators who are used to the canary's previous pure-cli experience. Let me make it so you ask for the canary webui; by default, it does not come up. (was: See parent issue. There is a CLASSPATH issue when running against hadoop3 that needs resolving. Meantime, the canary fails to run with a cryptic message. This will surprise operators. Let me make it so you ask for the canary webui; by default, it does not come up.) > Turn OFF Canary WebUI as default > > > Key: HBASE-24698 > URL: https://issues.apache.org/jira/browse/HBASE-24698 > Project: HBase > Issue Type: Sub-task > Components: canary >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > > See parent issue. There is a CLASSPATH issue when running against hadoop3 > that needs resolving. Meantime, the canary fails to run with a cryptic > message (mentioned in the parent in full but it is 'Caused by: > java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.shaded.org.eclipse.jetty.server.HttpConfiguration$Customizer' > in short). This will surprise operators who are used to the canary's > previous pure-cli experience. Let me make it so you ask for the canary webui; > by default, it does not come up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24698) Turn OFF Canary WebUI as default
[ https://issues.apache.org/jira/browse/HBASE-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24698. --- Hadoop Flags: Reviewed Release Note: Flips default for 'HBASE-23994 Add WebUI to Canary' The UI defaulted to on at port 16050. This JIRA changes it so new UI is off by default. To enable the UI, set property 'hbase.canary.info.port' to the port you want the UI to use. Resolution: Fixed Merged to branch-2.3+. Thanks for reviews [~ndimiduk] and [~zhangduo] > Turn OFF Canary WebUI as default > > > Key: HBASE-24698 > URL: https://issues.apache.org/jira/browse/HBASE-24698 > Project: HBase > Issue Type: Sub-task > Components: canary >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > > See parent issue. There is a CLASSPATH issue when running against hadoop3 > that needs resolving. Meantime, the canary fails to run with a cryptic > message. This will surprise operators. Let me make it so you ask for the > canary webui; by default, it does not come up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2080: HBASE-24722 Update commands with unintentional return values (#2058)
saintstack merged pull request #2080: URL: https://github.com/apache/hbase/pull/2080 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24742) Improve performance of SKIP vs SEEK logic
[
https://issues.apache.org/jira/browse/HBASE-24742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160250#comment-17160250
]
Hudson commented on HBASE-24742:
Results for branch branch-2.2
[build #915 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/915/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/915//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/915//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/915//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve performance of SKIP vs SEEK logic
> -
>
> Key: HBASE-24742
> URL: https://issues.apache.org/jira/browse/HBASE-24742
> Project: HBase
> Issue Type: Bug
> Components: Performance, regionserver
>Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.4.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0, 2.1.10, 2.2.6
>
> Attachments: 24742-master.txt, hbase-1.6-regression-flame-graph.png,
> hbase-24742-branch-1.txt
>
>
> In our testing of HBase 1.3 against the current tip of branch-1 we saw a 30%
> slowdown in scanning scenarios.
> We tracked it back to HBASE-17958 and HBASE-19863.
> Both add comparisons to one of the tightest HBase has.
> [~bharathv]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2037: HBASE-24698 Turn OFF Canary WebUI as default
saintstack commented on pull request #2037: URL: https://github.com/apache/hbase/pull/2037#issuecomment-660357609 Merged this. Shout if you not ok @Apache9 ... did 1/2 of what you asked ... other define not used. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2037: HBASE-24698 Turn OFF Canary WebUI as default
saintstack merged pull request #2037: URL: https://github.com/apache/hbase/pull/2037 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24746) The sort icons overlap the col name in master UI
[ https://issues.apache.org/jira/browse/HBASE-24746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160232#comment-17160232 ] Michael Stack commented on HBASE-24746: --- Very nice [~filtertip] > The sort icons overlap the col name in master UI > - > > Key: HBASE-24746 > URL: https://issues.apache.org/jira/browse/HBASE-24746 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > Attachments: HBASE-24746-afterpatch.png, HBASE-24746-beforepatch.png > > > In tables section, when description not few, the sort icons and col name are > overlap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24745) 'Failed report transition' logs too often
[
https://issues.apache.org/jira/browse/HBASE-24745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160231#comment-17160231
]
Michael Stack commented on HBASE-24745:
---
What do you see in your unit test [~wenfeiyi666] ? Thanks.
> 'Failed report transition' logs too often
> -
>
> Key: HBASE-24745
> URL: https://issues.apache.org/jira/browse/HBASE-24745
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Assignee: wenfeiyi666
>Priority: Minor
>
> The parent issue fixed a backoff that was too aggressive. Now I notice we try
> too much. Saw 9k logs in 17 seconds of the below type...
> {code:java}
> 2020-07-15 14:36:23,104 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Failed report transition
> server { host_name: "X.example.org" port: 16020 start_code: 1594823099666 }
> transition { transition_ code: CLOSED region_info { region_id:
> 1594814749475 table_name { namespace: "default" qualifier:
> "IntegrationTestBigLinkedList" } start_key: "\"\"\"\"\"\"\" " end_key:
> "#Q\352\f\003" offline: false split: false replica_id: 0 } proc_id:
> 81545 }; retry (#) after 200805ms delay (Master is coming online...).
> {code}
> The delay doesn't seem correct or respected.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24745) 'Failed report transition' logs too often
[
https://issues.apache.org/jira/browse/HBASE-24745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160227#comment-17160227
]
Michael Stack commented on HBASE-24745:
---
Thank you [~wenfeiyi666] for taking a look. I don't really have much beyond the
above At the time the Master is changing... Killed by ITBLL test and a new
one coming on line.
> 'Failed report transition' logs too often
> -
>
> Key: HBASE-24745
> URL: https://issues.apache.org/jira/browse/HBASE-24745
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Assignee: wenfeiyi666
>Priority: Minor
>
> The parent issue fixed a backoff that was too aggressive. Now I notice we try
> too much. Saw 9k logs in 17 seconds of the below type...
> {code:java}
> 2020-07-15 14:36:23,104 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Failed report transition
> server { host_name: "X.example.org" port: 16020 start_code: 1594823099666 }
> transition { transition_ code: CLOSED region_info { region_id:
> 1594814749475 table_name { namespace: "default" qualifier:
> "IntegrationTestBigLinkedList" } start_key: "\"\"\"\"\"\"\" " end_key:
> "#Q\352\f\003" offline: false split: false replica_id: 0 } proc_id:
> 81545 }; retry (#) after 200805ms delay (Master is coming online...).
> {code}
> The delay doesn't seem correct or respected.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
Apache-HBase commented on pull request #2079: URL: https://github.com/apache/hbase/pull/2079#issuecomment-660339511 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 203m 49s | hbase-server in the patch passed. | | | | 229m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2079 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4a771f0178f3 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1781185b7b | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/testReport/ | | Max. process+thread count | 3652 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24740) Enable journal logging for HBase snapshot operation
[ https://issues.apache.org/jira/browse/HBASE-24740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160195#comment-17160195 ] Bharath Vissapragada commented on HBASE-24740: -- Committed to branch-1, Sandeep is working on forward porting to master/branch-2 since there are a few conflicts. > Enable journal logging for HBase snapshot operation > --- > > Key: HBASE-24740 > URL: https://issues.apache.org/jira/browse/HBASE-24740 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.2.7 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Minor > Labels: observability > Fix For: 1.7.0 > > > The HBase snapshot operation contains multiple steps like actual snapshot > creation, consolidate phase (reading region manifests from HDFS) , verifier > phase ( validate the consolidated manifests against the actual number of > regions for the table) > > Sometimes it happens to be taking time in one of the phases and we don't know > exactly which one is taking time unless we have a thread dump at the very > same time. The journal logging would definitely help us give more insights > into the time taken for each phase -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24740) Enable journal logging for HBase snapshot operation
[ https://issues.apache.org/jira/browse/HBASE-24740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-24740: - Component/s: master > Enable journal logging for HBase snapshot operation > --- > > Key: HBASE-24740 > URL: https://issues.apache.org/jira/browse/HBASE-24740 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.2.7 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Minor > > The HBase snapshot operation contains multiple steps like actual snapshot > creation, consolidate phase (reading region manifests from HDFS) , verifier > phase ( validate the consolidated manifests against the actual number of > regions for the table) > > Sometimes it happens to be taking time in one of the phases and we don't know > exactly which one is taking time unless we have a thread dump at the very > same time. The journal logging would definitely help us give more insights > into the time taken for each phase -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24740) Enable journal logging for HBase snapshot operation
[ https://issues.apache.org/jira/browse/HBASE-24740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-24740: - Fix Version/s: 1.7.0 > Enable journal logging for HBase snapshot operation > --- > > Key: HBASE-24740 > URL: https://issues.apache.org/jira/browse/HBASE-24740 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.2.7 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Minor > Labels: observability > Fix For: 1.7.0 > > > The HBase snapshot operation contains multiple steps like actual snapshot > creation, consolidate phase (reading region manifests from HDFS) , verifier > phase ( validate the consolidated manifests against the actual number of > regions for the table) > > Sometimes it happens to be taking time in one of the phases and we don't know > exactly which one is taking time unless we have a thread dump at the very > same time. The journal logging would definitely help us give more insights > into the time taken for each phase -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24740) Enable journal logging for HBase snapshot operation
[ https://issues.apache.org/jira/browse/HBASE-24740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-24740: - Labels: observability (was: ) > Enable journal logging for HBase snapshot operation > --- > > Key: HBASE-24740 > URL: https://issues.apache.org/jira/browse/HBASE-24740 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.2.7 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Minor > Labels: observability > > The HBase snapshot operation contains multiple steps like actual snapshot > creation, consolidate phase (reading region manifests from HDFS) , verifier > phase ( validate the consolidated manifests against the actual number of > regions for the table) > > Sometimes it happens to be taking time in one of the phases and we don't know > exactly which one is taking time unless we have a thread dump at the very > same time. The journal logging would definitely help us give more insights > into the time taken for each phase -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24740) Enable journal logging for HBase snapshot operation
[ https://issues.apache.org/jira/browse/HBASE-24740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada updated HBASE-24740: - Affects Version/s: 2.2.7 1.7.0 3.0.0-alpha-1 > Enable journal logging for HBase snapshot operation > --- > > Key: HBASE-24740 > URL: https://issues.apache.org/jira/browse/HBASE-24740 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.2.7 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Minor > > The HBase snapshot operation contains multiple steps like actual snapshot > creation, consolidate phase (reading region manifests from HDFS) , verifier > phase ( validate the consolidated manifests against the actual number of > regions for the table) > > Sometimes it happens to be taking time in one of the phases and we don't know > exactly which one is taking time unless we have a thread dump at the very > same time. The journal logging would definitely help us give more insights > into the time taken for each phase -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bharathv merged pull request #2076: HBASE-24740 Enable journal logging for HBase snapshot operation
bharathv merged pull request #2076: URL: https://github.com/apache/hbase/pull/2076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2076: HBASE-24740 Enable journal logging for HBase snapshot operation
Apache-HBase commented on pull request #2076: URL: https://github.com/apache/hbase/pull/2076#issuecomment-660312944 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 43s | branch-1 passed | | +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | compile | 0m 44s | branch-1 passed with JDK v1.7.0_262 | | +1 :green_heart: | checkstyle | 1m 40s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 42s | branch-1 passed with JDK v1.7.0_262 | | +0 :ok: | spotbugs | 2m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 55s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 54s | the patch passed | | +1 :green_heart: | compile | 0m 43s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javac | 0m 43s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | checkstyle | 1m 29s | hbase-server: The patch generated 0 new + 20 unchanged - 10 fixed = 20 total (was 30) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 47s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 34s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | findbugs | 2m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 132m 0s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 173m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2076/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2076 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 76406a1354d4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-2076/out/precommit/personality/provided.sh | | git revision | branch-1 / 37903b0 | | Default Java | 1.7.0_262 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2076/3/testReport/ | | Max. process+thread count | 4280 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2076/3/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
Apache-HBase commented on pull request #2079: URL: https://github.com/apache/hbase/pull/2079#issuecomment-660310333 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 25s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 5m 48s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 5m 43s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 130m 56s | hbase-server in the patch passed. | | | | 157m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2079 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 15f5f50b0eec 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1781185b7b | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/testReport/ | | Max. process+thread count | 4464 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2080: HBASE-24722 Update commands with unintentional return values (#2058)
Apache-HBase commented on pull request #2080: URL: https://github.com/apache/hbase/pull/2080#issuecomment-660294213 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1935: HBASE-22146 SpaceQuotaViolationPolicy Disable is not working in Names…
Apache-HBase commented on pull request #1935: URL: https://github.com/apache/hbase/pull/1935#issuecomment-660290461 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | +1 :green_heart: | shadedjars | 6m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 44s | the patch passed | | +1 :green_heart: | compile | 1m 22s | the patch passed | | +1 :green_heart: | javac | 1m 22s | the patch passed | | +1 :green_heart: | shadedjars | 7m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 27s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 202m 55s | hbase-server in the patch passed. | | | | 234m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1935 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ec9de4fad3cc 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 16f306b4a1 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/testReport/ | | Max. process+thread count | 3575 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1935: HBASE-22146 SpaceQuotaViolationPolicy Disable is not working in Names…
Apache-HBase commented on pull request #1935: URL: https://github.com/apache/hbase/pull/1935#issuecomment-660290158 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 1m 37s | master passed | | +1 :green_heart: | shadedjars | 6m 20s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 26s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 42s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 33s | the patch passed | | +1 :green_heart: | compile | 1m 38s | the patch passed | | +1 :green_heart: | javac | 1m 38s | the patch passed | | +1 :green_heart: | shadedjars | 6m 21s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 25s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 25s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 199m 59s | hbase-server in the patch passed. | | | | 234m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1935 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9de714d8fa26 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 16f306b4a1 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/testReport/ | | Max. process+thread count | 3449 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2080: HBASE-24722 Update commands with unintentional return values (#2058)
Apache-HBase commented on pull request #2080: URL: https://github.com/apache/hbase/pull/2080#issuecomment-660288217 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 48s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | ||| _ Patch Compile Tests _ | | -0 :warning: | rubocop | 1m 11s | The patch generated 26 new + 276 unchanged - 6 fixed = 302 total (was 282) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 4m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2080/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2080 | | Optional Tests | dupname asflicense rubocop | | uname | Linux 726636370c55 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e08da3bf26 | | rubocop | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2080/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 42 (vs. ulimit of 12500) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2080/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24722) Address hbase-shell commands with unintentional return values
[ https://issues.apache.org/jira/browse/HBASE-24722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160144#comment-17160144 ] Michael Stack commented on HBASE-24722: --- Merged to master. Put up PR for backport to branch-2. > Address hbase-shell commands with unintentional return values > - > > Key: HBASE-24722 > URL: https://issues.apache.org/jira/browse/HBASE-24722 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.3.0 >Reporter: Elliot Miller >Assignee: Elliot Miller >Priority: Minor > > There are a number of commands in hbase-shell that return values that are > undocumented or otherwise do not makes sense. > The commands that will be addressed by this ticket were found in a survey of > all the hbase shell commands, > https://issues.apache.org/jira/browse/HBASE-24672. > Subtasks: > * Fix commands that return the number of printed lines (usually the number > 1). > ** Affected commands (culprits): is_disabled, balancer, normalize, > normalizer_switch, normalizer_enabled, catalogjanitor_switch, > catalogjanitor_enabled, cleaner_chore_switch, cleaner_chore_enabled, > splitormerge_switch, splitormerge_enabled, clear_deadservers, > clear_block_cache > ** I am updating these 13 commands to either return a value that makes sense > or return nil if a sensible return value doesn't exist. > * Fix commands that return true or false as a String > ** Affected commands (culprits): balance_switch, snapshot_cleanup_switch, > enable_rpc_throttle, disable_rpc_throttle, enable_exceed_throttle_quota, > disable_exceed_throttle_quota > ** I am updating these 6 commands to return ruby's TrueClass or FalseClass > rather than a String -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #2080: HBASE-24722 Update commands with unintentional return values (#2058)
saintstack opened a new pull request #2080: URL: https://github.com/apache/hbase/pull/2080 - Prior to this commit, there were 13 commands that unintentionally return the number of lines they print (usually one). This commit ensures that they return the value documented by the help text, or nil if there is not a simple logical value to return. - Fixes 6 hbase-shell commands that return String rather than TrueClass or FalseClass - Use double-bang to cast truthy values to TrueClass and FalseClass so that ruby's to_s can reliably print true or false without using ternary operators - Updates tests for is_disabled, is_enabled, disable_rpc_throttle, enable_rpc_throttle, disable_exceed_throttle_quota, enable_exceed_throttle_quota, clear_deadservers, snapshot_cleanup_switch, snapshot_cleanup_enabled, and balancer to check return values - Adds new tests for balance_switch, balancer_enabled, normalizer_switch, normalizer_enabled, catalog_janitor_switch, catalogjanitor_enabled, cleaner_chore_switch, cleaner_chore_enabled, splitormerge_switch, and splitormerge_enabled signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bitoffdev commented on a change in pull request #2058: HBASE-24722 Address hbase-shell commands with unintentional return values
bitoffdev commented on a change in pull request #2058:
URL: https://github.com/apache/hbase/pull/2058#discussion_r456618217
##
File path: hbase-shell/src/main/ruby/shell/commands/clear_deadservers.rb
##
@@ -35,18 +37,20 @@ def help
end
# rubocop:disable Metrics/AbcSize
- # rubocop:disable Metrics/MethodLength
def command(*dead_servers)
servers = admin.clear_deadservers(dead_servers)
if servers.size <= 0
formatter.row(['true'])
+ []
else
formatter.row(['Some dead server clear failed'])
formatter.row(['SERVERNAME'])
- servers.each do |server|
-formatter.row([server.toString])
+ server_names = servers.map { |server| server.toString }
Review comment:
This is just a change to collect all the server names as an array so
that we can both print and return the list of servers that could not be cleared
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bitoffdev commented on a change in pull request #2058: HBASE-24722 Address hbase-shell commands with unintentional return values
bitoffdev commented on a change in pull request #2058: URL: https://github.com/apache/hbase/pull/2058#discussion_r456617522 ## File path: hbase-shell/src/main/ruby/shell/commands/clear_block_cache.rb ## @@ -33,6 +33,7 @@ def help def command(table_name) formatter.row([admin.clear_block_cache(table_name)]) +nil Review comment: formatter.row returns the running total number of rows printed (`@row_count`) since the last call to formatter.header This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2058: HBASE-24722 Address hbase-shell commands with unintentional return values
saintstack merged pull request #2058: URL: https://github.com/apache/hbase/pull/2058 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2058: HBASE-24722 Address hbase-shell commands with unintentional return values
saintstack commented on a change in pull request #2058:
URL: https://github.com/apache/hbase/pull/2058#discussion_r456614056
##
File path: hbase-shell/src/test/ruby/hbase/admin_test.rb
##
@@ -212,13 +215,182 @@ def teardown
#---
define_test 'snapshot auto cleanup should work' do
- command(:snapshot_cleanup_switch, true)
- output = capture_stdout { command(:snapshot_cleanup_enabled) }
+ result = nil
+ command(:snapshot_cleanup_switch, false)
+
+ # enable snapshot cleanup and check that the previous state is returned
+ output = capture_stdout { result = command(:snapshot_cleanup_switch,
true) }
+ assert(output.include?('false'))
+ assert(result == false)
+
+ # check that snapshot_cleanup_enabled returns the current state
+ output = capture_stdout { result = command(:snapshot_cleanup_enabled) }
assert(output.include?('true'))
+ assert(result == true)
- command(:snapshot_cleanup_switch, false)
- output = capture_stdout { command(:snapshot_cleanup_enabled) }
+ # disable snapshot cleanup and check that the previous state is returned
+ output = capture_stdout { result = command(:snapshot_cleanup_switch,
false) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # check that snapshot_cleanup_enabled returns the current state
+ output = capture_stdout { result = command(:snapshot_cleanup_enabled) }
+ assert(output.include?('false'))
+ assert(result == false)
+end
+
+
#---
+
+define_test 'balancer switch should work' do
+ result = nil
+ command(:balance_switch, false)
+
+ # enable balancer and check that the previous state is returned
+ output = capture_stdout { result = command(:balance_switch, true) }
+ assert(output.include?('false'))
+ assert(result == false)
+
+ # check that balancer_enabled returns the current state
+ output = capture_stdout { result = command(:balancer_enabled) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # disable balancer and check that the previous state is returned
+ output = capture_stdout { result = command(:balance_switch, false) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # check that balancer_enabled returns the current state
+ output = capture_stdout { result = command(:balancer_enabled) }
assert(output.include?('false'))
+ assert(result == false)
+end
+
+
#---
+
+define_test 'normalizer switch should work' do
+ result = nil
+ command(:normalizer_switch, false)
+
+ # enable normalizer and check that the previous state is returned
+ output = capture_stdout { result = command(:normalizer_switch, true) }
+ assert(output.include?('false'))
+ assert(result == false)
+
+ # check that normalizer_enabled returns the current state
+ output = capture_stdout { result = command(:normalizer_enabled) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # disable normalizer and check that the previous state is returned
+ output = capture_stdout { result = command(:normalizer_switch, false) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # check that normalizer_enabled returns the current state
+ output = capture_stdout { result = command(:normalizer_enabled) }
+ assert(output.include?('false'))
+ assert(result == false)
+end
+
+
#---
+
+define_test 'catalogjanitor switch should work' do
+ result = nil
+ command(:catalogjanitor_switch, false)
+
+ # enable catalogjanitor and check that the previous state is returned
+ output = capture_stdout { result = command(:catalogjanitor_switch, true)
}
+ assert(output.include?('false'))
+ assert(result == false)
+
+ # check that catalogjanitor_enabled returns the current state
+ output = capture_stdout { result = command(:catalogjanitor_enabled) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # disable catalogjanitor and check that the previous state is returned
+ output = capture_stdout { result = command(:catalogjanitor_switch,
false) }
+ assert(output.include?('true'))
+ assert(result == true)
+
+ # check that catalogjanitor_enabled returns the current state
+ output = capture_stdout { result = command(:catalogjanitor_enabled) }
+ assert(output.include?('false'))
+ assert(result == false)
+end
+
+
#---
+
+define_test
[GitHub] [hbase] Apache-HBase commented on pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
Apache-HBase commented on pull request #2079: URL: https://github.com/apache/hbase/pull/2079#issuecomment-660256316 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | checkstyle | 1m 13s | master passed | | +1 :green_heart: | spotbugs | 2m 7s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | +1 :green_heart: | checkstyle | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 36m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2079 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 665b2603f67a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1781185b7b | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2079/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil merged pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
wchevreuil merged pull request #2079: URL: https://github.com/apache/hbase/pull/2079 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
wchevreuil commented on a change in pull request #2079:
URL: https://github.com/apache/hbase/pull/2079#discussion_r456587095
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcConnection.java
##
@@ -348,8 +348,14 @@ public void saslReadAndProcess(ByteBuff saslToken) throws
IOException,
byte[] replyToken;
try {
if (saslServer == null) {
- saslServer =
+ try {
+saslServer =
new HBaseSaslRpcServer(provider, rpcServer.saslProps,
rpcServer.secretManager);
+ } catch (Exception e){
Review comment:
Yeah, the goal here is to catch any uncaught exception that may be
thrown by faulty custom providers.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-24675) On Master restart all servers are assigned to default rsgroup.
[
https://issues.apache.org/jira/browse/HBASE-24675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani resolved HBASE-24675.
--
Fix Version/s: 2.4.0
2.3.1
3.0.0-alpha-1
Hadoop Flags: Reviewed
Resolution: Fixed
> On Master restart all servers are assigned to default rsgroup.
> --
>
> Key: HBASE-24675
> URL: https://issues.apache.org/jira/browse/HBASE-24675
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.3
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0
>
>
> Steps to reproduce:
> # Install a HBase cluster with three RS(rs1,rs2 and rs3) and one Master
> # Create two rsgroups r1 and r2 and move rs1 to r1 and rs2 to r2
> {code:java}
> add_rsgroup 'r1';add_rsgroup 'r2';move_servers_rsgroup
> 'r1',['host1:16020'];move_servers_rsgroup 'r2',['host2:16020']
> {code}
> # Restart Master
> # Run list_rsgroups for hbase shell, all region servers are assigned to
> default regroup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24747) Log an ERROR if HBaseSaslRpcServer initialisation fails with an uncaught exception
Wellington Chevreuil created HBASE-24747: Summary: Log an ERROR if HBaseSaslRpcServer initialisation fails with an uncaught exception Key: HBASE-24747 URL: https://issues.apache.org/jira/browse/HBASE-24747 Project: HBase Issue Type: Improvement Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil In ServerRpcConnection.saslReadAndProcess, if an uncaught exception happens while initialising HBaseSaslRpcServer, the same gets swallowed inside netty and never explicitly logged on RegionServer logs. With the introduction of pluggable sasl server providers by HBASE-23347, this situation can become more frequent with the addition of custom providers, if those cause HBaseSaslRpcServer to fail, that should be logged in ServerRpcConnection.saslReadAndProces. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
virajjasani commented on a change in pull request #2079:
URL: https://github.com/apache/hbase/pull/2079#discussion_r456575593
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcConnection.java
##
@@ -348,8 +348,14 @@ public void saslReadAndProcess(ByteBuff saslToken) throws
IOException,
byte[] replyToken;
try {
if (saslServer == null) {
- saslServer =
+ try {
+saslServer =
new HBaseSaslRpcServer(provider, rpcServer.saslProps,
rpcServer.secretManager);
+ } catch (Exception e){
Review comment:
nit: this can be IOException? Exception is also fine though since the
purpose is to log any uncaught exceptions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-24675) On Master restart all servers are assigned to default rsgroup.
[
https://issues.apache.org/jira/browse/HBASE-24675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160103#comment-17160103
]
Viraj Jasani commented on HBASE-24675:
--
Does this apply to branch-1 by any chance? If it does, can you raise a PR?
Thanks
> On Master restart all servers are assigned to default rsgroup.
> --
>
> Key: HBASE-24675
> URL: https://issues.apache.org/jira/browse/HBASE-24675
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.3
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0
>
>
> Steps to reproduce:
> # Install a HBase cluster with three RS(rs1,rs2 and rs3) and one Master
> # Create two rsgroups r1 and r2 and move rs1 to r1 and rs2 to r2
> {code:java}
> add_rsgroup 'r1';add_rsgroup 'r2';move_servers_rsgroup
> 'r1',['host1:16020'];move_servers_rsgroup 'r2',['host2:16020']
> {code}
> # Restart Master
> # Run list_rsgroups for hbase shell, all region servers are assigned to
> default regroup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] liuml07 commented on a change in pull request #2056: HBASE-24710 Incorrect checksum calculation in saveVersion.sh
liuml07 commented on a change in pull request #2056:
URL: https://github.com/apache/hbase/pull/2056#discussion_r456574712
##
File path: hbase-common/src/saveVersion.sh
##
@@ -30,34 +30,34 @@ nativeOutputDirectory="$2/native/utils/"
pushd .
cd ..
-user=`whoami | sed -n -e 's/\\\//p'`
+user=$(whoami | sed -n -e "s/\\\//p")
if [ "$user" == "" ]
then
- user=`whoami`
+ user=$(whoami)
fi
-date=`date`
-cwd=`pwd`
+date=$(date)
+cwd=$(pwd)
if [ -d .svn ]; then
- revision=`(svn info | sed -n -e 's/Last Changed Rev: \(.*\)/\1/p') || true`
- url=`(svn info | sed -n -e 's/^URL: \(.*\)/\1/p') || true`
+ revision=$( (svn info | sed -n -e "s/Last Changed Rev: \(.*\)/\1/p") || true)
+ url=$( (svn info | sed -n -e 's/^URL: \(.*\)/\1/p') || true)
elif [ -d .git ]; then
- revision=`git log -1 --no-show-signature --pretty=format:"%H" || true`
- hostname=`hostname`
+ revision=$(git log -1 --no-show-signature --pretty=format:"%H" || true)
+ hostname=$(hostname)
url="git://${hostname}${cwd}"
fi
if [ -z "${revision}" ]; then
echo "[WARN] revision info is empty! either we're not in VCS or VCS commands
failed." >&2
revision="Unknown"
url="file://$cwd"
fi
-if ! [ -x "$(command -v openssl)" ]; then
+if ! [ -x "$(command -v openssl)" ]; then
if ! [ -x "$(command -v gpg)" ]; then
srcChecksum="Unknown"
else
-srcChecksum=`find hbase-*/src/main/ | grep -e "\.java" -e "\.proto" |
LC_ALL=C sort | xargs gpg --print-md sha512 | gpg --print-md sha512 | cut -d '
' -f 1`
Review comment:
Does `tr -d '[:space:]'` work? It's simpler and shorter than `tr '\n' '
' | sed 's/[[:space:]]*//g'`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] wchevreuil opened a new pull request #2079: HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails w…
wchevreuil opened a new pull request #2079: URL: https://github.com/apache/hbase/pull/2079 …ith an uncaught exception This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on pull request #2076: HBASE-24740 Enable journal logging for HBase snapshot operation
liuml07 commented on pull request #2076: URL: https://github.com/apache/hbase/pull/2076#issuecomment-660234327 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani closed pull request #2053: HBASE-24675: On Master restart all servers are assigned to default rs…
virajjasani closed pull request #2053: URL: https://github.com/apache/hbase/pull/2053 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2044: HBASE-24709 Support MoveCostFunction use a lower multiplier in offpea…
virajjasani commented on a change in pull request #2044:
URL: https://github.com/apache/hbase/pull/2044#discussion_r456558482
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##
@@ -827,26 +828,34 @@ protected double scale(double min, double max, double
value) {
*/
static class MoveCostFunction extends CostFunction {
private static final String MOVE_COST_KEY =
"hbase.master.balancer.stochastic.moveCost";
+private static final String MOVE_COST_OFFPEAK_KEY =
"hbase.master.balancer.stochastic.moveCost.offpeak";
private static final String MAX_MOVES_PERCENT_KEY =
"hbase.master.balancer.stochastic.maxMovePercent";
-private static final float DEFAULT_MOVE_COST = 7;
+@VisibleForTesting
+protected static final float DEFAULT_MOVE_COST = 7;
+protected static final float DEFAULT_MOVE_COST_OFFPEAK = 3;
private static final int DEFAULT_MAX_MOVES = 600;
private static final float DEFAULT_MAX_MOVE_PERCENT = 0.25f;
private final float maxMovesPercent;
+private final Configuration conf;
MoveCostFunction(Configuration conf) {
super(conf);
-
- // Move cost multiplier should be the same cost or higher than the rest
of the costs to ensure
- // that large benefits are need to overcome the cost of a move.
- this.setMultiplier(conf.getFloat(MOVE_COST_KEY, DEFAULT_MOVE_COST));
+ this.conf = conf;
// What percent of the number of regions a single run of the balancer
can move.
maxMovesPercent = conf.getFloat(MAX_MOVES_PERCENT_KEY,
DEFAULT_MAX_MOVE_PERCENT);
}
@Override
protected double cost() {
+ // Move cost multiplier should be the same cost or higher than the rest
of the costs to ensure
+ // that large benefits are need to overcome the cost of a move.
+ if (OffPeakHours.getInstance(conf).isOffPeakHour()) {
+this.setMultiplier(conf.getFloat(MOVE_COST_OFFPEAK_KEY,
DEFAULT_MOVE_COST_OFFPEAK));
+ } else {
+this.setMultiplier(conf.getFloat(MOVE_COST_KEY, DEFAULT_MOVE_COST));
+ }
Review comment:
@bsglz I just realized that whenever we update offpeak config
dynamically, `MoveCostFunction` constructor is called so we should better have
this logic in constructor only.
Check this out:
```
@Override
public void onConfigurationChange(Configuration conf) {
setConf(conf);
}
@Override
public synchronized void setConf(Configuration conf) {
super.setConf(conf);
...
...
costFunctions = new ArrayList<>();
costFunctions.add(new RegionCountSkewCostFunction(conf));
costFunctions.add(new PrimaryRegionCountSkewCostFunction(conf));
costFunctions.add(new MoveCostFunction(conf));
costFunctions.add(localityCost);
...
...
```
Hence, basically we can remove `this.conf = conf;` from this PR and move
this if condition to constructor.
When we try to update the config dynamically for SLB,
`onConfigurationChange()` is executed and that will reassign entire
constFunctions list.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #2017: HBASE-24669 Logging of ppid should be consistent across all occurrences
virajjasani commented on pull request #2017: URL: https://github.com/apache/hbase/pull/2017#issuecomment-660205076 Nice one. +1 on bringing this to all branches. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24746) The sort icons overlap the col name in master UI
[ https://issues.apache.org/jira/browse/HBASE-24746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-24746. -- Fix Version/s: 2.4.0 2.3.1 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > The sort icons overlap the col name in master UI > - > > Key: HBASE-24746 > URL: https://issues.apache.org/jira/browse/HBASE-24746 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > Attachments: HBASE-24746-afterpatch.png, HBASE-24746-beforepatch.png > > > In tables section, when description not few, the sort icons and col name are > overlap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani closed pull request #2078: HBASE-24746 The sort icons overlap the col name in master UI
virajjasani closed pull request #2078: URL: https://github.com/apache/hbase/pull/2078 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1935: HBASE-22146 SpaceQuotaViolationPolicy Disable is not working in Names…
Apache-HBase commented on pull request #1935: URL: https://github.com/apache/hbase/pull/1935#issuecomment-660185927 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | checkstyle | 1m 40s | master passed | | +1 :green_heart: | spotbugs | 3m 7s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | -0 :warning: | checkstyle | 0m 28s | hbase-client: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 3 new + 6 unchanged - 0 fixed = 9 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 40m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1935 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 09cb01171fe1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 16f306b4a1 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1935/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24354) Make it so can make an hbase1 schema and hbase2 schema equate
[ https://issues.apache.org/jira/browse/HBASE-24354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159997#comment-17159997 ] Michael Stack commented on HBASE-24354: --- [~zghao] you need this in branch-2.2? It allows replication from hbase1 to hbase2 (I didn't think it needed in older branches). > Make it so can make an hbase1 schema and hbase2 schema equate > - > > Key: HBASE-24354 > URL: https://issues.apache.org/jira/browse/HBASE-24354 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0 > > > Currently, NEW_VERSION_BEHAVIOR displays in hbase2 schemas with default value > of 'false'. This default is added whenever we create a new column descriptor. > Its not needed. We'll return default if empty or undefined. Meantime, since > this is not defined in hbase1, it makes it so we can't make equivalent > schemas so not possible to replicate from hbase1 to hbase2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24354) Make it so can make an hbase1 schema and hbase2 schema equate
[
https://issues.apache.org/jira/browse/HBASE-24354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159996#comment-17159996
]
Michael Stack commented on HBASE-24354:
---
I did not include the exception seen when we had this issue which was a mistake
if only because it made it harder to turn up this Jira in search.
It this one in HBaseAdmin:
{color:#80}throw new {color}IllegalArgumentException({color:#008000}"Table
" {color}+ tableName.getNameAsString()
+ {color:#008000}" exists in peer cluster " {color}+ peerDesc.getPeerId()
+ {color:#008000}", but the table descriptors are not same when compared with
source cluster."
{color} + {color:#008000}" Thus can not enable the table's replication
switch."{color});
> Make it so can make an hbase1 schema and hbase2 schema equate
> -
>
> Key: HBASE-24354
> URL: https://issues.apache.org/jira/browse/HBASE-24354
> Project: HBase
> Issue Type: Sub-task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> Currently, NEW_VERSION_BEHAVIOR displays in hbase2 schemas with default value
> of 'false'. This default is added whenever we create a new column descriptor.
> Its not needed. We'll return default if empty or undefined. Meantime, since
> this is not defined in hbase1, it makes it so we can't make equivalent
> schemas so not possible to replicate from hbase1 to hbase2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] skochhar commented on a change in pull request #1935: HBASE-22146 SpaceQuotaViolationPolicy Disable is not working in Names…
skochhar commented on a change in pull request #1935:
URL: https://github.com/apache/hbase/pull/1935#discussion_r456509573
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java
##
@@ -628,6 +628,34 @@ static Put createPutForNamespaceSnapshotSize(String
namespace, long size) {
}
}
+ /**
+ * Remove table usage snapshots (u:p columns) for the namespace passed
+ * @param connection connection to re-use
+ * @param namespace the namespace to fetch the list of table usage snapshots
+ */
+ static void deleteTableUsageSnapshotsForNamespace(Connection connection,
String namespace)
+throws IOException {
+Scan s = new Scan();
+//Get rows for all tables in namespace
+s.setRowPrefixFilter(Bytes.toBytes("t." + namespace));
Review comment:
I added a test to check for the bug, in my test I create 2 namespaces
with similar names and tested that the code only deletes the usage columns for
the tables in the deleted namespace.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1895: HBASE-24555 Correct the description of hbase.hregion.max.filesize
Apache-HBase commented on pull request #1895: URL: https://github.com/apache/hbase/pull/1895#issuecomment-660160771 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 28s | master passed | | +1 :green_heart: | javadoc | 2m 21s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | +1 :green_heart: | javadoc | 2m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 318m 56s | root in the patch failed. | | | | 334m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1895/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1895 | | Optional Tests | javac javadoc unit | | uname | Linux 43e7682e0262 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a4e5ccc798 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1895/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1895/3/testReport/ | | Max. process+thread count | 5802 (vs. ulimit of 12500) | | modules | C: hbase-common . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1895/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2076: HBASE-24740 Enable journal logging for HBase snapshot operation
virajjasani commented on a change in pull request #2076:
URL: https://github.com/apache/hbase/pull/2076#discussion_r456480852
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
##
@@ -343,6 +343,13 @@ private void load() throws IOException {
}
}
+ /**
+ * Sets the status task for monitoring all the subtasks for Snapshot
operation
+ */
+ public void setMonitoredTask(MonitoredTask statusTask) {
Review comment:
+1
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTaskImpl.java
##
@@ -18,15 +18,13 @@
*/
package org.apache.hadoop.hbase.monitoring;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
-
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
-
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
Review comment:
This also looks formatter's doing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
