[GitHub] [hbase] brfrn169 opened a new pull request #987: HBASE-23165 [hbtop] Some modifications from HBASE-22988
brfrn169 opened a new pull request #987: HBASE-23165 [hbtop] Some modifications from HBASE-22988 URL: https://github.com/apache/hbase/pull/987 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23643) Add document for "HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs"
Toshihiro Suzuki created HBASE-23643: Summary: Add document for "HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs" Key: HBASE-23643 URL: https://issues.apache.org/jira/browse/HBASE-23643 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Add the new feature of hbtop in HBASE-23065 to the ref guide. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23305) Implement master based AsyncRegistry
[
https://issues.apache.org/jira/browse/HBASE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007938#comment-17007938
]
HBase QA commented on HBASE-23305:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
1s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
37s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 5m
22s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
22s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} hbase-common: The patch generated 0 new + 4
unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} hbase-client: The patch generated 0 new + 0
unchanged - 8 fixed = 0 total (was 8) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} hbase-server: The patch generated 0 new + 223
unchanged - 50 fixed = 223 total (was 273) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 59s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 11m
30s{color} | {color:red} hbase-server in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
47s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
19s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red}
[GitHub] [hbase] Apache-HBase commented on issue #954: HBASE-23305: Master based registry implementation
Apache-HBase commented on issue #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#issuecomment-570764147 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 1s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 1m 50s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 2m 37s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 5m 22s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 7m 22s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 57s | the patch passed | | +1 :green_heart: | compile | 2m 2s | the patch passed | | +1 :green_heart: | javac | 2m 2s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | hbase-common: The patch generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5) | | +1 :green_heart: | checkstyle | 0m 35s | hbase-client: The patch generated 0 new + 0 unchanged - 8 fixed = 0 total (was 8) | | +1 :green_heart: | checkstyle | 1m 36s | hbase-server: The patch generated 0 new + 223 unchanged - 50 fixed = 223 total (was 273) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 59s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 43s | the patch passed | | -1 :x: | findbugs | 11m 30s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 47s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 19s | hbase-client in the patch passed. | | -1 :x: | unit | 193m 28s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 15s | The patch does not generate ASF License warnings. | | | | 284m 46s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.client.TestFromClientSide | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/954 | | JIRA Issue | HBASE-23305 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux eff02e1348bf 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-954/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / d016cd9962 | | Default Java | 1.8.0_181 | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/9/artifact/out/patch-findbugs-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/9/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/9/testReport/ | | Max. process+thread count | 6672 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/9/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache
[GitHub] [hbase] Apache-HBase commented on issue #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
Apache-HBase commented on issue #982: HBASE-23636 Disable table may hang when regionserver stop or abort. URL: https://github.com/apache/hbase/pull/982#issuecomment-570761100 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 59s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 4m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 5m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 5s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 37s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | checkstyle | 1m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 59s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 5m 15s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 184m 22s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 250m 9s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestRegionReplicasWithRestartScenarios | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-982/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/982 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7cf05c83e51d 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-982/out/precommit/personality/provided.sh | | git revision | master / ab9766599d | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-982/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-982/2/testReport/ | | Max. process+thread count | 5488 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-982/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
virajjasani commented on a change in pull request #982: HBASE-23636 Disable
table may hang when regionserver stop or abort.
URL: https://github.com/apache/hbase/pull/982#discussion_r363018415
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
##
@@ -482,13 +482,18 @@ private void assignRegions(MasterProcedureEnv env,
List regions) thr
regionNode.getProcedure().serverCrashed(env, regionNode,
getServerName());
continue;
}
-if
(env.getMasterServices().getTableStateManager().isTableState(regionNode.getTable(),
- TableState.State.DISABLING, TableState.State.DISABLED)) {
+if (env.getMasterServices().getTableStateManager()
+ .isTableState(regionNode.getTable(), TableState.State.DISABLING)) {
env.getAssignmentManager().regionClosedAbnormally(regionNode);
- LOG.info("{} found table disabled for region {}, set it state to
ABNORMALLY_CLOSED.",
+ LOG.info("{} found table disabling for region {}, set it state to
ABNORMALLY_CLOSED.",
this, regionNode);
continue;
}
+if (env.getMasterServices().getTableStateManager()
+ .isTableState(regionNode.getTable(), TableState.State.DISABLED)) {
+ LOG.warn("{} found table disabled for region {}.", this, regionNode);
Review comment:
Seems like it will print everything, is it good to log:
```
LOG.warn("Found table disabled for region {}, procDetails: {}", regionNode,
this);
```
if you feel this might be more readable
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
virajjasani commented on a change in pull request #982: HBASE-23636 Disable
table may hang when regionserver stop or abort.
URL: https://github.com/apache/hbase/pull/982#discussion_r363018415
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
##
@@ -482,13 +482,18 @@ private void assignRegions(MasterProcedureEnv env,
List regions) thr
regionNode.getProcedure().serverCrashed(env, regionNode,
getServerName());
continue;
}
-if
(env.getMasterServices().getTableStateManager().isTableState(regionNode.getTable(),
- TableState.State.DISABLING, TableState.State.DISABLED)) {
+if (env.getMasterServices().getTableStateManager()
+ .isTableState(regionNode.getTable(), TableState.State.DISABLING)) {
env.getAssignmentManager().regionClosedAbnormally(regionNode);
- LOG.info("{} found table disabled for region {}, set it state to
ABNORMALLY_CLOSED.",
+ LOG.info("{} found table disabling for region {}, set it state to
ABNORMALLY_CLOSED.",
this, regionNode);
continue;
}
+if (env.getMasterServices().getTableStateManager()
+ .isTableState(regionNode.getTable(), TableState.State.DISABLED)) {
+ LOG.warn("{} found table disabled for region {}.", this, regionNode);
Review comment:
Seems like it will print everything, is it good to log:
```
LOG.warn("Found table disabled for region {}, procDetails: {}", regionNode,
this);
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
virajjasani commented on a change in pull request #982: HBASE-23636 Disable
table may hang when regionserver stop or abort.
URL: https://github.com/apache/hbase/pull/982#discussion_r363018205
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
##
@@ -482,13 +482,18 @@ private void assignRegions(MasterProcedureEnv env,
List regions) thr
regionNode.getProcedure().serverCrashed(env, regionNode,
getServerName());
continue;
}
-if
(env.getMasterServices().getTableStateManager().isTableState(regionNode.getTable(),
- TableState.State.DISABLING, TableState.State.DISABLED)) {
+if (env.getMasterServices().getTableStateManager()
+ .isTableState(regionNode.getTable(), TableState.State.DISABLING)) {
env.getAssignmentManager().regionClosedAbnormally(regionNode);
- LOG.info("{} found table disabled for region {}, set it state to
ABNORMALLY_CLOSED.",
+ LOG.info("{} found table disabling for region {}, set it state to
ABNORMALLY_CLOSED.",
this, regionNode);
continue;
}
+if (env.getMasterServices().getTableStateManager()
+ .isTableState(regionNode.getTable(), TableState.State.DISABLED)) {
+ LOG.warn("{} found table disabled for region {}.", this, regionNode);
Review comment:
`this` would print `server`, `splitWal` and `meta` correct? or it will print
`pid`, `ppid`, `state` etc additionally?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mnpoonia commented on issue #978: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on issue #978: HBASE-22285 A normalizer which merges small size regions with adjacen… URL: https://github.com/apache/hbase/pull/978#issuecomment-570756563 > How does this relate to #931 ? Different branches? Thanks. Yes . That is for branch-1. And they are bit different so created two PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on a change in pull request #978: HBASE-22285 A normalizer
which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/978#discussion_r363016055
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizer.java
##
@@ -0,0 +1,131 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.sql.Timestamp;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HBaseIOException;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Implementation of MergeNormalizer Logic in use:
+ *
+ * get all regions of a given table
+ * get avg size S of each region (by total size of store files reported in
RegionLoad)
+ * two regions R1 and its neighbour R2 are merged, if R1 + R2 < S, and
all such regions are
+ * returned to be merged
+ * Otherwise, no action is performed
+ *
+ *
+ * Region sizes are coarse and approximate on the order of megabytes. Also,
empty regions (less than
+ * 1MB) are also merged if the age of region is > MIN_DURATION_FOR_MERGE
(default 2)
+ *
+ */
+
+@InterfaceAudience.Private
+public class MergeNormalizer extends BaseNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(MergeNormalizer.class);
+ private static final int MIN_REGION_COUNT = 3;
+
+ private int minRegionCount;
+ private static long[] skippedCount = new
long[NormalizationPlan.PlanType.values().length];
+
+ public MergeNormalizer() {
+minRegionCount =
HBaseConfiguration.create().getInt("hbase.normalizer.min.region.count", 3);
+ }
+
+ @Override
+ public void planSkipped(RegionInfo hri, NormalizationPlan.PlanType type) {
+skippedCount[type.ordinal()]++;
+ }
+
+ @Override
+ public long getSkippedCount(NormalizationPlan.PlanType type) {
+return skippedCount[type.ordinal()];
+ }
+
+ @Override
+ public List computePlanForTable(TableName table) throws
HBaseIOException {
+List plans = new ArrayList<>();
+if (!shouldNormalize(table)) {
+ return null;
+}
+// atleast one of the two regions should be older than MIN_REGION_DURATION
days
+List normalizationPlans =
getMergeNormalizationPlan(table);
+for (NormalizationPlan plan : normalizationPlans) {
+ if (plan instanceof MergeNormalizationPlan) {
+RegionInfo hri = ((MergeNormalizationPlan) plan).getFirstRegion();
+RegionInfo hri2 = ((MergeNormalizationPlan) plan).getSecondRegion();
+if (isOldEnoughToMerge(hri) || isOldEnoughToMerge(hri2)) {
+ plans.add(plan);
+}
+ }
+}
+if (plans.isEmpty()) {
+ LOG.debug("No normalization needed, regions look good for table: {}",
table);
+ return null;
+}
+return plans;
+ }
+
+ private boolean isOldEnoughToMerge(RegionInfo hri) {
+Timestamp currentTime = new Timestamp(System.currentTimeMillis());
+Timestamp hriTime = new Timestamp(hri.getRegionId());
+boolean isOld =
+ new Timestamp(hriTime.getTime() +
TimeUnit.DAYS.toMillis(getMinimumDurationBeforeMerge()))
+.before(currentTime);
+return isOld;
+ }
+
+ private boolean shouldNormalize(TableName table) {
+boolean normalize = false;
+if (table == null || table.isSystemTable()) {
+ LOG.debug("Normalization of system table {} isn't allowed", table);
Review comment:
Agreed. But we don't know why someone put it there in first place. SO better
to keep it. We can remove it later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007907#comment-17007907
]
Hudson commented on HBASE-2:
Results for branch branch-2.1
[build #1761 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23587) The FSYNC_WAL flag does not work on branch-2.x
[
https://issues.apache.org/jira/browse/HBASE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007906#comment-17007906
]
Hudson commented on HBASE-23587:
Results for branch branch-2.1
[build #1761 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1761//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The FSYNC_WAL flag does not work on branch-2.x
> --
>
> Key: HBASE-23587
> URL: https://issues.apache.org/jira/browse/HBASE-23587
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> You can check the reference to SyncFuture.isForceSync, no one calls it. And
> in FSHLog, we always use the field useHsync instead of getting one from the
> SyncFuture. I think the problem here is that we have done some optimization
> for batching the sync futures on branch-2.x so it is not easy to get the
> forceSync flag from a single SyncFuture when doing sync.
> But anyway, we need to fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
Apache-HBase commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing URL: https://github.com/apache/hbase/pull/917#issuecomment-570752567 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 34s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 4m 27s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | checkstyle | 1m 16s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 4m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 152m 1s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 210m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/917 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2266046366c2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-917/out/precommit/personality/provided.sh | | git revision | master / ab9766599d | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/7/testReport/ | | Max. process+thread count | 5230 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/7/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on issue #954: HBASE-23305: Master based registry implementation
bharathv commented on issue #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#issuecomment-570749954 Fixed the check style issues and rebased on the latest tip of the feature branch (and squashed all the commits). Test failures seem to be flakes and run fine for me locally. I think this is good to go for another round of review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] binlijin commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
binlijin commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort. URL: https://github.com/apache/hbase/pull/982#discussion_r363009642 ## File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRaceBetweenSCPAndDTP.java ## @@ -0,0 +1,146 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.master.assignment; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.HBaseClassTestRule; +import org.apache.hadoop.hbase.HBaseTestingUtility; +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.ServerName; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.client.RegionInfo; +import org.apache.hadoop.hbase.master.HMaster; +import org.apache.hadoop.hbase.master.MasterServices; +import org.apache.hadoop.hbase.master.RegionPlan; +import org.apache.hadoop.hbase.master.procedure.DisableTableProcedure; +import org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure; +import org.apache.hadoop.hbase.procedure2.ProcedureExecutor; +import org.apache.hadoop.hbase.testclassification.LargeTests; +import org.apache.hadoop.hbase.testclassification.MasterTests; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.zookeeper.KeeperException; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.ClassRule; +import org.junit.Test; +import org.junit.experimental.categories.Category; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.util.List; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.Future; Review comment: OK This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] binlijin commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
binlijin commented on a change in pull request #982: HBASE-23636 Disable table
may hang when regionserver stop or abort.
URL: https://github.com/apache/hbase/pull/982#discussion_r363009646
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
##
@@ -484,6 +484,9 @@ private void assignRegions(MasterProcedureEnv env,
List regions) thr
}
if
(env.getMasterServices().getTableStateManager().isTableState(regionNode.getTable(),
TableState.State.DISABLING, TableState.State.DISABLED)) {
+ env.getAssignmentManager().regionClosedAbnormally(regionNode);
Review comment:
Looks like no need, let me fix it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] binlijin commented on a change in pull request #982: HBASE-23636 Disable table may hang when regionserver stop or abort.
binlijin commented on a change in pull request #982: HBASE-23636 Disable table
may hang when regionserver stop or abort.
URL: https://github.com/apache/hbase/pull/982#discussion_r363009645
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRaceBetweenSCPAndDTP.java
##
@@ -0,0 +1,146 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.assignment;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.master.HMaster;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.hadoop.hbase.master.RegionPlan;
+import org.apache.hadoop.hbase.master.procedure.DisableTableProcedure;
+import org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureExecutor;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MasterTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.zookeeper.KeeperException;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.Future;
+
+/**
+ * Testcase for HBASE-23636.
+ */
+@Category({ MasterTests.class, LargeTests.class })
+public class TestRaceBetweenSCPAndDTP {
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRaceBetweenSCPAndDTP.class);
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23098) [bulkload] If one of the peers in a cluster is configured with NAMESPACE level, its hfile-refs(zk) will be backlogged
[
https://issues.apache.org/jira/browse/HBASE-23098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007868#comment-17007868
]
Hudson commented on HBASE-23098:
Results for branch branch-2
[build #2405 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [bulkload] If one of the peers in a cluster is configured with NAMESPACE
> level, its hfile-refs(zk) will be backlogged
> -
>
> Key: HBASE-23098
> URL: https://issues.apache.org/jira/browse/HBASE-23098
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Yiran Wu
>Assignee: Yiran Wu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23098.v0.patch, HBASE-23098.v1.patch
>
>
> If one of the peers in a cluster is configured with NAMESPACE level, its
> hfile-refs(zk) will be backlogged.
> We have three tables in the cluster and table name is
> ns1:test
> default:table1
> default:bulkload
> and this Cluster have three peers
> {code:java}
> add_peer 's1', CLUSTER_KEY => "cluster1:2181:/hbase"
> add_peer 's2', CLUSTER_KEY => "cluster2:2181:/hbase", TABLE_CFS => \{
> "table1" => [] }
> add_peer 's3', CLUSTER_KEY => "cluster3:2182:/hbase-prod", NAMESPACES =>
> ["ns1"]
> {code}
> ns1's hfile-refs backlogged when we bulk load files to the default:bulkload
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23587) The FSYNC_WAL flag does not work on branch-2.x
[
https://issues.apache.org/jira/browse/HBASE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007870#comment-17007870
]
Hudson commented on HBASE-23587:
Results for branch branch-2
[build #2405 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The FSYNC_WAL flag does not work on branch-2.x
> --
>
> Key: HBASE-23587
> URL: https://issues.apache.org/jira/browse/HBASE-23587
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> You can check the reference to SyncFuture.isForceSync, no one calls it. And
> in FSHLog, we always use the field useHsync instead of getting one from the
> SyncFuture. I think the problem here is that we have done some optimization
> for batching the sync futures on branch-2.x so it is not easy to get the
> forceSync flag from a single SyncFuture when doing sync.
> But anyway, we need to fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23286) Improve MTTR: Split WAL to HFile
[
https://issues.apache.org/jira/browse/HBASE-23286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007869#comment-17007869
]
Hudson commented on HBASE-23286:
Results for branch branch-2
[build #2405 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve MTTR: Split WAL to HFile
>
>
> Key: HBASE-23286
> URL: https://issues.apache.org/jira/browse/HBASE-23286
> Project: HBase
> Issue Type: Improvement
> Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> After HBASE-20724, the compaction event marker is not used anymore when
> failover. So our new proposal is split WAL to HFile to imporve MTTR. It has 3
> steps:
> # Read WAL and write HFile to region’s column family’s recovered.hfiles
> directory.
> # Open region.
> # Bulkload the recovered.hfiles for every column family.
> The design doc was attathed by a google doc. Any suggestions are welcomed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23596) HBCKServerCrashProcedure can double assign
[
https://issues.apache.org/jira/browse/HBASE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007866#comment-17007866
]
Hudson commented on HBASE-23596:
Results for branch branch-2
[build #2405 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBCKServerCrashProcedure can double assign
> --
>
> Key: HBASE-23596
> URL: https://issues.apache.org/jira/browse/HBASE-23596
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> The new SCP that does SCP plus cleanup 'Unknown Servers' with mentions in
> hbase:meta added by the below can make for double assignments.
> {code}
> commit c238891a26734e1e4276b6b1677a58cf83de5dc4
> Author: stack
> Date: Wed Nov 13 22:36:26 2019 -0800
> HBASE-23282 HBCKServerCrashProcedure for 'Unknown Servers'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23369) Auto-close 'unknown' Regions reported as OPEN on RegionServers
[
https://issues.apache.org/jira/browse/HBASE-23369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007871#comment-17007871
]
Hudson commented on HBASE-23369:
Results for branch branch-2
[build #2405 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Auto-close 'unknown' Regions reported as OPEN on RegionServers
> --
>
> Key: HBASE-23369
> URL: https://issues.apache.org/jira/browse/HBASE-23369
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> In old days, if a RegionServer reported a variance that didn't agree w/
> Master view of the cluster, we'd kill the RegionServer.
> Lately, in tests that overrun a cluster, after a sustained high-load, Master
> can start failing its updates against Meta (CallQueueTooBigException <= More
> on this later). It then can lose proper accounting of all Region members. One
> variant has a RegionServer reporting its list of open Regions to the Master
> and the Master doesn't 'know' of a particular Region or the Master may know
> the Region but expects it open on another RegionServer.
> Here is an example of how it looks each time RS reports:
> {code}
> 2019-12-03 07:07:00,757 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode
> but reported ONLINE at server.example.org,16020,1575354666245
> (inServerRegionList=false).
> 2019-12-03 07:07:03,793 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode
> but reported ONLINE at server.example.org,16020,1575354666245
> (inServerRegionList=false).
> {code}
> Will also show as an 'inconsistency' in the 'HBCK' tab on the Master UI.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23632) DeadServer cleanup
[
https://issues.apache.org/jira/browse/HBASE-23632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007867#comment-17007867
]
Hudson commented on HBASE-23632:
Results for branch branch-2
[build #2405 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2405//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> DeadServer cleanup
> --
>
> Key: HBASE-23632
> URL: https://issues.apache.org/jira/browse/HBASE-23632
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Cleanup of DeadServer class shutting down access, undoing duplication, adding
> doc., and removing unused code.
> One change is that we do not remove a server from 'processing' list when we
> 'remove' deadservers; we let SCP do it since it owns processing list (Saw
> issue where on fast restart of a server, the server was removed from
> deadserver and from processing list though the SCP for the dead server was
> still running -- no repercussions that I could see but a little confusing).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23635) Reduce number of Checkstyle violations in hbase-mapreduce
[
https://issues.apache.org/jira/browse/HBASE-23635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23635:
--
Fix Version/s: 2.1.9
2.2.3
2.3.0
3.0.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Reduce number of Checkstyle violations in hbase-mapreduce
> -
>
> Key: HBASE-23635
> URL: https://issues.apache.org/jira/browse/HBASE-23635
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-mapreduce}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #954: HBASE-23305: Master based registry implementation
Apache-HBase commented on issue #954: HBASE-23305: Master based registry implementation URL: https://github.com/apache/hbase/pull/954#issuecomment-570737099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 31 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 59s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 2m 3s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 2m 39s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 4m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 35s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 34s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | hbase-common: The patch generated 0 new + 4 unchanged - 1 fixed = 4 total (was 5) | | -1 :x: | checkstyle | 0m 33s | hbase-client: The patch generated 2 new + 14 unchanged - 10 fixed = 16 total (was 24) | | +1 :green_heart: | checkstyle | 1m 34s | hbase-server: The patch generated 0 new + 224 unchanged - 50 fixed = 224 total (was 274) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 34s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 16s | the patch passed | | +1 :green_heart: | findbugs | 7m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 7s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 55s | hbase-client in the patch passed. | | -1 :x: | unit | 166m 9s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 13s | The patch does not generate ASF License warnings. | | | | 247m 11s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.client.TestFromClientSide | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/954 | | JIRA Issue | HBASE-23305 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1bcc84663300 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-954/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / dffa9be899 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/8/artifact/out/diff-checkstyle-hbase-client.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/8/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/8/testReport/ | | Max. process+thread count | 6395 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-954/8/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.1
[jira] [Commented] (HBASE-23305) Implement master based AsyncRegistry
[
https://issues.apache.org/jira/browse/HBASE-23305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007850#comment-17007850
]
HBase QA commented on HBASE-23305:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 31 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
59s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
39s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
43s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
35s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} hbase-common: The patch generated 0 new + 4
unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
33s{color} | {color:red} hbase-client: The patch generated 2 new + 14 unchanged
- 10 fixed = 16 total (was 24) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} hbase-server: The patch generated 0 new + 224
unchanged - 50 fixed = 224 total (was 274) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 34s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
7s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
55s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color}
[jira] [Commented] (HBASE-23642) Reintroduce ReplicationUtils.contains as deprecated
[ https://issues.apache.org/jira/browse/HBASE-23642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007849#comment-17007849 ] Guanghao Zhang commented on HBASE-23642: I searched issues yesterday but not found this. Thanks for reminding. :) > Reintroduce ReplicationUtils.contains as deprecated > --- > > Key: HBASE-23642 > URL: https://issues.apache.org/jira/browse/HBASE-23642 > Project: HBase > Issue Type: Task > Components: Replication >Affects Versions: 2.3.0, 2.2.3, 2.1.9 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.3.0, 2.2.3, 2.1.9 > > > ReplicationUtils.contains was removed in HBASE-23345. This is an IA.Private > class but HBCK2 relied on this method. > The plan is to add back the method as deprecated and also move HBCK2 off of > this to the new ReplicationPeerConfig#needToReplicate(TableName) instead > (HBASE-23641). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23642) Reintroduce ReplicationUtils.contains as deprecated
[ https://issues.apache.org/jira/browse/HBASE-23642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007844#comment-17007844 ] Guanghao Zhang commented on HBASE-23642: OK. +1 for branch-2.2. > Reintroduce ReplicationUtils.contains as deprecated > --- > > Key: HBASE-23642 > URL: https://issues.apache.org/jira/browse/HBASE-23642 > Project: HBase > Issue Type: Task > Components: Replication >Affects Versions: 2.3.0, 2.2.3, 2.1.9 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.3.0, 2.2.3, 2.1.9 > > > ReplicationUtils.contains was removed in HBASE-23345. This is an IA.Private > class but HBCK2 relied on this method. > The plan is to add back the method as deprecated and also move HBCK2 off of > this to the new ReplicationPeerConfig#needToReplicate(TableName) instead > (HBASE-23641). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] HorizonNet merged pull request #981: HBASE-23635 Reduced number of Checkstyle violations in hbase-mapreduce
HorizonNet merged pull request #981: HBASE-23635 Reduced number of Checkstyle violations in hbase-mapreduce URL: https://github.com/apache/hbase/pull/981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe…
saintstack commented on issue #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe… URL: https://github.com/apache/hbase/pull/902#issuecomment-570730091 Thanks. Fixing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23638) Set version to 2.2.3 in branch-2.2 for first RC of 2.2.3
[
https://issues.apache.org/jira/browse/HBASE-23638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007819#comment-17007819
]
Hudson commented on HBASE-23638:
Results for branch branch-2.2
[build #743 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Set version to 2.2.3 in branch-2.2 for first RC of 2.2.3
>
>
> Key: HBASE-23638
> URL: https://issues.apache.org/jira/browse/HBASE-23638
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23596) HBCKServerCrashProcedure can double assign
[
https://issues.apache.org/jira/browse/HBASE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007814#comment-17007814
]
Hudson commented on HBASE-23596:
Results for branch branch-2.2
[build #743 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBCKServerCrashProcedure can double assign
> --
>
> Key: HBASE-23596
> URL: https://issues.apache.org/jira/browse/HBASE-23596
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> The new SCP that does SCP plus cleanup 'Unknown Servers' with mentions in
> hbase:meta added by the below can make for double assignments.
> {code}
> commit c238891a26734e1e4276b6b1677a58cf83de5dc4
> Author: stack
> Date: Wed Nov 13 22:36:26 2019 -0800
> HBASE-23282 HBCKServerCrashProcedure for 'Unknown Servers'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23587) The FSYNC_WAL flag does not work on branch-2.x
[
https://issues.apache.org/jira/browse/HBASE-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007816#comment-17007816
]
Hudson commented on HBASE-23587:
Results for branch branch-2.2
[build #743 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The FSYNC_WAL flag does not work on branch-2.x
> --
>
> Key: HBASE-23587
> URL: https://issues.apache.org/jira/browse/HBASE-23587
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> You can check the reference to SyncFuture.isForceSync, no one calls it. And
> in FSHLog, we always use the field useHsync instead of getting one from the
> SyncFuture. I think the problem here is that we have done some optimization
> for batching the sync futures on branch-2.x so it is not easy to get the
> forceSync flag from a single SyncFuture when doing sync.
> But anyway, we need to fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23637) Generate CHANGES.md and RELEASENOTES.md for 2.2.3
[
https://issues.apache.org/jira/browse/HBASE-23637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007818#comment-17007818
]
Hudson commented on HBASE-23637:
Results for branch branch-2.2
[build #743 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Generate CHANGES.md and RELEASENOTES.md for 2.2.3
> -
>
> Key: HBASE-23637
> URL: https://issues.apache.org/jira/browse/HBASE-23637
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23632) DeadServer cleanup
[
https://issues.apache.org/jira/browse/HBASE-23632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007815#comment-17007815
]
Hudson commented on HBASE-23632:
Results for branch branch-2.2
[build #743 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> DeadServer cleanup
> --
>
> Key: HBASE-23632
> URL: https://issues.apache.org/jira/browse/HBASE-23632
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Cleanup of DeadServer class shutting down access, undoing duplication, adding
> doc., and removing unused code.
> One change is that we do not remove a server from 'processing' list when we
> 'remove' deadservers; we let SCP do it since it owns processing list (Saw
> issue where on fast restart of a server, the server was removed from
> deadserver and from processing list though the SCP for the dead server was
> still running -- no repercussions that I could see but a little confusing).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23175) Yarn unable to acquire delegation token for HBase Spark jobs
[
https://issues.apache.org/jira/browse/HBASE-23175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007817#comment-17007817
]
Hudson commented on HBASE-23175:
Results for branch branch-2.2
[build #743 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/743//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Yarn unable to acquire delegation token for HBase Spark jobs
>
>
> Key: HBASE-23175
> URL: https://issues.apache.org/jira/browse/HBASE-23175
> Project: HBase
> Issue Type: Bug
> Components: security, spark
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.8, 2.2.3
>
> Attachments: HBASE-23175.master.001.patch
>
>
> Spark rely on the TokenUtil.obtainToken(conf) API which is removed in
> HBase-2.0, though it has been fixed in SPARK-26432 to use the new API but
> planned for Spark-3.0, hence we need the fix in HBase until they release it
> and we upgrade it
> {code}
> 18/03/20 20:39:07 ERROR ApplicationMaster: User class threw exception:
> org.apache.hadoop.hbase.HBaseIOException:
> com.google.protobuf.ServiceException: Error calling method
> hbase.pb.AuthenticationService.GetAuthenticationToken
> org.apache.hadoop.hbase.HBaseIOException:
> com.google.protobuf.ServiceException: Error calling method
> hbase.pb.AuthenticationService.GetAuthenticationToken
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:346)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:86)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:121)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:118)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:118)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:272)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:533)
> at
> org.apache.hadoop.hbase.spark.HBaseContext.(HBaseContext.scala:73)
> at
> org.apache.hadoop.hbase.spark.JavaHBaseContext.(JavaHBaseContext.scala:46)
> at
> org.apache.hadoop.hbase.spark.example.hbasecontext.JavaHBaseBulkDeleteExample.main(JavaHBaseBulkDeleteExample.java:64)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:706)
> Caused by: com.google.protobuf.ServiceException: Error calling method
> hbase.pb.AuthenticationService.GetAuthenticationToken
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:71)
> at
> org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:81)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
Apache-HBase commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing URL: https://github.com/apache/hbase/pull/917#issuecomment-570722056 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 2m 55s | root in master failed. | | -1 :x: | compile | 0m 38s | hbase-server in master failed. | | +1 :green_heart: | checkstyle | 1m 15s | master passed | | -1 :x: | shadedjars | 3m 24s | branch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | master passed | | +0 :ok: | spotbugs | 4m 39s | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 :x: | findbugs | 0m 38s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 2m 27s | root in the patch failed. | | -1 :x: | compile | 0m 36s | hbase-server in the patch failed. | | -1 :x: | javac | 0m 36s | hbase-server in the patch failed. | | +1 :green_heart: | checkstyle | 1m 13s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | shadedjars | 3m 25s | patch has 16 errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 1m 50s | The patch causes 16 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 3m 43s | The patch causes 16 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 5m 41s | The patch causes 16 errors with Hadoop v3.1.2. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | | -1 :x: | findbugs | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 37s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 26m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/917 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c9633d5615b5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-917/out/precommit/personality/provided.sh | | git revision | master / af3bb1feca | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/branch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/branch-compile-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/branch-shadedjars.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/branch-findbugs-hbase-server.txt | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-compile-hbase-server.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-compile-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-shadedjars.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-javac-2.8.5.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-javac-2.9.2.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-javac-3.1.2.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/6/artifact/out/patch-findbugs-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-9
[jira] [Commented] (HBASE-23640) Move HBaseClusterManager.CommandProvider and it's different implementations as seperate class/interface
[
https://issues.apache.org/jira/browse/HBASE-23640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007810#comment-17007810
]
Nick Dimiduk commented on HBASE-23640:
--
You sure we want to do this? {{HBaseClusterManager}} is a class under
{{src/test/java}} and {{InterfaceAudience.Private}}. We that means it's not an
API we expect users to consume. I think what you're asking for is an
{{hbase-testing-utilties}} artifact that ships a suite of testing utilities
that can be consumed inside and outside of the HBase project.
> Move HBaseClusterManager.CommandProvider and it's different implementations
> as seperate class/interface
> ---
>
> Key: HBASE-23640
> URL: https://issues.apache.org/jira/browse/HBASE-23640
> Project: HBase
> Issue Type: Improvement
>Reporter: Mihir Monani
>Assignee: Mihir Monani
>Priority: Major
>
> HBaseClusterManager.CommandProvider and it's implementations are private to
> HBaseClusterManager and hbase-it package.
> If any one wants to do custom implementations of ClusterManager Interface,
> they cant's use this code as it is not visible outside the hbase-it package.
> This Jira is also dependents on
> [HBASE-23639|https://issues.apache.org/jira/browse/HBASE-23639]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe…
ndimiduk commented on issue #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe… URL: https://github.com/apache/hbase/pull/902#issuecomment-570719971 I'm getting a build failure after this merge, boss: ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project hbase-server: Compilation failure [ERROR] /Users/ndimiduk/repos/apache/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java:[1108,59] cannot find symbol [ERROR] symbol: method getClusterConnection() [ERROR] location: variable master of type org.apache.hadoop.hbase.master.MasterServices ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23604) Clarify AsyncRegistry usage in the code
[ https://issues.apache.org/jira/browse/HBASE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007807#comment-17007807 ] Bharath Vissapragada commented on HBASE-23604: -- ack. > Clarify AsyncRegistry usage in the code > --- > > Key: HBASE-23604 > URL: https://issues.apache.org/jira/browse/HBASE-23604 > Project: HBase > Issue Type: Task > Components: Client >Affects Versions: master >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Minor > Fix For: HBASE-18095 > > > As [~stack] noted in the code review > https://github.com/apache/hbase/pull/954, the usage of registry in the client > code is not super clear. The ask here is to rename it something that makes > the context more clear. > Creating a separate jira because the patch touches a lots of files. I don't > want to mix it with the patch for HBASE-23305. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23604) Clarify AsyncRegistry usage in the code
[ https://issues.apache.org/jira/browse/HBASE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007803#comment-17007803 ] Nick Dimiduk commented on HBASE-23604: -- [~bharathv] FYI we strive to have Jira subject and the git commit title match. > Clarify AsyncRegistry usage in the code > --- > > Key: HBASE-23604 > URL: https://issues.apache.org/jira/browse/HBASE-23604 > Project: HBase > Issue Type: Task > Components: Client >Affects Versions: master >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Minor > Fix For: HBASE-18095 > > > As [~stack] noted in the code review > https://github.com/apache/hbase/pull/954, the usage of registry in the client > code is not super clear. The ask here is to rename it something that makes > the context more clear. > Creating a separate jira because the patch touches a lots of files. I don't > want to mix it with the patch for HBASE-23305. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23604) Clarify AsyncRegistry usage in the code
[ https://issues.apache.org/jira/browse/HBASE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23604: - Summary: Clarify AsyncRegistry usage in the code (was: Usage of "registry" is unclear in the code.) > Clarify AsyncRegistry usage in the code > --- > > Key: HBASE-23604 > URL: https://issues.apache.org/jira/browse/HBASE-23604 > Project: HBase > Issue Type: Task > Components: Client >Affects Versions: master >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Minor > Fix For: HBASE-18095 > > > As [~stack] noted in the code review > https://github.com/apache/hbase/pull/954, the usage of registry in the client > code is not super clear. The ask here is to rename it something that makes > the context more clear. > Creating a separate jira because the patch touches a lots of files. I don't > want to mix it with the patch for HBASE-23305. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[ https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-18095: - Status: In Progress (was: Patch Available) > Provide an option for clients to find the server hosting META that does not > involve the ZooKeeper client > > > Key: HBASE-18095 > URL: https://issues.apache.org/jira/browse/HBASE-18095 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch > > > Clients are required to connect to ZooKeeper to find the location of the > regionserver hosting the meta table region. Site configuration provides the > client a list of ZK quorum peers and the client uses an embedded ZK client to > query meta location. Timeouts and retry behavior of this embedded ZK client > are managed orthogonally to HBase layer settings and in some cases the ZK > cannot manage what in theory the HBase client can, i.e. fail fast upon outage > or network partition. > We should consider new configuration settings that provide a list of > well-known master and backup master locations, and with this information the > client can contact any of the master processes directly. Any master in either > active or passive state will track meta location and respond to requests for > it with its cached last known location. If this location is stale, the client > can ask again with a flag set that requests the master refresh its location > cache and return the up-to-date location. Every client interaction with the > cluster thus uses only HBase RPC as transport, with appropriate settings > applied to the connection. The configuration toggle that enables this > alternative meta location lookup should be false by default. > This removes the requirement that HBase clients embed the ZK client and > contact the ZK service directly at the beginning of the connection lifecycle. > This has several benefits. ZK service need not be exposed to clients, and > their potential abuse, yet no benefit ZK provides the HBase server cluster is > compromised. Normalizing HBase client and ZK client timeout settings and > retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer > necessary. > And, from [~ghelmling]: There is an additional complication here for > token-based authentication. When a delegation token is used for SASL > authentication, the client uses the cluster ID obtained from Zookeeper to > select the token identifier to use. So there would also need to be some > Zookeeper-less, unauthenticated way to obtain the cluster ID as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23604) Usage of "registry" is unclear in the code.
[ https://issues.apache.org/jira/browse/HBASE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23604: - Fix Version/s: HBASE-18095 > Usage of "registry" is unclear in the code. > --- > > Key: HBASE-23604 > URL: https://issues.apache.org/jira/browse/HBASE-23604 > Project: HBase > Issue Type: Task > Components: Client >Affects Versions: master >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Minor > Fix For: HBASE-18095 > > > As [~stack] noted in the code review > https://github.com/apache/hbase/pull/954, the usage of registry in the client > code is not super clear. The ask here is to rename it something that makes > the context more clear. > Creating a separate jira because the patch touches a lots of files. I don't > want to mix it with the patch for HBASE-23305. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23604) Usage of "registry" is unclear in the code.
[ https://issues.apache.org/jira/browse/HBASE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23604: - Resolution: Fixed Status: Resolved (was: Patch Available) Merged to feature branch. > Usage of "registry" is unclear in the code. > --- > > Key: HBASE-23604 > URL: https://issues.apache.org/jira/browse/HBASE-23604 > Project: HBase > Issue Type: Task > Components: Client >Affects Versions: master >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Minor > Fix For: HBASE-18095 > > > As [~stack] noted in the code review > https://github.com/apache/hbase/pull/954, the usage of registry in the client > code is not super clear. The ask here is to rename it something that makes > the context more clear. > Creating a separate jira because the patch touches a lots of files. I don't > want to mix it with the patch for HBASE-23305. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #957: HBASE-23604: Clarify AsyncRegistry usage in the code.
ndimiduk merged pull request #957: HBASE-23604: Clarify AsyncRegistry usage in the code. URL: https://github.com/apache/hbase/pull/957 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
ndimiduk commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing URL: https://github.com/apache/hbase/pull/917#issuecomment-570715907 Scoped way back the changes. Using the existing roundRobinAssignment strategy to assign the new regions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23604) Usage of "registry" is unclear in the code.
[
https://issues.apache.org/jira/browse/HBASE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007792#comment-17007792
]
HBase QA commented on HBASE-23604:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
1s{color} | {color:green} The patch appears to include 23 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile
Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
24s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
31s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} hbase-client: The patch generated 0 new + 14
unchanged - 2 fixed = 14 total (was 16) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} The patch passed checkstyle in hbase-server {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 4s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
59s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}157m
21s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
6s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}223m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsy
[GitHub] [hbase] Apache-HBase commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code.
Apache-HBase commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code. URL: https://github.com/apache/hbase/pull/957#issuecomment-570714356 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 1s | The patch appears to include 23 new or modified test files. | ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 16s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | compile | 1m 16s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | checkstyle | 1m 49s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +1 :green_heart: | shadedjars | 4m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | HBASE-18095/client-locate-meta-no-zookeeper passed | | +0 :ok: | spotbugs | 4m 24s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 31s | HBASE-18095/client-locate-meta-no-zookeeper passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 0s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 31s | hbase-client: The patch generated 0 new + 14 unchanged - 2 fixed = 14 total (was 16) | | +1 :green_heart: | checkstyle | 1m 17s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 4s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | | +1 :green_heart: | findbugs | 5m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 59s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 157m 21s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | | | | 223m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-957/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/957 | | JIRA Issue | HBASE-23604 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 49619b10215a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-957/out/precommit/personality/provided.sh | | git revision | HBASE-18095/client-locate-meta-no-zookeeper / dffa9be899 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-957/6/testReport/ | | Max. process+thread count | 4779 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-957/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007790#comment-17007790
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #28 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/28/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/28//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/28//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/28//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
Apache-HBase commented on issue #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing URL: https://github.com/apache/hbase/pull/917#issuecomment-570707821 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 7s | https://github.com/apache/hbase/pull/917 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/917 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-917/5/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-2.
--
Resolution: Fixed
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-2:
-
Fix Version/s: 2.1.9
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #986: HBASE-23333 Include Call.toShortString() in sendCall exceptions
ndimiduk merged pull request #986: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/986 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23585) MetricsRegionServerWrapperImpl.getL1CacheHitCount always returns 200
[
https://issues.apache.org/jira/browse/HBASE-23585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007751#comment-17007751
]
Hudson commented on HBASE-23585:
Results for branch branch-1.4
[build #1136 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1136/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1136//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1136//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1136//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> MetricsRegionServerWrapperImpl.getL1CacheHitCount always returns 200
>
>
> Key: HBASE-23585
> URL: https://issues.apache.org/jira/browse/HBASE-23585
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 1.4.12
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Major
> Fix For: 1.6.1, 1.4.13
>
>
> Looks like it was copied from a UT class and forgot to change it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #986: HBASE-23333 Include Call.toShortString() in sendCall exceptions
Apache-HBase commented on issue #986: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/986#issuecomment-570689346 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-2.1 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 35s | branch-2.1 passed | | +1 :green_heart: | compile | 0m 46s | branch-2.1 passed | | +1 :green_heart: | checkstyle | 1m 6s | branch-2.1 passed | | +1 :green_heart: | shadedjars | 4m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | branch-2.1 passed | | +0 :ok: | spotbugs | 0m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 0s | branch-2.1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 19s | the patch passed | | +1 :green_heart: | compile | 0m 48s | the patch passed | | +1 :green_heart: | javac | 0m 48s | the patch passed | | +1 :green_heart: | checkstyle | 0m 25s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 37s | hbase-client: The patch generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 22m 2s | Patch does not cause any errors with Hadoop 2.7.7 2.8.5 or 3.0.3 3.1.2. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | | +1 :green_heart: | findbugs | 2m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 0s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 3m 32s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 66m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-986/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/986 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4ea48860b125 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-986/out/precommit/personality/provided.sh | | git revision | branch-2.1 / 4dfc170c08 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-986/1/testReport/ | | Max. process+thread count | 254 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-986/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-2:
-
Fix Version/s: 2.2.3
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #985: HBASE-23333 Include Call.toShortString() in sendCall exceptions
ndimiduk merged pull request #985: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/985 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-2:
-
Fix Version/s: 2.3.0
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #984: HBASE-23333 Include Call.toShortString() in sendCall exceptions
ndimiduk merged pull request #984: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/984 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #985: HBASE-23333 Include Call.toShortString() in sendCall exceptions
Apache-HBase commented on issue #985: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/985#issuecomment-570687879 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-2.2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 30s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 46s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 1m 4s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2.2 passed | | +0 :ok: | spotbugs | 0m 48s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 3s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 16s | the patch passed | | +1 :green_heart: | compile | 0m 48s | the patch passed | | +1 :green_heart: | javac | 0m 48s | the patch passed | | +1 :green_heart: | checkstyle | 0m 25s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 37s | hbase-client: The patch generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 23s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | | +1 :green_heart: | findbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 54s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 3m 29s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 60m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-985/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/985 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9cdd851ecaa4 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-985/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 8d22b7ee11 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-985/1/testReport/ | | Max. process+thread count | 319 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-985/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #984: HBASE-23333 Include Call.toShortString() in sendCall exceptions
Apache-HBase commented on issue #984: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/984#issuecomment-570687297 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 15s | branch-2 passed | | +1 :green_heart: | compile | 0m 50s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 2s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-2 passed | | +0 :ok: | spotbugs | 0m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 59s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 57s | the patch passed | | +1 :green_heart: | compile | 0m 50s | the patch passed | | +1 :green_heart: | javac | 0m 50s | the patch passed | | +1 :green_heart: | checkstyle | 0m 25s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 35s | hbase-client: The patch generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 19s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 46s | the patch passed | | +1 :green_heart: | findbugs | 2m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 7s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 3m 44s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 58m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-984/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/984 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6b12d4ae9d38 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-984/out/precommit/personality/provided.sh | | git revision | branch-2 / c44a5c47dd | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-984/1/testReport/ | | Max. process+thread count | 309 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-984/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362926824
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import static org.apache.hadoop.hbase.HConstants.MASTER_ADDRS_DEFAULT;
+import static org.apache.hadoop.hbase.HConstants.MASTER_ADDRS_KEY;
+import static
org.apache.hadoop.hbase.HConstants.MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT;
+import static
org.apache.hadoop.hbase.HConstants.MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.RegionLocations;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.exceptions.MasterRegistryFetchException;
+import org.apache.hadoop.hbase.ipc.HBaseRpcController;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.yetus.audience.InterfaceAudience;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Function;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.common.net.HostAndPort;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.ClientMetaService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetActiveMasterRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetActiveMasterResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaRegionLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaRegionLocationsResponse;
+
+/**
+ * Master based registry implementation. Makes RPCs to the configured master
addresses from config
+ * {@value org.apache.hadoop.hbase.HConstants#MASTER_ADDRS_KEY}.
+ *
+ * It supports hedged reads, which can be enabled by setting
+ * {@value
org.apache.hadoop.hbase.HConstants#MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY} to
True. Fan
+ * out the requests batch is controlled by
+ * {@value
org.apache.hadoop.hbase.HConstants#HBASE_RPCS_HEDGED_REQS_FANOUT_KEY}.
+ *
+ * TODO: Handle changes to the configuration dynamically without having to
restart the client.
+ */
+@InterfaceAudience.Private
+public class MasterRegistry implements AsyncRegistry {
+ private static final String MASTER_ADDRS_CONF_SEPARATOR = ",";
+
+ // Configured list of masters to probe the meta information from.
+ private final List masterServers;
+
+ // RPC client used to talk to the masters.
+ private final RpcClient rpcClient;
+ private final RpcControllerFactory rpcControllerFactory;
+ private final int rpcTimeoutMs;
+
+ MasterRegistry(Configuration conf) {
+boolean hedgedReadsEnabled =
conf.getBoolean(MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY,
+MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT);
+Configuration finalConf;
+if (!hedgedReadsEnabled) {
+ // If hedged reads are disabled, it is equivalent to setting a fan out
of 1. We make a copy of
+ // the configuration so that other places reusing this reference is not
affected.
+ finalConf = new Configuration(conf);
+ finalConf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, 1);
+} else {
+ final
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362930798
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java
##
@@ -44,35 +45,38 @@
import org.apache.hadoop.hbase.HTestConst;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.MiniHBaseCluster;
-import org.apache.hadoop.hbase.exceptions.DeserializationException;
-import org.apache.hadoop.hbase.filter.FilterBase;
-import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.StartMiniClusterOption;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.TableNotFoundException;
+import org.apache.hadoop.hbase.TestTableName;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
import org.apache.hadoop.hbase.filter.BinaryComparator;
import org.apache.hadoop.hbase.filter.ColumnPrefixFilter;
import org.apache.hadoop.hbase.filter.ColumnRangeFilter;
+import org.apache.hadoop.hbase.filter.FilterBase;
import org.apache.hadoop.hbase.filter.QualifierFilter;
import org.apache.hadoop.hbase.regionserver.HRegionServer;
import org.apache.hadoop.hbase.testclassification.ClientTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
-import org.junit.After;
import org.junit.AfterClass;
-import org.junit.Before;
-import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-import org.junit.rules.TestName;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
/**
- * A client-side test, mostly testing scanners with various parameters.
+ * A client-side test, mostly testing scanners with various parameters.
Parameterized on different
+ * registry implementations.
*/
@Category({MediumTests.class, ClientTests.class})
+@RunWith(Parameterized.class)
Review comment:
ya, I think its per parameterized run, not across all of them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362930568
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
##
@@ -2642,7 +2676,7 @@ public void testJiraTest1182() throws Exception {
*/
@Test
public void testJiraTest52() throws Exception {
-final TableName tableName = TableName.valueOf(name.getMethodName());
+final TableName tableName = name.getTableName();
Review comment:
Missed this too. Not an expert on junit, but I think its possible with
chaining. That is probably a bit more cleaner way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362942554
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HedgedRpcChannel.java
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.PrettyPrinter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+/**
+ * A non-blocking implementation of RpcChannel that hedges requests to
multiple service end points.
+ * First received response is returned to the caller. This abstracts out the
logic needed to batch
+ * requests to multiple end points underneath and presents itself as a single
logical RpcChannel to
+ * the client.
+ *
+ * Hedging Details:
+ * ---
+ * - Hedging of RPCs happens in multiple batches. In each iteration, we select
a 'batch' of address
+ * end points to make the call to. We do multiple iterations until we get a
proper response to the
+ * rpc call or all the service addresses are exhausted, which ever happens
first.
+ *
+ * - We randomize the addresses up front so that the batch order per client is
non deterministic.
+ * This avoids hot spots on the service side. The size of each batch is
controlled via 'fanOutSize'.
+ * Higher fanOutSize implies we make more rpc calls in a single batch. One
needs to mindful of the
+ * load on the client and server side when configuring the fan out.
+ *
+ * - In a happy case, once we receive a response from one end point, we cancel
all the
+ * other inflight rpcs in the same batch and return the response to the
caller. If we do not get a
+ * valid response from any address end point, we propagate the error back to
the caller.
+ *
+ * - Rpc timeouts are applied to every hedged rpc.
+ *
+ * - Callers need to be careful about what rpcs they are trying to hedge. Not
every kind of call can
+ * be hedged (for example: cluster state changing rpcs).
+ *
+ * (TODO) Retries and Adaptive hedging policy:
+ * --
+ *
+ * - No retries are handled at the channel level. Retries can be built in
upper layers. However the
+ * question is, do we even need retries? Hedging in fact is a substitute for
retries.
+ *
+ * - Clearly hedging puts more load on the service side. To mitigate this, we
can make the hedging
+ * policy more adaptive. In most happy cases, the rpcs from the first few end
points should return
+ * right away (especially short lived rpcs, that do not take up much time). In
such cases, hedging
+ * is not needed. So, the idea is to make this request pattern pluggable so
that the requests are
+ * hedged only when needed.
+ */
+class HedgedRpcChannel implements RpcChannel {
+ private static final Logger LOG =
LoggerFactory.getLogger(HedgedRpcChannel.class);
+
+ private final AbstractRpcClient rpcClient;
+ // List of service addresses to hedge the requests to.
+ private final List addrs;
+ private final User ticket;
+ private final int rpcTimeout;
+ // Controls the size of request fan out (number of rpcs per a single batch).
+ private final int fanOutSize;
+
+ /**
+ * A simple rpc call back implementation to notify the batch context if any
rpc is successful.
+ */
+ private static class BatchRpcCtxCallBack implements RpcCallback {
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362928069
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
##
@@ -106,23 +108,27 @@
import org.apache.hadoop.hbase.util.NonRepeatedEnvironmentEdge;
import org.apache.hadoop.hbase.util.TableDescriptorChecker;
import org.junit.AfterClass;
-import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-import org.junit.rules.TestName;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
/**
* Run tests that use the HBase clients; {@link Table}.
* Sets up the HBase mini cluster once at start and runs through all client
tests.
* Each creates a table named for the method and does its stuff against that.
+ *
+ * Parameterized to run with different registry implementations.
*/
@Category({LargeTests.class, ClientTests.class})
@SuppressWarnings ("deprecation")
+@RunWith(Parameterized.class)
Review comment:
Missed this in the first iteration. I think it applies to each parameter
instance. Thats the behavior I saw in my local tests. Even if one parameterized
run times out, others start and finish.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #954: HBASE-23305: Master based registry implementation
bharathv commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362939851
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HedgedRpcChannel.java
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.PrettyPrinter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+/**
+ * A non-blocking implementation of RpcChannel that hedges requests to
multiple service end points.
+ * First received response is returned to the caller. This abstracts out the
logic needed to batch
+ * requests to multiple end points underneath and presents itself as a single
logical RpcChannel to
+ * the client.
+ *
+ * Hedging Details:
+ * ---
+ * - Hedging of RPCs happens in multiple batches. In each iteration, we select
a 'batch' of address
+ * end points to make the call to. We do multiple iterations until we get a
proper response to the
+ * rpc call or all the service addresses are exhausted, which ever happens
first.
+ *
+ * - We randomize the addresses up front so that the batch order per client is
non deterministic.
+ * This avoids hot spots on the service side. The size of each batch is
controlled via 'fanOutSize'.
+ * Higher fanOutSize implies we make more rpc calls in a single batch. One
needs to mindful of the
+ * load on the client and server side when configuring the fan out.
+ *
+ * - In a happy case, once we receive a response from one end point, we cancel
all the
+ * other inflight rpcs in the same batch and return the response to the
caller. If we do not get a
+ * valid response from any address end point, we propagate the error back to
the caller.
+ *
+ * - Rpc timeouts are applied to every hedged rpc.
+ *
+ * - Callers need to be careful about what rpcs they are trying to hedge. Not
every kind of call can
+ * be hedged (for example: cluster state changing rpcs).
+ *
+ * (TODO) Retries and Adaptive hedging policy:
+ * --
+ *
+ * - No retries are handled at the channel level. Retries can be built in
upper layers. However the
+ * question is, do we even need retries? Hedging in fact is a substitute for
retries.
+ *
+ * - Clearly hedging puts more load on the service side. To mitigate this, we
can make the hedging
+ * policy more adaptive. In most happy cases, the rpcs from the first few end
points should return
+ * right away (especially short lived rpcs, that do not take up much time). In
such cases, hedging
+ * is not needed. So, the idea is to make this request pattern pluggable so
that the requests are
+ * hedged only when needed.
+ */
+class HedgedRpcChannel implements RpcChannel {
+ private static final Logger LOG =
LoggerFactory.getLogger(HedgedRpcChannel.class);
+
+ private final AbstractRpcClient rpcClient;
+ // List of service addresses to hedge the requests to.
+ private final List addrs;
+ private final User ticket;
+ private final int rpcTimeout;
+ // Controls the size of request fan out (number of rpcs per a single batch).
+ private final int fanOutSize;
+
+ /**
+ * A simple rpc call back implementation to notify the batch context if any
rpc is successful.
+ */
+ private static class BatchRpcCtxCallBack implements RpcCallback {
[GitHub] [hbase] ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
ndimiduk commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r362929553
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -86,74 +87,176 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to
hbase:meta.
* Does not assign.
*/
- void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+ void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
if (holes.isEmpty()) {
- LOG.debug("No holes.");
+ LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
return;
}
-for (Pair p: holes) {
- RegionInfo ri = getHoleCover(p);
- if (ri == null) {
-continue;
- }
- Configuration configuration = this.masterServices.getConfiguration();
- HRegion.createRegionDir(configuration, ri,
FSUtils.getRootDir(configuration));
- // If an error here, then we'll have a region in the filesystem but not
- // in hbase:meta (if the below fails). Should be able to rerun the fix.
- // Add to hbase:meta and then update in-memory state so it knows of new
- // Region; addRegionToMeta adds region and adds a state column set to
CLOSED.
- MetaTableAccessor.addRegionToMeta(this.masterServices.getConnection(),
ri);
- this.masterServices.getAssignmentManager().getRegionStates().
- updateRegionState(ri, RegionState.State.CLOSED);
- LOG.info("Fixed hole by adding {} in CLOSED state; region NOT assigned
(assign to ONLINE).",
- ri);
-}
+
+LOG.info("Identified {} region holes to fix.", holes.size());
+
+final List newRegionInfos = mapHolesToRegionInfos(holes);
+final List newRegionDirectories =
+ createRegionDirectories(masterServices, newRegionInfos);
+final List newMetaEntries = createMetaEntries(masterServices,
newRegionDirectories);
+final List scheduledProcedures =
+
masterServices.getAssignmentManager().createAssignProceduresBestEffort(newMetaEntries);
+LOG.info(
+ "Scheduled {}/{} new regions for assignment.",
scheduledProcedures.size(), holes.size());
}
/**
- * @return Calculated RegionInfo that covers the hole hole
+ * Create a new {@link RegionInfo} corresponding to each provided "hole"
pair.
*/
- private RegionInfo getHoleCover(Pair hole) {
-RegionInfo holeCover = null;
-RegionInfo left = hole.getFirst();
-RegionInfo right = hole.getSecond();
+ private static List mapHolesToRegionInfos(
+final List> holes) {
+final List newRegionInfos = holes.stream()
+ .map(MetaFixer::getHoleCover)
+ .filter(Optional::isPresent)
+ .map(Optional::get)
+ .collect(Collectors.toList());
+LOG.debug("Constructed {}/{} RegionInfo descriptors corresponding to
identified holes.",
+ newRegionInfos.size(), holes.size());
+return newRegionInfos;
+ }
+
+ /**
+ * @return Attempts to calculate a new {@link RegionInfo} that covers the
region range described
+ * in {@code hole}.
+ */
+ private static Optional getHoleCover(Pair hole) {
+final RegionInfo left = hole.getFirst();
+final RegionInfo right = hole.getSecond();
+
if (left.getTable().equals(right.getTable())) {
// Simple case.
if (Bytes.compareTo(left.getEndKey(), right.getStartKey()) >= 0) {
-LOG.warn("Skipping hole fix; left-side endKey is not less than
right-side startKey; " +
-"left=<{}>, right=<{}>", left, right);
-return holeCover;
- }
- holeCover = buildRegionInfo(left.getTable(), left.getEndKey(),
right.getStartKey());
-} else {
- boolean leftUndefined = left.equals(RegionInfo.UNDEFINED);
- boolean rightUnefined = right.equals(RegionInfo.UNDEFINED);
- boolean last = left.isLast();
- boolean first = right.isFirst();
- if (leftUndefined && rightUnefined) {
-LOG.warn("Skipping hole fix; both the hole left-side and right-side
RegionInfos are " +
-"UNDEFINED; left=<{}>, right=<{}>", left, right);
-return holeCover;
- }
- if (leftUndefined || last) {
-holeCover = buildRegionInfo(right.getTable(),
HConstants.EMPTY_START_ROW,
-right.getStartKey());
- } else if (rightUnefined || first) {
-holeCover = buildRegionInfo(left.getTable(), left.getEndKey(),
HConstants.EMPTY_END_ROW);
- } else {
-LOG.warn("Skipping hole fix; don't know what to do with left=<{}>,
right=<{}>",
-left, right);
-return holeCover;
+LOG.warn("Skipping hole fix; left-side endKey is not less than
right-side startKey;"
+ + " left=<{}>, right=<{}>", left, right);
+return Optional.empty();
}
+ return Optiona
[jira] [Updated] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-2:
-
Fix Version/s: 3.0.0
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0
>
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #986: HBASE-23333 Include Call.toShortString() in sendCall exceptions
ndimiduk opened a new pull request #986: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/986 Backport #872 to branch-2.1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk merged pull request #872: HBASE-23333 Provide call context around timeouts and other failure scenarios
ndimiduk merged pull request #872: HBASE-2 Provide call context around timeouts and other failure scenarios URL: https://github.com/apache/hbase/pull/872 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #985: HBASE-23333 Include Call.toShortString() in sendCall exceptions
ndimiduk opened a new pull request #985: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/985 Backport #872 to branch-2.2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #984: HBASE-23333 Include Call.toShortString() in sendCall exceptions
ndimiduk opened a new pull request #984: HBASE-2 Include Call.toShortString() in sendCall exceptions URL: https://github.com/apache/hbase/pull/984 Backport #872 to branch-2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #872: HBASE-23333 Provide call context around timeouts and other failure scenarios
Apache-HBase commented on issue #872: HBASE-2 Provide call context around timeouts and other failure scenarios URL: https://github.com/apache/hbase/pull/872#issuecomment-570662446 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 21s | master passed | | +1 :green_heart: | compile | 0m 49s | master passed | | +1 :green_heart: | checkstyle | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 4m 37s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | master passed | | +0 :ok: | spotbugs | 1m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 54s | the patch passed | | +1 :green_heart: | compile | 0m 49s | the patch passed | | +1 :green_heart: | javac | 0m 49s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 31s | hbase-client: The patch generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 38s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | | +1 :green_heart: | findbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 15s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 58s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 58m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-872/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/872 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 79f2b61f1698 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-872/out/precommit/personality/provided.sh | | git revision | master / 85ecdd10a9 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-872/4/testReport/ | | Max. process+thread count | 286 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-872/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23633) Find a way to handle the corrupt recovered hfiles
[ https://issues.apache.org/jira/browse/HBASE-23633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007690#comment-17007690 ] Michael Stack commented on HBASE-23633: --- What I want is an indication in Master log that a Region is not opening because hfiles are corrupt or we have dangling references. Currently it just says fail but not why (smile). > Find a way to handle the corrupt recovered hfiles > - > > Key: HBASE-23633 > URL: https://issues.apache.org/jira/browse/HBASE-23633 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Priority: Major > > Copy the comment from PR review. > > If the file is a corrupt HFile, an exception will be thrown here, which will > cause the region to fail to open. > Maybe we can add a new parameter to control whether to skip the exception, > similar to recover edits which has a parameter > "hbase.hregion.edits.replay.skip.errors"; > > Regions that can't be opened because of detached References or corrupt hfiles > are a fact-of-life. We need work on this issue. This will be a new variant on > the problem -- i.e. bad recovered hfiles. > On adding a config to ignore bad files and just open, thats a bit dangerous > as per @infraio as it could mean silent data loss. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23634) Enable "Split WAL to HFile" by default
[ https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007689#comment-17007689 ] Michael Stack commented on HBASE-23634: --- +1 > Enable "Split WAL to HFile" by default > -- > > Key: HBASE-23634 > URL: https://issues.apache.org/jira/browse/HBASE-23634 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.3.0 >Reporter: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23634) Enable "Split WAL to HFile" by default
[ https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007689#comment-17007689 ] Michael Stack edited comment on HBASE-23634 at 1/3/20 6:43 PM: --- +1 Made it a blocker on 2.3.0 was (Author: stack): +1 > Enable "Split WAL to HFile" by default > -- > > Key: HBASE-23634 > URL: https://issues.apache.org/jira/browse/HBASE-23634 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.3.0 >Reporter: Guanghao Zhang >Priority: Blocker > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23634) Enable "Split WAL to HFile" by default
[ https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-23634: -- Priority: Blocker (was: Major) > Enable "Split WAL to HFile" by default > -- > > Key: HBASE-23634 > URL: https://issues.apache.org/jira/browse/HBASE-23634 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.3.0 >Reporter: Guanghao Zhang >Priority: Blocker > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23369) Auto-close 'unknown' Regions reported as OPEN on RegionServers
[
https://issues.apache.org/jira/browse/HBASE-23369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007687#comment-17007687
]
Michael Stack commented on HBASE-23369:
---
I've been running cluster tests with this patch in place the last few weeks.
Good for tampering down the mayhem when cluster goes haywire when overdriven by
sustained loading causing Master lose accounting.
> Auto-close 'unknown' Regions reported as OPEN on RegionServers
> --
>
> Key: HBASE-23369
> URL: https://issues.apache.org/jira/browse/HBASE-23369
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> In old days, if a RegionServer reported a variance that didn't agree w/
> Master view of the cluster, we'd kill the RegionServer.
> Lately, in tests that overrun a cluster, after a sustained high-load, Master
> can start failing its updates against Meta (CallQueueTooBigException <= More
> on this later). It then can lose proper accounting of all Region members. One
> variant has a RegionServer reporting its list of open Regions to the Master
> and the Master doesn't 'know' of a particular Region or the Master may know
> the Region but expects it open on another RegionServer.
> Here is an example of how it looks each time RS reports:
> {code}
> 2019-12-03 07:07:00,757 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode
> but reported ONLINE at server.example.org,16020,1575354666245
> (inServerRegionList=false).
> 2019-12-03 07:07:03,793 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode
> but reported ONLINE at server.example.org,16020,1575354666245
> (inServerRegionList=false).
> {code}
> Will also show as an 'inconsistency' in the 'HBCK' tab on the Master UI.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362913375
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HedgedRpcChannel.java
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.PrettyPrinter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+/**
+ * A non-blocking implementation of RpcChannel that hedges requests to
multiple service end points.
+ * First received response is returned to the caller. This abstracts out the
logic needed to batch
+ * requests to multiple end points underneath and presents itself as a single
logical RpcChannel to
+ * the client.
+ *
+ * Hedging Details:
+ * ---
+ * - Hedging of RPCs happens in multiple batches. In each iteration, we select
a 'batch' of address
+ * end points to make the call to. We do multiple iterations until we get a
proper response to the
+ * rpc call or all the service addresses are exhausted, which ever happens
first.
+ *
+ * - We randomize the addresses up front so that the batch order per client is
non deterministic.
+ * This avoids hot spots on the service side. The size of each batch is
controlled via 'fanOutSize'.
+ * Higher fanOutSize implies we make more rpc calls in a single batch. One
needs to mindful of the
+ * load on the client and server side when configuring the fan out.
+ *
+ * - In a happy case, once we receive a response from one end point, we cancel
all the
+ * other inflight rpcs in the same batch and return the response to the
caller. If we do not get a
+ * valid response from any address end point, we propagate the error back to
the caller.
+ *
+ * - Rpc timeouts are applied to every hedged rpc.
+ *
+ * - Callers need to be careful about what rpcs they are trying to hedge. Not
every kind of call can
+ * be hedged (for example: cluster state changing rpcs).
+ *
+ * (TODO) Retries and Adaptive hedging policy:
+ * --
+ *
+ * - No retries are handled at the channel level. Retries can be built in
upper layers. However the
+ * question is, do we even need retries? Hedging in fact is a substitute for
retries.
+ *
+ * - Clearly hedging puts more load on the service side. To mitigate this, we
can make the hedging
+ * policy more adaptive. In most happy cases, the rpcs from the first few end
points should return
+ * right away (especially short lived rpcs, that do not take up much time). In
such cases, hedging
+ * is not needed. So, the idea is to make this request pattern pluggable so
that the requests are
+ * hedged only when needed.
+ */
+class HedgedRpcChannel implements RpcChannel {
+ private static final Logger LOG =
LoggerFactory.getLogger(HedgedRpcChannel.class);
+
+ private final AbstractRpcClient rpcClient;
+ // List of service addresses to hedge the requests to.
+ private final List addrs;
+ private final User ticket;
+ private final int rpcTimeout;
+ // Controls the size of request fan out (number of rpcs per a single batch).
+ private final int fanOutSize;
+
+ /**
+ * A simple rpc call back implementation to notify the batch context if any
rpc is successful.
+ */
+ private static class BatchRpcCtxCallBack implements RpcCallback {
[jira] [Resolved] (HBASE-23369) Auto-close 'unknown' Regions reported as OPEN on RegionServers
[
https://issues.apache.org/jira/browse/HBASE-23369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23369.
---
Fix Version/s: 2.3.0
3.0.0
Hadoop Flags: Incompatible change
Release Note: If a RegionServer reports a Region as OPEN in disagreement
with Master's status on the Region, the Master now tells the RegionServer to
silently close the Region.
Assignee: Michael Stack
Resolution: Fixed
Merged to branch-2 and. master branch.
I think this belongs in branch-2.2 too. Shout and I'll pull it back.
> Auto-close 'unknown' Regions reported as OPEN on RegionServers
> --
>
> Key: HBASE-23369
> URL: https://issues.apache.org/jira/browse/HBASE-23369
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> In old days, if a RegionServer reported a variance that didn't agree w/
> Master view of the cluster, we'd kill the RegionServer.
> Lately, in tests that overrun a cluster, after a sustained high-load, Master
> can start failing its updates against Meta (CallQueueTooBigException <= More
> on this later). It then can lose proper accounting of all Region members. One
> variant has a RegionServer reporting its list of open Regions to the Master
> and the Master doesn't 'know' of a particular Region or the Master may know
> the Region but expects it open on another RegionServer.
> Here is an example of how it looks each time RS reports:
> {code}
> 2019-12-03 07:07:00,757 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode
> but reported ONLINE at server.example.org,16020,1575354666245
> (inServerRegionList=false).
> 2019-12-03 07:07:03,793 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: No
> t1,08f5c285,1573094375485.ee78a0c951c1c902d8f3f3912394a0e5. RegionStateNode
> but reported ONLINE at server.example.org,16020,1575354666245
> (inServerRegionList=false).
> {code}
> Will also show as an 'inconsistency' in the 'HBCK' tab on the Master UI.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362913523
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HedgedRpcChannel.java
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.PrettyPrinter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+/**
+ * A non-blocking implementation of RpcChannel that hedges requests to
multiple service end points.
+ * First received response is returned to the caller. This abstracts out the
logic needed to batch
+ * requests to multiple end points underneath and presents itself as a single
logical RpcChannel to
+ * the client.
+ *
+ * Hedging Details:
+ * ---
+ * - Hedging of RPCs happens in multiple batches. In each iteration, we select
a 'batch' of address
+ * end points to make the call to. We do multiple iterations until we get a
proper response to the
+ * rpc call or all the service addresses are exhausted, which ever happens
first.
+ *
+ * - We randomize the addresses up front so that the batch order per client is
non deterministic.
+ * This avoids hot spots on the service side. The size of each batch is
controlled via 'fanOutSize'.
+ * Higher fanOutSize implies we make more rpc calls in a single batch. One
needs to mindful of the
+ * load on the client and server side when configuring the fan out.
+ *
+ * - In a happy case, once we receive a response from one end point, we cancel
all the
+ * other inflight rpcs in the same batch and return the response to the
caller. If we do not get a
+ * valid response from any address end point, we propagate the error back to
the caller.
+ *
+ * - Rpc timeouts are applied to every hedged rpc.
+ *
+ * - Callers need to be careful about what rpcs they are trying to hedge. Not
every kind of call can
+ * be hedged (for example: cluster state changing rpcs).
+ *
+ * (TODO) Retries and Adaptive hedging policy:
+ * --
+ *
+ * - No retries are handled at the channel level. Retries can be built in
upper layers. However the
+ * question is, do we even need retries? Hedging in fact is a substitute for
retries.
+ *
+ * - Clearly hedging puts more load on the service side. To mitigate this, we
can make the hedging
+ * policy more adaptive. In most happy cases, the rpcs from the first few end
points should return
+ * right away (especially short lived rpcs, that do not take up much time). In
such cases, hedging
+ * is not needed. So, the idea is to make this request pattern pluggable so
that the requests are
+ * hedged only when needed.
+ */
+class HedgedRpcChannel implements RpcChannel {
+ private static final Logger LOG =
LoggerFactory.getLogger(HedgedRpcChannel.class);
+
+ private final AbstractRpcClient rpcClient;
+ // List of service addresses to hedge the requests to.
+ private final List addrs;
+ private final User ticket;
+ private final int rpcTimeout;
+ // Controls the size of request fan out (number of rpcs per a single batch).
+ private final int fanOutSize;
+
+ /**
+ * A simple rpc call back implementation to notify the batch context if any
rpc is successful.
+ */
+ private static class BatchRpcCtxCallBack implements RpcCallback {
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362908992
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import static org.apache.hadoop.hbase.HConstants.MASTER_ADDRS_DEFAULT;
+import static org.apache.hadoop.hbase.HConstants.MASTER_ADDRS_KEY;
+import static
org.apache.hadoop.hbase.HConstants.MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT;
+import static
org.apache.hadoop.hbase.HConstants.MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.RegionLocations;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.exceptions.MasterRegistryFetchException;
+import org.apache.hadoop.hbase.ipc.HBaseRpcController;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.yetus.audience.InterfaceAudience;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Function;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.common.net.HostAndPort;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.ClientMetaService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetActiveMasterRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetActiveMasterResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaRegionLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaRegionLocationsResponse;
+
+/**
+ * Master based registry implementation. Makes RPCs to the configured master
addresses from config
+ * {@value org.apache.hadoop.hbase.HConstants#MASTER_ADDRS_KEY}.
+ *
+ * It supports hedged reads, which can be enabled by setting
+ * {@value
org.apache.hadoop.hbase.HConstants#MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY} to
True. Fan
+ * out the requests batch is controlled by
+ * {@value
org.apache.hadoop.hbase.HConstants#HBASE_RPCS_HEDGED_REQS_FANOUT_KEY}.
+ *
+ * TODO: Handle changes to the configuration dynamically without having to
restart the client.
+ */
+@InterfaceAudience.Private
+public class MasterRegistry implements AsyncRegistry {
+ private static final String MASTER_ADDRS_CONF_SEPARATOR = ",";
+
+ // Configured list of masters to probe the meta information from.
+ private final List masterServers;
+
+ // RPC client used to talk to the masters.
+ private final RpcClient rpcClient;
+ private final RpcControllerFactory rpcControllerFactory;
+ private final int rpcTimeoutMs;
+
+ MasterRegistry(Configuration conf) {
+boolean hedgedReadsEnabled =
conf.getBoolean(MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY,
+MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT);
+Configuration finalConf;
+if (!hedgedReadsEnabled) {
+ // If hedged reads are disabled, it is equivalent to setting a fan out
of 1. We make a copy of
+ // the configuration so that other places reusing this reference is not
affected.
+ finalConf = new Configuration(conf);
+ finalConf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, 1);
+} else {
+ final
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362908253
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import static org.apache.hadoop.hbase.HConstants.MASTER_ADDRS_DEFAULT;
+import static org.apache.hadoop.hbase.HConstants.MASTER_ADDRS_KEY;
+import static
org.apache.hadoop.hbase.HConstants.MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT;
+import static
org.apache.hadoop.hbase.HConstants.MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.RegionLocations;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.exceptions.MasterRegistryFetchException;
+import org.apache.hadoop.hbase.ipc.HBaseRpcController;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.yetus.audience.InterfaceAudience;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+import org.apache.hbase.thirdparty.com.google.common.base.Function;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.common.net.HostAndPort;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.ClientMetaService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetActiveMasterRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetActiveMasterResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaRegionLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaRegionLocationsResponse;
+
+/**
+ * Master based registry implementation. Makes RPCs to the configured master
addresses from config
+ * {@value org.apache.hadoop.hbase.HConstants#MASTER_ADDRS_KEY}.
+ *
+ * It supports hedged reads, which can be enabled by setting
+ * {@value
org.apache.hadoop.hbase.HConstants#MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY} to
True. Fan
+ * out the requests batch is controlled by
+ * {@value
org.apache.hadoop.hbase.HConstants#HBASE_RPCS_HEDGED_REQS_FANOUT_KEY}.
+ *
+ * TODO: Handle changes to the configuration dynamically without having to
restart the client.
+ */
+@InterfaceAudience.Private
+public class MasterRegistry implements AsyncRegistry {
+ private static final String MASTER_ADDRS_CONF_SEPARATOR = ",";
+
+ // Configured list of masters to probe the meta information from.
+ private final List masterServers;
+
+ // RPC client used to talk to the masters.
+ private final RpcClient rpcClient;
+ private final RpcControllerFactory rpcControllerFactory;
+ private final int rpcTimeoutMs;
+
+ MasterRegistry(Configuration conf) {
+boolean hedgedReadsEnabled =
conf.getBoolean(MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY,
+MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT);
+Configuration finalConf;
+if (!hedgedReadsEnabled) {
+ // If hedged reads are disabled, it is equivalent to setting a fan out
of 1. We make a copy of
+ // the configuration so that other places reusing this reference is not
affected.
+ finalConf = new Configuration(conf);
+ finalConf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, 1);
+} else {
+ final
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362910060
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HedgedRpcChannel.java
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.PrettyPrinter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+/**
+ * A non-blocking implementation of RpcChannel that hedges requests to
multiple service end points.
+ * First received response is returned to the caller. This abstracts out the
logic needed to batch
+ * requests to multiple end points underneath and presents itself as a single
logical RpcChannel to
+ * the client.
+ *
+ * Hedging Details:
+ * ---
+ * - Hedging of RPCs happens in multiple batches. In each iteration, we select
a 'batch' of address
+ * end points to make the call to. We do multiple iterations until we get a
proper response to the
+ * rpc call or all the service addresses are exhausted, which ever happens
first.
+ *
+ * - We randomize the addresses up front so that the batch order per client is
non deterministic.
+ * This avoids hot spots on the service side. The size of each batch is
controlled via 'fanOutSize'.
+ * Higher fanOutSize implies we make more rpc calls in a single batch. One
needs to mindful of the
+ * load on the client and server side when configuring the fan out.
+ *
+ * - In a happy case, once we receive a response from one end point, we cancel
all the
+ * other inflight rpcs in the same batch and return the response to the
caller. If we do not get a
+ * valid response from any address end point, we propagate the error back to
the caller.
+ *
+ * - Rpc timeouts are applied to every hedged rpc.
+ *
+ * - Callers need to be careful about what rpcs they are trying to hedge. Not
every kind of call can
+ * be hedged (for example: cluster state changing rpcs).
+ *
+ * (TODO) Retries and Adaptive hedging policy:
+ * --
+ *
+ * - No retries are handled at the channel level. Retries can be built in
upper layers. However the
+ * question is, do we even need retries? Hedging in fact is a substitute for
retries.
+ *
+ * - Clearly hedging puts more load on the service side. To mitigate this, we
can make the hedging
+ * policy more adaptive. In most happy cases, the rpcs from the first few end
points should return
+ * right away (especially short lived rpcs, that do not take up much time). In
such cases, hedging
+ * is not needed. So, the idea is to make this request pattern pluggable so
that the requests are
+ * hedged only when needed.
+ */
+class HedgedRpcChannel implements RpcChannel {
+ private static final Logger LOG =
LoggerFactory.getLogger(HedgedRpcChannel.class);
+
+ private final AbstractRpcClient rpcClient;
+ // List of service addresses to hedge the requests to.
+ private final List addrs;
+ private final User ticket;
+ private final int rpcTimeout;
+ // Controls the size of request fan out (number of rpcs per a single batch).
+ private final int fanOutSize;
+
+ /**
+ * A simple rpc call back implementation to notify the batch context if any
rpc is successful.
+ */
+ private static class BatchRpcCtxCallBack implements RpcCallback {
[GitHub] [hbase] ndimiduk commented on a change in pull request #954: HBASE-23305: Master based registry implementation
ndimiduk commented on a change in pull request #954: HBASE-23305: Master based
registry implementation
URL: https://github.com/apache/hbase/pull/954#discussion_r362912035
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HedgedRpcChannel.java
##
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.ipc;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.PrettyPrinter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+/**
+ * A non-blocking implementation of RpcChannel that hedges requests to
multiple service end points.
+ * First received response is returned to the caller. This abstracts out the
logic needed to batch
+ * requests to multiple end points underneath and presents itself as a single
logical RpcChannel to
+ * the client.
+ *
+ * Hedging Details:
+ * ---
+ * - Hedging of RPCs happens in multiple batches. In each iteration, we select
a 'batch' of address
+ * end points to make the call to. We do multiple iterations until we get a
proper response to the
+ * rpc call or all the service addresses are exhausted, which ever happens
first.
+ *
+ * - We randomize the addresses up front so that the batch order per client is
non deterministic.
+ * This avoids hot spots on the service side. The size of each batch is
controlled via 'fanOutSize'.
+ * Higher fanOutSize implies we make more rpc calls in a single batch. One
needs to mindful of the
+ * load on the client and server side when configuring the fan out.
+ *
+ * - In a happy case, once we receive a response from one end point, we cancel
all the
+ * other inflight rpcs in the same batch and return the response to the
caller. If we do not get a
+ * valid response from any address end point, we propagate the error back to
the caller.
+ *
+ * - Rpc timeouts are applied to every hedged rpc.
+ *
+ * - Callers need to be careful about what rpcs they are trying to hedge. Not
every kind of call can
+ * be hedged (for example: cluster state changing rpcs).
+ *
+ * (TODO) Retries and Adaptive hedging policy:
+ * --
+ *
+ * - No retries are handled at the channel level. Retries can be built in
upper layers. However the
+ * question is, do we even need retries? Hedging in fact is a substitute for
retries.
+ *
+ * - Clearly hedging puts more load on the service side. To mitigate this, we
can make the hedging
+ * policy more adaptive. In most happy cases, the rpcs from the first few end
points should return
+ * right away (especially short lived rpcs, that do not take up much time). In
such cases, hedging
+ * is not needed. So, the idea is to make this request pattern pluggable so
that the requests are
+ * hedged only when needed.
+ */
+class HedgedRpcChannel implements RpcChannel {
+ private static final Logger LOG =
LoggerFactory.getLogger(HedgedRpcChannel.class);
+
+ private final AbstractRpcClient rpcClient;
+ // List of service addresses to hedge the requests to.
+ private final List addrs;
+ private final User ticket;
+ private final int rpcTimeout;
+ // Controls the size of request fan out (number of rpcs per a single batch).
+ private final int fanOutSize;
+
+ /**
+ * A simple rpc call back implementation to notify the batch context if any
rpc is successful.
+ */
+ private static class BatchRpcCtxCallBack implements RpcCallback {
[GitHub] [hbase] bharathv commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code.
bharathv commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code. URL: https://github.com/apache/hbase/pull/957#issuecomment-570656926 > TestZooKeeperTableArchiveClient fails consistently for me. Looks like it's built around a mock registry instance. Mind taking a look first? @ndimiduk Oops missed it. Was a legit bug. Fixed it. NPE shows up because the CONNECTION was not set up correctly (and hence == null) and we are trying to do a null.close(). Fixed the underlying bug. Verified locally that the test passes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe…
saintstack commented on issue #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe… URL: https://github.com/apache/hbase/pull/902#issuecomment-570656279 Hand-merged to branch-2 and master. Closed this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack closed pull request #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe…
saintstack closed pull request #902: HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionSe… URL: https://github.com/apache/hbase/pull/902 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId
saintstack commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId URL: https://github.com/apache/hbase/pull/910#issuecomment-570654273 Will commit in a day or so. Give @busbey a chance to looksee. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23378) Clean Up FSUtil setClusterId
[
https://issues.apache.org/jira/browse/HBASE-23378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007679#comment-17007679
]
HBase QA commented on HBASE-23378:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
34s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} master passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 4m
42s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} hbase-server: The patch generated 0 new + 65
unchanged - 1 fixed = 65 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}151m
29s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}209m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-910/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/910 |
| JIRA Issue | HBASE-23378 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ca6ebc9e3714 4.15.0-60-generic #6
[GitHub] [hbase] Apache-HBase commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId
Apache-HBase commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId URL: https://github.com/apache/hbase/pull/910#issuecomment-570653216 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 26s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | | +0 :ok: | spotbugs | 4m 34s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 33s | master passed | | -0 :warning: | patch | 4m 42s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | checkstyle | 1m 18s | hbase-server: The patch generated 0 new + 65 unchanged - 1 fixed = 65 total (was 66) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 47s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 4m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 151m 29s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 209m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-910/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/910 | | JIRA Issue | HBASE-23378 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ca6ebc9e3714 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-910/out/precommit/personality/provided.sh | | git revision | master / 85ecdd10a9 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-910/4/testReport/ | | Max. process+thread count | 5647 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-910/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #958: HBASE-23584 : Descrease rpc getFileStatus count when open a storefile
saintstack commented on a change in pull request #958: HBASE-23584 : Descrease
rpc getFileStatus count when open a storefile
URL: https://github.com/apache/hbase/pull/958#discussion_r362648386
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
##
@@ -362,21 +360,23 @@ private HDFSBlocksDistribution
computeHDFSBlocksDistributionInternal(final FileS
return FSUtils.computeHDFSBlocksDistribution(fs, status, 0,
status.getLen());
}
}
-
+
/**
* Get the {@link FileStatus} of the file referenced by this StoreFileInfo
* @param fs The current file system to use.
* @return The {@link FileStatus} of the file referenced by this
StoreFileInfo
*/
public FileStatus getReferencedFileStatus(final FileSystem fs) throws
IOException {
FileStatus status;
+if(this.localStatus != null) {return this.localStatus;}
Review comment:
Couldn't you simplify this method if you fetched status on construction
(using the bulk of this method but taking it private/internal?)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #958: HBASE-23584 : Descrease rpc getFileStatus count when open a storefile
saintstack commented on a change in pull request #958: HBASE-23584 : Descrease
rpc getFileStatus count when open a storefile
URL: https://github.com/apache/hbase/pull/958#discussion_r36264
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
##
@@ -168,9 +169,8 @@ private StoreFileInfo(final Configuration conf, final
FileSystem fs, final FileS
this.createdTimestamp = fileStatus.getModificationTime();
this.size = fileStatus.getLen();
} else {
-FileStatus fStatus = fs.getFileStatus(initialPath);
-this.createdTimestamp = fStatus.getModificationTime();
-this.size = fStatus.getLen();
+this.createdTimestamp = getFileStatus().getModificationTime();
Review comment:
Seems odd that we set modification time to created but that is not of your
making; it was there before you came along (perhaps only modification 'works'
or this is more trustworthy than create time...)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #958: HBASE-23584 : Descrease rpc getFileStatus count when open a storefile
saintstack commented on a change in pull request #958: HBASE-23584 : Descrease rpc getFileStatus count when open a storefile URL: https://github.com/apache/hbase/pull/958#discussion_r362647394 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java ## @@ -107,7 +108,7 @@ private RegionCoprocessorHost coprocessorHost; - // timestamp on when the file was created, is 0 and ignored for reference or link files + // change to use the createdTimestamp of reference or link files , will it create some problem later ? Review comment: Its good to mark data members final if your intent is that they are not ever to change. It helps the compiler and it helps the dev reading the code later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId
saintstack commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId URL: https://github.com/apache/hbase/pull/910#issuecomment-570650550 `...and I too am trying to raise the bar here and clean things up` Sorry. Yes. This is plain. This is great stuff. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #956: HBASE-23601: OutputSink.WriterThread exception gets stuck and repeated indefinietly
saintstack commented on a change in pull request #956: HBASE-23601:
OutputSink.WriterThread exception gets stuck and repeated indefinietly
URL: https://github.com/apache/hbase/pull/956#discussion_r362901505
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/OutputSink.java
##
@@ -91,6 +93,25 @@ public synchronized void startWriterThreads() {
}
}
+ public synchronized void restartWriterThreadsIfNeeded() {
+Iterator writerIterator = writerThreads.iterator();
+//remove dead threads
+List names = new ArrayList();
+while(writerIterator.hasNext()){
+ WriterThread t = writerIterator.next();
+ if (!t.isAlive()){
+names.add(t.getName());
Review comment:
Would it be easier to move through the writerThread array using an index
replacing dead threads with new live ones as you go? Should log when you
replace a thread I'd say at DEBUG level?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #956: HBASE-23601: OutputSink.WriterThread exception gets stuck and repeated indefinietly
saintstack commented on a change in pull request #956: HBASE-23601:
OutputSink.WriterThread exception gets stuck and repeated indefinietly
URL: https://github.com/apache/hbase/pull/956#discussion_r362903238
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/OutputSink.java
##
@@ -194,6 +215,14 @@ public boolean flush() throws IOException {
outputSink = sink;
}
+WriterThread(WALSplitter.PipelineController controller, EntryBuffers
entryBuffers,
+OutputSink sink, String threadName) {
+ super(threadName);
+ this.controller = controller;
+ this.entryBuffers = entryBuffers;
+ outputSink = sink;
+}
+
Review comment:
Have the constructor just above this call this one passing the constructed
thread name? nit.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-23333) Include simple Call.toShortString() in sendCall exceptions
[
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-2:
-
Summary: Include simple Call.toShortString() in sendCall exceptions (was:
Include simple Call.toString() in sendCall exceptions)
> Include simple Call.toShortString() in sendCall exceptions
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
> Issue Type: Improvement
> Components: Client, Operability
>Affects Versions: 3.0.0, 2.2.0, 2.3.0, 2.1.6
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
>
> Failure diagnosis isn't very straightforward with call stack traces like
> {noformat}
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to
> c501d28b0dfa/172.17.0.2:45657 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508,
> waitTime=60006, rpcTimeout=6
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more{noformat}
> Probably the "affectsVersions" goes back farther than this.
> See if we can provide more calling context, even stack trace from the call
> origin, in these exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
