[jira] [Created] (HBASE-24586) Add table level locality in table.jsp
Zheng Wang created HBASE-24586: -- Summary: Add table level locality in table.jsp Key: HBASE-24586 URL: https://issues.apache.org/jira/browse/HBASE-24586 Project: HBase Issue Type: Improvement Components: UI Reporter: Zheng Wang Assignee: Zheng Wang For now, we only show locality for each region, it should be apply to table as what Base Stats and Compactions does. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24582) The current implementation of assignMetaReplicas() may assign replica meta regions to the same server hosting primary meta region.
[
https://issues.apache.org/jira/browse/HBASE-24582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139122#comment-17139122
]
Duo Zhang commented on HBASE-24582:
---
No, I just mean the method in MasterMetaBootstrap...
> The current implementation of assignMetaReplicas() may assign replica meta
> regions to the same server hosting primary meta region.
> --
>
> Key: HBASE-24582
> URL: https://issues.apache.org/jira/browse/HBASE-24582
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
>
> We need to take the approach similar to SplitTableRegionProcedure, which uses
> round robin algo to assign replica regions and excludes the primary server.
>
> {code:java}
> return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env,
> hris,
> getRegionReplication(env), getParentRegionServerName(env));{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24582) The current implementation of assignMetaReplicas() may assign replica meta regions to the same server hosting primary meta region.
[
https://issues.apache.org/jira/browse/HBASE-24582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139120#comment-17139120
]
Huaxiang Sun commented on HBASE-24582:
--
[~zhangduo] Are you referring the algorithm used in SplitTableRegionProcedure
could also double assign replica regions?
> The current implementation of assignMetaReplicas() may assign replica meta
> regions to the same server hosting primary meta region.
> --
>
> Key: HBASE-24582
> URL: https://issues.apache.org/jira/browse/HBASE-24582
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
>
> We need to take the approach similar to SplitTableRegionProcedure, which uses
> round robin algo to assign replica regions and excludes the primary server.
>
> {code:java}
> return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env,
> hris,
> getRegionReplication(env), getParentRegionServerName(env));{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24585) If RSProcedureHandler throws exception, it aborts the hosting RS
[
https://issues.apache.org/jira/browse/HBASE-24585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139119#comment-17139119
]
Duo Zhang commented on HBASE-24585:
---
I think the design here is the task should not throw any exceptions, as it
should just reports the error to master. If we do meet an exception, the only
safe way is to abort. In SCP, master will know that the remote task is
failed(as the RS itself is gone), and then reschedule the remote task.
So here the problem is that we should not propagate the exception to the upper
layer?
> If RSProcedureHandler throws exception, it aborts the hosting RS
>
>
> Key: HBASE-24585
> URL: https://issues.apache.org/jira/browse/HBASE-24585
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Priority: Major
>
> HBASE-24574 proc v2 distributed log splitting is enabled. A remote split
> fails because it was interrupted. The InterruptedException became an IOE and
> then bubbled up and out of the RSPH below causing a RS abort.
> {code}
> 2020-06-17 21:20:37,472 ERROR
> [RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0]
> handler.RSProcedureHandler: Error when call RSProcedureCallable:
> java.io.IOException: Failed WAL split, status=RESIGNED,
> wal=file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/localhost%2C16020%2C1592440848604.meta.1592440852959.meta
>at
> org.apache.hadoop.hbase.regionserver.SplitWALCallable.splitWal(SplitWALCallable.java:106)
>at
> org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:86)
>at
> org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:49)
>at
> org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
>at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>at java.lang.Thread.run(Thread.java:748)
> {code}
> The remote-procedure framework needs to be more resilient? Log the exception
> unless an ERROR and keep going? Otherwise, makes features like procedurev2
> distributed log splitting brittle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24577) Doc WALSplitter classes
[
https://issues.apache.org/jira/browse/HBASE-24577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139115#comment-17139115
]
Hudson commented on HBASE-24577:
Results for branch branch-2.3
[build #144 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/144/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/144/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/144/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/144/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/144/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Doc WALSplitter classes
> ---
>
> Key: HBASE-24577
> URL: https://issues.apache.org/jira/browse/HBASE-24577
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0-alpha-1, 2.3.1
>
>
> Trivial. Added doc to classes and added trace-level logging so can better
> study this feature in action (added it to old impl too).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24585) If RSProcedureHandler throws exception, it aborts the hosting RS
[
https://issues.apache.org/jira/browse/HBASE-24585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24585:
--
Description:
HBASE-24574 proc v2 distributed log splitting is enabled. A remote split fails
because it was interrupted. The InterruptedException became an IOE and then
bubbled up and out of the RSPH below causing a RS abort.
{code}
2020-06-17 21:20:37,472 ERROR
[RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0] handler.RSProcedureHandler:
Error when call RSProcedureCallable:
java.io.IOException: Failed WAL split, status=RESIGNED,
wal=file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/localhost%2C16020%2C1592440848604.meta.1592440852959.meta
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.splitWal(SplitWALCallable.java:106)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:86)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:49)
at
org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
The remote-procedure framework needs to be more resilient? Log the exception
unless an ERROR and keep going? Otherwise, makes features like procedurev2
distributed log splitting brittle.
was:
HBASE-24574 proc v2 distributed log splitting is enabled. A remote split fails
because it was interrupted. The InterruptedException became an IOE and then
bubbled up and out of the RSPH below causing a RS abort.
{code}
2020-06-17 21:20:37,472 ERROR
[RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0] handler.RSProcedureHandler:
Error when call RSProcedureCallable:
java.io.IOException: Failed WAL split, status=RESIGNED,
wal=file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/localhost%2C16020%2C1592440848604.meta.1592440852959.meta
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.splitWal(SplitWALCallable.java:106)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:86)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:49)
at
org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
> If RSProcedureHandler throws exception, it aborts the hosting RS
>
>
> Key: HBASE-24585
> URL: https://issues.apache.org/jira/browse/HBASE-24585
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Priority: Major
>
> HBASE-24574 proc v2 distributed log splitting is enabled. A remote split
> fails because it was interrupted. The InterruptedException became an IOE and
> then bubbled up and out of the RSPH below causing a RS abort.
> {code}
> 2020-06-17 21:20:37,472 ERROR
> [RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0]
> handler.RSProcedureHandler: Error when call RSProcedureCallable:
> java.io.IOException: Failed WAL split, status=RESIGNED,
> wal=file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/localhost%2C16020%2C1592440848604.meta.1592440852959.meta
>at
> org.apache.hadoop.hbase.regionserver.SplitWALCallable.splitWal(SplitWALCallable.java:106)
>at
> org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:86)
>at
> org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:49)
>at
> org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
>at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>at java.lang.Thread.run(Thread.java:748)
> {code}
> The remote-procedure framework needs to be more resilient? Log the exception
> unless an ERROR and keep going? Otherwise, makes features like procedurev2
> distributed log splitting brittle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23739) BoundedRecoveredHFilesOutputSink should read the table descriptor directly
[ https://issues.apache.org/jira/browse/HBASE-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139111#comment-17139111 ] Michael Stack commented on HBASE-23739: --- Yeah, the HBASE-19126 execute a procedure on the RS-side will abort the RS if an exception out of a Procedure. We probably don't want this. Let me open issue... HBASE-24585. > BoundedRecoveredHFilesOutputSink should read the table descriptor directly > -- > > Key: HBASE-23739 > URL: https://issues.apache.org/jira/browse/HBASE-23739 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0 > > > Read from meta or filesystem? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24585) If RSProcedureHandler throws exception, it aborts the hosting RS
Michael Stack created HBASE-24585:
-
Summary: If RSProcedureHandler throws exception, it aborts the
hosting RS
Key: HBASE-24585
URL: https://issues.apache.org/jira/browse/HBASE-24585
Project: HBase
Issue Type: Bug
Reporter: Michael Stack
HBASE-24574 proc v2 distributed log splitting is enabled. A remote split fails
because it was interrupted. The InterruptedException became an IOE and then
bubbled up and out of the RSPH below causing a RS abort.
{code}
2020-06-17 21:20:37,472 ERROR
[RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0] handler.RSProcedureHandler:
Error when call RSProcedureCallable:
java.io.IOException: Failed WAL split, status=RESIGNED,
wal=file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/localhost%2C16020%2C1592440848604.meta.1592440852959.meta
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.splitWal(SplitWALCallable.java:106)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:86)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:49)
at
org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-operator-tools] clarax opened a new pull request #68: HBASE-23927 hbck2 command should accept one or more files containing …
clarax opened a new pull request #68: URL: https://github.com/apache/hbase-operator-tools/pull/68 …a list of region names This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-23927) hbck2 command should accept one or more files containing a list of region names
[ https://issues.apache.org/jira/browse/HBASE-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clara Xiong updated HBASE-23927: Summary: hbck2 command should accept one or more files containing a list of region names (was: hbck2 assigns command should accept a file containing a list of region names) > hbck2 command should accept one or more files containing a list of region > names > --- > > Key: HBASE-23927 > URL: https://issues.apache.org/jira/browse/HBASE-23927 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2, Operability, Usability >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: Nick Dimiduk >Assignee: Clara Xiong >Priority: Major > Fix For: hbase-operator-tools-1.1.0 > > > The interface is not very ergonomic. Currently the command accepts a list of > region names on the command line. If you have 100's of regions to assign, > this sucks. We should accept a path to a file that contains these encoded > regions, one per line. That way, this command tails nicely into an operator's > incantation using grep/sed over log files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Reidddddd commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
Reidd commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r441971912
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
##
@@ -78,4 +82,30 @@ public void execute(Admin admin) {
LOG.error("Error during region merge: ", ex);
}
}
+
+ @Override
+ public boolean equals(Object o) {
+if (this == o) {
+ return true;
+}
+
+if (o == null || getClass() != o.getClass()) {
+ return false;
+}
+
+MergeNormalizationPlan that = (MergeNormalizationPlan) o;
+
+return new EqualsBuilder()
+ .append(firstRegion, that.firstRegion)
+ .append(secondRegion, that.secondRegion)
+ .isEquals();
+ }
+
+ @Override
+ public int hashCode() {
+return new HashCodeBuilder(17, 37)
Review comment:
wtb generate the hashCode at the beginning (in constructor)? instead of
calculating it every time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Reidddddd commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
Reidd commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r441972265
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
##
@@ -78,4 +82,30 @@ public void execute(Admin admin) {
LOG.error("Error during region merge: ", ex);
}
}
+
+ @Override
+ public boolean equals(Object o) {
+if (this == o) {
+ return true;
+}
+
+if (o == null || getClass() != o.getClass()) {
+ return false;
+}
+
+MergeNormalizationPlan that = (MergeNormalizationPlan) o;
+
+return new EqualsBuilder()
+ .append(firstRegion, that.firstRegion)
+ .append(secondRegion, that.secondRegion)
+ .isEquals();
+ }
+
+ @Override
+ public int hashCode() {
+return new HashCodeBuilder(17, 37)
Review comment:
the same below
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Reidddddd commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
Reidd commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r441971912
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
##
@@ -78,4 +82,30 @@ public void execute(Admin admin) {
LOG.error("Error during region merge: ", ex);
}
}
+
+ @Override
+ public boolean equals(Object o) {
+if (this == o) {
+ return true;
+}
+
+if (o == null || getClass() != o.getClass()) {
+ return false;
+}
+
+MergeNormalizationPlan that = (MergeNormalizationPlan) o;
+
+return new EqualsBuilder()
+ .append(firstRegion, that.firstRegion)
+ .append(secondRegion, that.secondRegion)
+ .isEquals();
+ }
+
+ @Override
+ public int hashCode() {
+return new HashCodeBuilder(17, 37)
Review comment:
wtb generate the hashCode at the beginning (in constructor)? instead of
calculating every time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ramkrish86 commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
ramkrish86 commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441966355
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
##
@@ -81,6 +83,7 @@ private RecoveredEditsWriter
getRecoveredEditsWriter(TableName tableName, byte[]
if (ret == null) {
return null;
}
+LOG.trace("Created {}", ret.path);
Review comment:
Just asking. this can be in DEBUG mode right rather than trace?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-24518) waitForNamespaceOnline() should return false if any region is offline
[
https://issues.apache.org/jira/browse/HBASE-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139063#comment-17139063
]
Hudson commented on HBASE-24518:
Results for branch branch-2
[build #2709 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> waitForNamespaceOnline() should return false if any region is offline
> -
>
> Key: HBASE-24518
> URL: https://issues.apache.org/jira/browse/HBASE-24518
> Project: HBase
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> HMaster waits for namespace regions to come online only for rolling upgrade
> scenario (from 1.x to 2.x), however, waitForNamespaceOnline() does not return
> false for any offline region. We should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24564) Make RS abort call idempotent
[
https://issues.apache.org/jira/browse/HBASE-24564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139062#comment-17139062
]
Hudson commented on HBASE-24564:
Results for branch branch-2
[build #2709 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2709/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make RS abort call idempotent
> -
>
> Key: HBASE-24564
> URL: https://issues.apache.org/jira/browse/HBASE-24564
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
>
> We noticed this in our deployment based on branch-1, but it affects other
> branches too.
> 1. abort() is not idempotent. There can be multiple aborts that can
> un-necessarily complicate the state machine. Following is the timeline of
> actions.
> - HMaster detected that the RS lost its ZK session and started the SCP. This
> was caused by ZK flakiness
> {noformat}
> 2020-06-11 01:08:39,110 DEBUG [ProcedureExecutor-34] master.DeadServer -
> Started processing foo,60020,1591683150711; numProcessing=2
> 2020-06-11 01:08:39,110 INFO [ProcedureExecutor-34]
> procedure.ServerCrashProcedure - Start processing crashed
> foo,60020,1591683150711
> {noformat}
> - RS wakes up and attempts to report to master and receives a YouAreDead...
> This triggers an abort
> {noformat}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.YouAreDeadException):
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing foo,60020,1591683150711 as dead server
> at
> org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:438)
> at
> org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:343)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:359)
> at
> org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:8617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2421)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:311)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:291)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:94)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:413)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:409)
> at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
> at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
> at
> org.apache.hadoop.hbase.ipc.BlockingRpcConnection.readResponse(BlockingRpcConnection.java:600)
> at
> org.apache.hadoop.hbase.ipc.BlockingRpcConnection.run(BlockingRpcConnection.java:334)
> ... 1 more
> {noformat}
> - After a few seconds, RS also realizes that it lost the ZK session and
> initiates a second abort
> {noformat}
> 2020-06-11 01:08:50,321 FATAL [main-EventThread] regionserver.HRegionServer -
> ABORTING region server foo,60020,1591683150711:
> regionserver:60020-0x1725cd18ff3c55f, quorum=foo:2181,bar:2181,baz:2181
> baseZNode=/hbase regionserver:60020-0x1725cd18ff3c55f received expired from
> ZooKeeper, aborting
> org.apache.zookeeper.KeeperException$Session
[jira] [Commented] (HBASE-18659) Use HDFS ACL to give user the ability to read snapshot directly on HDFS
[
https://issues.apache.org/jira/browse/HBASE-18659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139058#comment-17139058
]
Guanghao Zhang commented on HBASE-18659:
{quote}we can close this parent issue as resolved against 2.3.0?
{quote}
Agreed.
> Use HDFS ACL to give user the ability to read snapshot directly on HDFS
> ---
>
> Key: HBASE-18659
> URL: https://issues.apache.org/jira/browse/HBASE-18659
> Project: HBase
> Issue Type: New Feature
>Reporter: Duo Zhang
>Assignee: Yi Mei
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> On the dev meetup notes in Shenzhen after HBaseCon Asia, there is a topic
> about the permission to read hfiles on HDFS directly.
> {quote}
> For client-side scanner going against hfiles directly; is there a means of
> being able to pass the permissions from hbase to hdfs?
> {quote}
> And at Xiaomi we also face the same problem. {{SnapshotScanner}} is much
> faster and consumes less resources, but only super use has the ability to
> read hfile directly on HDFS.
> So here we want to use HDFS ACL to address this problem.
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#ACLs_File_System_API
> The basic idea is to set acl and default acl on the ns/table/cf directory on
> HDFS for the users who have the permission to read the table on HBase.
> Suggestions are welcomed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23739) BoundedRecoveredHFilesOutputSink should read the table descriptor directly
[
https://issues.apache.org/jira/browse/HBASE-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139052#comment-17139052
]
Michael Stack commented on HBASE-23739:
---
I ran into this issue. We try 18 times to get the descriptor from Master before
we time out... about 3 minutes. We finally fail w/ this (someone interrupts us):
{code}
2020-06-17 21:20:37,469 WARN [split-log-closeStream--pool6-t1]
wal.BoundedRecoveredHFilesOutputSink: Failed to get table descriptor for
hbase:meta
java.io.InterruptedIOException: Interrupted after 18 tries while maxAttempts=46
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:173)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3007)
at
org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:550)
at
org.apache.hadoop.hbase.client.HBaseAdmin.getDescriptor(HBaseAdmin.java:365)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.getTableDescriptor(BoundedRecoveredHFilesOutputSink.java:227)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$createRecoveredHFileWriter$4(BoundedRecoveredHFilesOutputSink.java:203)
at
java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.createRecoveredHFileWriter(BoundedRecoveredHFilesOutputSink.java:203)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.append(BoundedRecoveredHFilesOutputSink.java:109)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.lambda$writeRemainingEntryBuffers$3(BoundedRecoveredHFilesOutputSink.java:149)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
...
{code}
... which seems to interrupt the split worker that was running on the
regionserver
{code}
2020-06-17 21:20:37,469 WARN
[RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0] regionserver.SplitLogWorker:
Resigning, interrupted splitting of WAL
file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/
localhost%2C16020%2C1592440848604.meta.1592440852959.meta
java.io.InterruptedIOException
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:163)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.close(BoundedRecoveredHFilesOutputSink.java:136)
at org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:391)
at org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:215)
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:102)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.splitWal(SplitWALCallable.java:104)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:86)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:49)
at
org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
java.util.concurrent.ExecutorCompletionService.take(ExecutorCompletionService.java:193)
at
org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink.writeRemainingEntryBuffers(BoundedRecoveredHFilesOutputSink.java:156)
... 12 more
{code}
The interrupt comes out here which causes the RS to crash:
{code}
2020-06-17 21:20:37,472 ERROR
[RS_LOG_REPLAY_OPS-regionserver/localhost:16020-0] handler.RSProcedureHandler:
Error when call RSProcedureCallable:
java.io.IOException: Failed WAL split, status=RESIGNED,
wal=file:/Users/stack/checkouts/hbase.apache.git/tmp/hbase/WALs/localhost,16020,1592440848604-splitting/localhost%2C16020%2C1592440848604.meta.1592440852959.meta
at
org.apache.hadoop.h
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Apache-HBase commented on pull request #1912: URL: https://github.com/apache/hbase/pull/1912#issuecomment-645756082 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | branch-2 passed | | +1 :green_heart: | compile | 0m 59s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 5m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 212m 56s | hbase-server in the patch passed. | | | | 237m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1912 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6407fd031f2f 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 812d1e2bb5 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/testReport/ | | Max. process+thread count | 2342 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
WenFeiYi commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r441931716
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv
env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir,
tableDescriptor.getTableName());
+if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.",
tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+}
Review comment:
yes, RestoreSnapshotHelper.restoreHdfsRegions will remove the region
dirs, but CloneSnapshotProcedure.createFilesystemLayout not allow exist remove.
retrying will have the same effect
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1914: HBASE-24550 Passing '-h' or '--help' to bin/hbase doesn't do as expected
WenFeiYi commented on a change in pull request #1914: URL: https://github.com/apache/hbase/pull/1914#discussion_r441929778 ## File path: bin/hbase-config.sh ## @@ -94,6 +94,10 @@ do shift # shellcheck disable=SC2034 DEBUG="true" + elif [ "--help" = "$1" ] || [ "-h" = "$1" ] + then +echo "Error usage! You should use it like 'hbase --help' or 'hbase -h'"; Review comment: hbase will call hbase-config, and pass the parameters This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
Apache-HBase commented on pull request #1922: URL: https://github.com/apache/hbase/pull/1922#issuecomment-645727205 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 41s | master passed | | +1 :green_heart: | compile | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 5m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 5m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 24s | hbase-common in the patch passed. | | -1 :x: | unit | 156m 16s | hbase-server in the patch failed. | | | | 184m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1922 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 47f797ff87e8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6a576696dc | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/testReport/ | | Max. process+thread count | 3840 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
bsglz commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441929197
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws
IOException {
}
}
-// The key point is create a new writer for each column family, write
edits then close writer.
+// Create a new hfile writer for each column family, write edits then
close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName,
regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
IIRC, this extra check could be used only if the parameter self will
cause a heavy call.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1914: HBASE-24550 Passing '-h' or '--help' to bin/hbase doesn't do as expected
WenFeiYi commented on a change in pull request #1914: URL: https://github.com/apache/hbase/pull/1914#discussion_r441928940 ## File path: bin/hbase-config.sh ## @@ -94,6 +94,10 @@ do shift # shellcheck disable=SC2034 DEBUG="true" + elif [ "--help" = "$1" ] || [ "-h" = "$1" ] + then +echo "Error usage! You should use it like 'hbase --help' or 'hbase -h'"; Review comment: bin/hbase --config ./conf/ --help version This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
Apache-HBase commented on pull request #1922: URL: https://github.com/apache/hbase/pull/1922#issuecomment-645724876 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 5m 56s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | +1 :green_heart: | compile | 1m 34s | the patch passed | | +1 :green_heart: | javac | 1m 34s | the patch passed | | +1 :green_heart: | shadedjars | 6m 13s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 45s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 47s | hbase-common in the patch passed. | | -1 :x: | unit | 144m 43s | hbase-server in the patch failed. | | | | 175m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1922 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5048ef0e15e1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6a576696dc | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/testReport/ | | Max. process+thread count | 4362 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24582) The current implementation of assignMetaReplicas() may assign replica meta regions to the same server hosting primary meta region.
[
https://issues.apache.org/jira/browse/HBASE-24582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138943#comment-17138943
]
Duo Zhang commented on HBASE-24582:
---
It could also double assign a region and cause problem...
> The current implementation of assignMetaReplicas() may assign replica meta
> regions to the same server hosting primary meta region.
> --
>
> Key: HBASE-24582
> URL: https://issues.apache.org/jira/browse/HBASE-24582
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
>
> We need to take the approach similar to SplitTableRegionProcedure, which uses
> round robin algo to assign replica regions and excludes the primary server.
>
> {code:java}
> return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env,
> hris,
> getRegionReplication(env), getParentRegionServerName(env));{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Apache-HBase commented on pull request #1913: URL: https://github.com/apache/hbase/pull/1913#issuecomment-645714547 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 53s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 32s | branch-2 passed | | +1 :green_heart: | compile | 1m 4s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 5m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 199m 35s | hbase-server in the patch passed. | | | | 226m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1913 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 459e52911708 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/testReport/ | | Max. process+thread count | 2541 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Apache-HBase commented on pull request #1913: URL: https://github.com/apache/hbase/pull/1913#issuecomment-645710536 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 43s | branch-2 passed | | +1 :green_heart: | compile | 1m 9s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 25s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 23s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 6m 24s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 183m 34s | hbase-server in the patch passed. | | | | 212m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1913 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 438b448c14c7 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/testReport/ | | Max. process+thread count | 3206 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24574) Procedure V2 - Distributed WAL Splitting => LOGGING
[
https://issues.apache.org/jira/browse/HBASE-24574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24574:
--
Description:
The procedure-based distributed WAL splitter uses the old SplitLogManager
class. It registers it as a chore even though the chore #run immediately skips
out. Meantime we log SLM ran in #0ms frequently. For example...
{code}
...
2020-06-17 17:48:30,567 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:31,567 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:32,570 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:33,567 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:34,569 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:35,569 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
2020-06-17 17:48:36,568 DEBUG [localhost:16000.splitLogManager..Chore.1]
hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
...
{code}
The new pv2 WAL splitter introduces new procedures that log their class name
only w/o detail on log that is being split or the server that is running the
split.
was:
The procedure-based distributed WAL splitter uses the old SplitLogManager
class. It registers it as a chore even though the chore #run immediately skips
out. Meantime we log SLM ran in #0ms frequently.
The new pv2 WAL splitter introduces new procedures that log their class name
only w/o detail on log that is being split or the server that is running the
split.
> Procedure V2 - Distributed WAL Splitting => LOGGING
> ---
>
> Key: HBASE-24574
> URL: https://issues.apache.org/jira/browse/HBASE-24574
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
>
> The procedure-based distributed WAL splitter uses the old SplitLogManager
> class. It registers it as a chore even though the chore #run immediately
> skips out. Meantime we log SLM ran in #0ms frequently. For example...
> {code}
> ...
> 2020-06-17 17:48:30,567 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> 2020-06-17 17:48:31,567 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> 2020-06-17 17:48:32,570 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> 2020-06-17 17:48:33,567 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> 2020-06-17 17:48:34,569 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> 2020-06-17 17:48:35,569 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> 2020-06-17 17:48:36,568 DEBUG [localhost:16000.splitLogManager..Chore.1]
> hbase.ScheduledChore: SplitLogManager Timeout Monitor execution time: 0 ms.
> ...
> {code}
> The new pv2 WAL splitter introduces new procedures that log their class name
> only w/o detail on log that is being split or the server that is running the
> split.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-native-client] bharathv commented on pull request #8: HBASE-24559: Fix test logging initialization
bharathv commented on pull request #8: URL: https://github.com/apache/hbase-native-client/pull/8#issuecomment-645702428 Rebased on top of my other patches. Had to make some small fixes to get the tests running. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Apache-HBase commented on pull request #1912: URL: https://github.com/apache/hbase/pull/1912#issuecomment-645697980 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 11s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 56s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 11s | the patch passed | | +1 :green_heart: | checkstyle | 1m 8s | hbase-server: The patch generated 0 new + 85 unchanged - 2 fixed = 85 total (was 87) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 37s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 32m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1912 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux f2d8554a8b85 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 812d1e2bb5 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-24584) Release 2.2.6
Guanghao Zhang created HBASE-24584: -- Summary: Release 2.2.6 Key: HBASE-24584 URL: https://issues.apache.org/jira/browse/HBASE-24584 Project: HBase Issue Type: Umbrella Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24584) Release 2.2.6
[ https://issues.apache.org/jira/browse/HBASE-24584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-24584: -- Assignee: Guanghao Zhang > Release 2.2.6 > - > > Key: HBASE-24584 > URL: https://issues.apache.org/jira/browse/HBASE-24584 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Apache-HBase commented on pull request #1912: URL: https://github.com/apache/hbase/pull/1912#issuecomment-645686333 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 0m 55s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 14s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 4m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 132m 12s | hbase-server in the patch passed. | | | | 154m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1912 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7e573f915928 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/testReport/ | | Max. process+thread count | 4416 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
Apache-HBase commented on pull request #1922: URL: https://github.com/apache/hbase/pull/1922#issuecomment-645684759 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 34s | master passed | | +1 :green_heart: | spotbugs | 2m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | checkstyle | 1m 35s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 39m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1922 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 82980db53df8 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6a576696dc | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1922/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Apache-HBase commented on pull request #1912: URL: https://github.com/apache/hbase/pull/1912#issuecomment-645684435 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 11s | branch-2 passed | | +1 :green_heart: | compile | 1m 4s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 48s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 122m 11s | hbase-server in the patch passed. | | | | 148m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1912 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6e5ea84c884e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/testReport/ | | Max. process+thread count | 3931 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
ndimiduk opened a new pull request #1922: URL: https://github.com/apache/hbase/pull/1922 * add `testMergeEmptyRegions` to explicitly cover different interleaving of 0-sized regions. * fix bug where merging a 0-size region is skipped due to large neighbor. * remove unused `splitPoint` from `SplitNormalizationPlan`. * generate `toString`, `hashCode`, and `equals` methods from Apache Commons Lang3 template on `SplitNormalizationPlan` and `MergeNormalizationPlan`. * simplify test to use equality matching over `*NormalizationPlan` instances as plain pojos. * test make use of this handy `TableNameTestRule`. * fix line-length issues in `TestSimpleRegionNormalizer` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24583) Normalizer can't actually merge empty regions when neighbor is larger than average size
[
https://issues.apache.org/jira/browse/HBASE-24583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-24583:
-
Status: Patch Available (was: Open)
> Normalizer can't actually merge empty regions when neighbor is larger than
> average size
> ---
>
> Key: HBASE-24583
> URL: https://issues.apache.org/jira/browse/HBASE-24583
> Project: HBase
> Issue Type: Bug
> Components: master, Normalizer
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> There are plenty of cases where empty regions can accumulate -- incorrect
> guessing at split points, old data is automatically expiring off, &c. The
> normalizer stubbornly refuses to handle this case, despite this being an
> original feature it was intended to support (HBASE-6613).
> Earlier discussion had concerns for a user pre-splitting a table and then the
> normalizer coming along and merging those splits away before they could be
> populated. Thus, right now, the default behavior via
> {{hbase.normalizer.merge.min_region_size.mb=1}} is to not split any region
> that's so small. Later, we added
> {{hbase.normalizer.merge.min_region_age.days=3}}, which prevents us from
> merging any region too young. So there's plenty of nobs for an operator to
> customize their behavior.
> But when I set {{hbase.normalizer.merge.min_region_size.mb=0}}, I still end
> up with stubborn regions that won't merge away. Looks like a large neighbor
> will prevent a merge from going through.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24583) Normalizer can't actually merge empty regions when neighbor is larger than average size
Nick Dimiduk created HBASE-24583:
Summary: Normalizer can't actually merge empty regions when
neighbor is larger than average size
Key: HBASE-24583
URL: https://issues.apache.org/jira/browse/HBASE-24583
Project: HBase
Issue Type: Bug
Components: master, Normalizer
Affects Versions: 2.3.0
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
There are plenty of cases where empty regions can accumulate -- incorrect
guessing at split points, old data is automatically expiring off, &c. The
normalizer stubbornly refuses to handle this case, despite this being an
original feature it was intended to support (HBASE-6613).
Earlier discussion had concerns for a user pre-splitting a table and then the
normalizer coming along and merging those splits away before they could be
populated. Thus, right now, the default behavior via
{{hbase.normalizer.merge.min_region_size.mb=1}} is to not split any region
that's so small. Later, we added
{{hbase.normalizer.merge.min_region_age.days=3}}, which prevents us from
merging any region too young. So there's plenty of nobs for an operator to
customize their behavior.
But when I set {{hbase.normalizer.merge.min_region_size.mb=0}}, I still end up
with stubborn regions that won't merge away. Looks like a large neighbor will
prevent a merge from going through.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Apache-HBase commented on pull request #1913: URL: https://github.com/apache/hbase/pull/1913#issuecomment-645658012 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 53s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 32s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 20s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 17s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | -0 :warning: | checkstyle | 1m 19s | hbase-server: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 28s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 40m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1913 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b7e71b0e79c1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1832: HBASE-24493 WIP Increase hadoop logging
Apache-HBase commented on pull request #1832: URL: https://github.com/apache/hbase/pull/1832#issuecomment-645651373 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 41s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 22s | master passed | | +1 :green_heart: | compile | 0m 53s | master passed | | +1 :green_heart: | shadedjars | 7m 57s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-logging in master failed. | | -0 :warning: | javadoc | 0m 26s | hbase-mapreduce in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 22s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed | | +1 :green_heart: | javac | 0m 45s | the patch passed | | +1 :green_heart: | shadedjars | 7m 19s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-logging in the patch failed. | | -0 :warning: | javadoc | 0m 26s | hbase-mapreduce in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 15s | hbase-logging in the patch passed. | | +1 :green_heart: | unit | 16m 0s | hbase-mapreduce in the patch passed. | | | | 49m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1832 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux fcc3932c1e58 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-logging.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-mapreduce.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-logging.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-mapreduce.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/testReport/ | | Max. process+thread count | 2924 (vs. ulimit of 12500) | | modules | C: hbase-logging hbase-mapreduce U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24577) Doc WALSplitter classes
[ https://issues.apache.org/jira/browse/HBASE-24577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24577. --- Fix Version/s: 2.3.1 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-2.3+ Thanks for reviews. > Doc WALSplitter classes > --- > > Key: HBASE-24577 > URL: https://issues.apache.org/jira/browse/HBASE-24577 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0-alpha-1, 2.3.1 > > > Trivial. Added doc to classes and added trace-level logging so can better > study this feature in action (added it to old impl too). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack merged pull request #1913: URL: https://github.com/apache/hbase/pull/1913 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
Apache-HBase commented on pull request #1912: URL: https://github.com/apache/hbase/pull/1912#issuecomment-645647017 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 25s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 32s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | the patch passed | | +1 :green_heart: | checkstyle | 1m 25s | hbase-server: The patch generated 0 new + 85 unchanged - 2 fixed = 85 total (was 87) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 48s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 37m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1912 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 7e98a43abb06 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1912/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1832: HBASE-24493 WIP Increase hadoop logging
Apache-HBase commented on pull request #1832: URL: https://github.com/apache/hbase/pull/1832#issuecomment-645647092 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 0m 39s | master passed | | +1 :green_heart: | shadedjars | 6m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 0m 38s | the patch passed | | +1 :green_heart: | javac | 0m 38s | the patch passed | | +1 :green_heart: | shadedjars | 6m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 11s | hbase-logging in the patch passed. | | +1 :green_heart: | unit | 13m 21s | hbase-mapreduce in the patch passed. | | | | 39m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1832 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 10824575e57f 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/testReport/ | | Max. process+thread count | 3209 (vs. ulimit of 12500) | | modules | C: hbase-logging hbase-mapreduce U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1832: HBASE-24493 WIP Increase hadoop logging
Apache-HBase commented on pull request #1832: URL: https://github.com/apache/hbase/pull/1832#issuecomment-645645762 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 40s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +1 :green_heart: | checkstyle | 0m 30s | master passed | | +1 :green_heart: | spotbugs | 1m 10s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 1m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 36m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1832 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 6e3602460683 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-logging hbase-mapreduce U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1832/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Apache-HBase commented on pull request #1913: URL: https://github.com/apache/hbase/pull/1913#issuecomment-645640215 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | branch-2 passed | | +1 :green_heart: | compile | 0m 56s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 187m 27s | hbase-server in the patch passed. | | | | 211m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1913 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ee6fc565 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/testReport/ | | Max. process+thread count | 2575 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24562) Stabilize master startup with meta replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138851#comment-17138851 ] Huaxiang Sun commented on HBASE-24562: -- Hi [~bszabolcs], I included this Jira in HBASE-24554 as part of the effort to improve read replica, thanks. > Stabilize master startup with meta replicas enabled > --- > > Key: HBASE-24562 > URL: https://issues.apache.org/jira/browse/HBASE-24562 > Project: HBase > Issue Type: Improvement > Components: meta, read replicas >Reporter: Szabolcs Bukros >Assignee: Szabolcs Bukros >Priority: Major > > This is related to HBASE-21624 . > I created a separate ticket because in the original one a "complete solution > for meta replicas" was requested and this is not one. I'm just trying to make > master startup more stable by making assigning meta replicas asynchronous and > preventing a potential assignment failure from crashing master. > The idea is that starting master with less or even no meta replicas assigned > is preferable to not having a running master. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23927) hbck2 assigns command should accept a file containing a list of region names
[ https://issues.apache.org/jira/browse/HBASE-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138850#comment-17138850 ] Nick Dimiduk commented on HBASE-23927: -- I think you're right, we should support this mode of operation for any command that accepts a list of items. > hbck2 assigns command should accept a file containing a list of region names > > > Key: HBASE-23927 > URL: https://issues.apache.org/jira/browse/HBASE-23927 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2, Operability, Usability >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: Nick Dimiduk >Assignee: Clara Xiong >Priority: Major > Fix For: hbase-operator-tools-1.1.0 > > > The interface is not very ergonomic. Currently the command accepts a list of > region names on the command line. If you have 100's of regions to assign, > this sucks. We should accept a path to a file that contains these encoded > regions, one per line. That way, this command tails nicely into an operator's > incantation using grep/sed over log files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24582) The current implementation of assignMetaReplicas() may assign replica meta regions to the same server hosting primary meta region.
[
https://issues.apache.org/jira/browse/HBASE-24582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huaxiang Sun updated HBASE-24582:
-
Description:
We need to take the approach similar to SplitTableRegionProcedure, which uses
round robin algo to assign replica regions and excludes the primary server.
{code:java}
return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env,
hris,
getRegionReplication(env), getParentRegionServerName(env));{code}
was:
We need to take the approach similar to SplitTableRegionProcedure, which uses
round robin algo to assign replica regions and excludes the primary server.
'''return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env,
hris,
getRegionReplication(env), getParentRegionServerName(env));'''
> The current implementation of assignMetaReplicas() may assign replica meta
> regions to the same server hosting primary meta region.
> --
>
> Key: HBASE-24582
> URL: https://issues.apache.org/jira/browse/HBASE-24582
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
>
> We need to take the approach similar to SplitTableRegionProcedure, which uses
> round robin algo to assign replica regions and excludes the primary server.
>
> {code:java}
> return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env,
> hris,
> getRegionReplication(env), getParentRegionServerName(env));{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24582) The current implementation of assignMetaReplicas() may assign replica meta regions to the same server hosting primary meta region.
Huaxiang Sun created HBASE-24582: Summary: The current implementation of assignMetaReplicas() may assign replica meta regions to the same server hosting primary meta region. Key: HBASE-24582 URL: https://issues.apache.org/jira/browse/HBASE-24582 Project: HBase Issue Type: Bug Affects Versions: 2.3.0 Reporter: Huaxiang Sun Assignee: Huaxiang Sun We need to take the approach similar to SplitTableRegionProcedure, which uses round robin algo to assign replica regions and excludes the primary server. '''return AssignmentManagerUtil.createAssignProceduresForOpeningNewRegions(env, hris, getRegionReplication(env), getParentRegionServerName(env));''' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441840561
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws
IOException {
}
}
-// The key point is create a new writer for each column family, write
edits then close writer.
+// Create a new hfile writer for each column family, write edits then
close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName,
regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
Thanks @virajjasani
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-24565) Bump version on branch-2.3 to 2.3.1-SNAPSHOT
[ https://issues.apache.org/jira/browse/HBASE-24565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24565. -- Assignee: (was: Nick Dimiduk) Resolution: Invalid Seems the do-release script does this for me. > Bump version on branch-2.3 to 2.3.1-SNAPSHOT > > > Key: HBASE-24565 > URL: https://issues.apache.org/jira/browse/HBASE-24565 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Priority: Major > > Upgrade pom version after successful 2.3.0 release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on pull request #1832: HBASE-24493 WIP Increase hadoop logging
ndimiduk commented on pull request #1832: URL: https://github.com/apache/hbase/pull/1832#issuecomment-645629656 The test simply won't fail :( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24581) Replica regions should not trigger any compaction
[
https://issues.apache.org/jira/browse/HBASE-24581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huaxiang Sun updated HBASE-24581:
-
Description:
I found that in certain cases replica regions can trigger compaction, one
example as follows, need to check all places to avoid compaction for replica
regions.
{code:java}
@Override
public void postOpenDeployTasks(final PostOpenDeployContext context) throws
IOException {
HRegion r = context.getRegion();
long openProcId = context.getOpenProcId();
long masterSystemTime = context.getMasterSystemTime();
rpcServices.checkOpen();
LOG.info("Post open deploy tasks for {}, openProcId={}, masterSystemTime={}",
r.getRegionInfo().getRegionNameAsString(), openProcId, masterSystemTime);
// Do checks to see if we need to compact (references or too many files)
// TODO: SHX, do not do this for replica regions? Otherwise, it is going to
lost data locality for primary regions.
for (HStore s : r.stores.values()) {
if (s.hasReferences() || s.needsCompaction()) {
this.compactSplitThread.requestSystemCompaction(r, s, "Opening Region");
}
}
{code}
was:
I found that in certain cases replica regions can trigger compaction, such as
{code:java}
@Override
public void postOpenDeployTasks(final PostOpenDeployContext context) throws
IOException {
HRegion r = context.getRegion();
long openProcId = context.getOpenProcId();
long masterSystemTime = context.getMasterSystemTime();
rpcServices.checkOpen();
LOG.info("Post open deploy tasks for {}, openProcId={}, masterSystemTime={}",
r.getRegionInfo().getRegionNameAsString(), openProcId, masterSystemTime);
// Do checks to see if we need to compact (references or too many files)
// TODO: SHX, do not do this for replica regions? Otherwise, it is going to
lost data locality for primary regions.
for (HStore s : r.stores.values()) {
if (s.hasReferences() || s.needsCompaction()) {
this.compactSplitThread.requestSystemCompaction(r, s, "Opening Region");
}
}
{code}
> Replica regions should not trigger any compaction
> -
>
> Key: HBASE-24581
> URL: https://issues.apache.org/jira/browse/HBASE-24581
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Critical
>
> I found that in certain cases replica regions can trigger compaction, one
> example as follows, need to check all places to avoid compaction for replica
> regions.
> {code:java}
> @Override
> public void postOpenDeployTasks(final PostOpenDeployContext context) throws
> IOException {
> HRegion r = context.getRegion();
> long openProcId = context.getOpenProcId();
> long masterSystemTime = context.getMasterSystemTime();
> rpcServices.checkOpen();
> LOG.info("Post open deploy tasks for {}, openProcId={},
> masterSystemTime={}",
> r.getRegionInfo().getRegionNameAsString(), openProcId, masterSystemTime);
> // Do checks to see if we need to compact (references or too many files)
> // TODO: SHX, do not do this for replica regions? Otherwise, it is going to
> lost data locality for primary regions.
> for (HStore s : r.stores.values()) {
> if (s.hasReferences() || s.needsCompaction()) {
> this.compactSplitThread.requestSystemCompaction(r, s, "Opening Region");
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24581) Replica regions should not trigger any compaction
Huaxiang Sun created HBASE-24581:
Summary: Replica regions should not trigger any compaction
Key: HBASE-24581
URL: https://issues.apache.org/jira/browse/HBASE-24581
Project: HBase
Issue Type: Bug
Components: read replicas
Affects Versions: 2.3.0
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun
I found that in certain cases replica regions can trigger compaction, such as
{code:java}
@Override
public void postOpenDeployTasks(final PostOpenDeployContext context) throws
IOException {
HRegion r = context.getRegion();
long openProcId = context.getOpenProcId();
long masterSystemTime = context.getMasterSystemTime();
rpcServices.checkOpen();
LOG.info("Post open deploy tasks for {}, openProcId={}, masterSystemTime={}",
r.getRegionInfo().getRegionNameAsString(), openProcId, masterSystemTime);
// Do checks to see if we need to compact (references or too many files)
// TODO: SHX, do not do this for replica regions? Otherwise, it is going to
lost data locality for primary regions.
for (HStore s : r.stores.values()) {
if (s.hasReferences() || s.needsCompaction()) {
this.compactSplitThread.requestSystemCompaction(r, s, "Opening Region");
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441837527
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
##
@@ -122,7 +123,9 @@ public void deleteSplitWAL(String wal) throws IOException {
public void deleteWALDir(ServerName serverName) throws IOException {
Path splitDir = getWALSplitDir(serverName);
-fs.delete(splitDir, false);
+if (!fs.delete(splitDir, false)) {
+ LOG.warn("Failed delete {}", splitDir);
Review comment:
Let me fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441837300
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
##
@@ -96,7 +97,7 @@ public SplitWALManager(MasterServices master) {
// 2. create corresponding procedures
return createSplitWALProcedures(splittingFiles, crashedServer);
} catch (IOException e) {
- LOG.error("failed to create procedures for splitting logs of {}",
crashedServer, e);
+ LOG.error("Failed to create procedures for splitting logs of {}",
crashedServer, e);
Review comment:
Let me do.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-19365) FSTableDescriptors#get() can return null reference, in some cases, it is not checked
[
https://issues.apache.org/jira/browse/HBASE-19365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser resolved HBASE-19365.
Hadoop Flags: Reviewed
Resolution: Fixed
Thanks for the review, [~vjasani]!
> FSTableDescriptors#get() can return null reference, in some cases, it is not
> checked
>
>
> Key: HBASE-19365
> URL: https://issues.apache.org/jira/browse/HBASE-19365
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.6
>Reporter: Hua Xiang
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 1.7.0
>
>
> In one of our cases, 1.2.0 based master could not start because the null
> reference is not checked. Master crashed because of the following exception.
> {code}
> 2017-11-20 08:30:20,178 FATAL org.apache.hadoop.hbase.master.HMaster: Failed
> to become active master
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2993)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:494)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:821)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:192)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1827)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-19365) FSTableDescriptors#get() can return null reference, in some cases, it is not checked
[
https://issues.apache.org/jira/browse/HBASE-19365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19365:
---
Fix Version/s: 1.7.0
> FSTableDescriptors#get() can return null reference, in some cases, it is not
> checked
>
>
> Key: HBASE-19365
> URL: https://issues.apache.org/jira/browse/HBASE-19365
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.6
>Reporter: Hua Xiang
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 1.7.0
>
>
> In one of our cases, 1.2.0 based master could not start because the null
> reference is not checked. Master crashed because of the following exception.
> {code}
> 2017-11-20 08:30:20,178 FATAL org.apache.hadoop.hbase.master.HMaster: Failed
> to become active master
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2993)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:494)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:821)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:192)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1827)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Apache-HBase commented on pull request #1913: URL: https://github.com/apache/hbase/pull/1913#issuecomment-645612480 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | branch-2 passed | | +1 :green_heart: | compile | 1m 3s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 47s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 5m 45s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 123m 31s | hbase-server in the patch passed. | | | | 149m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1913 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a5b94087c197 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/testReport/ | | Max. process+thread count | 4801 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23927) hbck2 assigns command should accept a file containing a list of region names
[ https://issues.apache.org/jira/browse/HBASE-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138788#comment-17138788 ] Clara Xiong commented on HBASE-23927: - I didn't add this option to unassigns. I would imagine that will be useful too. [~ndimiduk] What do you think? > hbck2 assigns command should accept a file containing a list of region names > > > Key: HBASE-23927 > URL: https://issues.apache.org/jira/browse/HBASE-23927 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2, Operability, Usability >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: Nick Dimiduk >Assignee: Clara Xiong >Priority: Major > Fix For: hbase-operator-tools-1.1.0 > > > The interface is not very ergonomic. Currently the command accepts a list of > region names on the command line. If you have 100's of regions to assign, > this sucks. We should accept a path to a file that contains these encoded > regions, one per line. That way, this command tails nicely into an operator's > incantation using grep/sed over log files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24580) Persist map of regions-to-last-flush-sequenceid across restarts
[ https://issues.apache.org/jira/browse/HBASE-24580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138771#comment-17138771 ] Nick Dimiduk commented on HBASE-24580: -- I don't actually know if this is a good idea or not, but seems like we could stream these entries to a pbuf file, maybe even reuse our existing WAL format. We have everything we need for durable streaming, and these entries shouldn't happen a too terribly high of a clip, so maybe? We'd need another chore to truncate the wal periodically, is that solved already too? > Persist map of regions-to-last-flush-sequenceid across restarts > --- > > Key: HBASE-24580 > URL: https://issues.apache.org/jira/browse/HBASE-24580 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Priority: Minor > > A nice suggestion by our [~ndimiduk] was that we persist the > last-successful-flush-sequenceid per region kept up by an in-memory map in > Master. We could add it to our nice new general local master store. On > restart, could rehydrate the map. Would be helpful on cold start in > particular, especially if a big backlog of WALs to replay. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1903: HBASE-24562: Stabilize master startup with meta replicas enabled
huaxiangsun commented on a change in pull request #1903:
URL: https://github.com/apache/hbase/pull/1903#discussion_r441783237
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1166,7 +1166,11 @@ private void
finishActiveMasterInitialization(MonitoredTask status) throws IOExc
assignmentManager.checkIfShouldMoveSystemRegionAsync();
status.setStatus("Assign meta replicas");
MasterMetaBootstrap metaBootstrap = createMetaBootstrap();
-metaBootstrap.assignMetaReplicas();
+try {
+ metaBootstrap.assignMetaReplicas();
+} catch (HBaseIOException e){
Review comment:
assignMetaReplicas() throws IOException, InterruptedException, and
KeeperException. Why only HBaseIOException is processed here? Should it catch
all exceptions and totally ignore all exceptions caused by assignMetaReplicas()?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -616,6 +616,24 @@ public long assign(RegionInfo regionInfo) throws
IOException {
return assign(regionInfo, null);
}
+ public Future assignAsync(RegionInfo regionInfo, ServerName sn)
throws IOException {
Review comment:
Most of this method's code is a duplicate of the assign() above, the
common part can be wrapped in a private method.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -616,6 +616,24 @@ public long assign(RegionInfo regionInfo) throws
IOException {
return assign(regionInfo, null);
}
+ public Future assignAsync(RegionInfo regionInfo, ServerName sn)
throws IOException {
Review comment:
If we are adding a new public method, can we have doc for these two new
methods?
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
##
@@ -355,4 +362,58 @@ public void testShutdownOfReplicaHolder() throws Exception
{
assertNotEquals(3, i);
}
}
+
+ @Test
+ public void testFailedReplicaAssigment() throws InterruptedException,
IOException {
Review comment:
Have one question about the test case, do not see any exceptions thrown
out during assigning meta replica regions, how do we make sure the following
code is working.
```
try {
metaBootstrap.assignMetaReplicas();
} catch (HBaseIOException e){
```
If we can make sure that the test can compile with or without the changes
in non-test modules, it will be great as we can verify that without those
changes, the test will fail. Seems with current implementation, it is not
possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-24575) Cleanup the build directory structure
[
https://issues.apache.org/jira/browse/HBASE-24575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Vissapragada updated HBASE-24575:
-
Fix Version/s: master
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Cleanup the build directory structure
> -
>
> Key: HBASE-24575
> URL: https://issues.apache.org/jira/browse/HBASE-24575
> Project: HBase
> Issue Type: Sub-task
> Components: Client, native-client
>Affects Versions: master
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Minor
> Fix For: master
>
>
> Currently all the build binaries (along with shared-libs and dependencies)
> are installed in the same current build directory. That is super messy to
> work with. It looks something like this
> {noformat}
> .
> ├── apachehbase-build
> ├── apachehbase-download
> ├── apachehbase-src
> ├── append-test
> ├── async-batch-rpc-retrying-test
> ├── async-rpc-retrying-test
> ├── build
> ├── bytes-util-test
> ├── cell-test
> ├── client-deserializer-test
> ├── client-serializer-test
> ├── client-test
> ├── CMakeCache.txt
> ├── CMakeFiles
> ├── cmake_install.cmake
> ├── concurrent-map-test
> ├── configuration-test
> ├── connection-pool-test
> ├── CTestTestfile.cmake
> ├── delete-test
> ├── dependencies
> ├── exception-test
> ├── facebook-folly-proj-prefix
> ├── facebook-wangle-proj-prefix
> ├── filter-test
> ├── get-test
> ├── hbase-configuration-test
> ├── increment-test
> ├── libhbaseclient.a
> ├── libhbaseclient.so
> ├── libtestutil.a
> ├── load-client
> ├── location-cache-retry-test
> ├── location-cache-test
> ├── Makefile
> ├── Protobuf-build
> ├── Protobuf-download
> ├── Protobuf-src
> ├── put-test
> ├── region-info-deserializer-test
> ├── request-converter-test
> ├── result-test
> ├── rpc-test
> ├── scanner-test
> ├── scan-result-cache-test
> ├── scan-test
> ├── server-name-test
> ├── simple-client
> ├── table-name-test
> ├── target
> ├── Testing
> ├── time-range-test
> ├── user-util-test
> ├── zk-deserializer-test
> ├── zk-util-test
> └── ZooKeeper-prefix
> {noformat}
> I have a patch that switches the build layout to the following (logically
> grouped). Very clean and easy to work with.
> {noformat}
> .
> ├── bin
> ├── build
> ├── CMakeCache.txt
> ├── CMakeFiles
> ├── cmake_install.cmake
> ├── CTestTestfile.cmake
> ├── dependencies
> ├── libs
> ├── Makefile
> └── tests
> {noformat}
> With Depth 2 (to show the layout better)
> {noformat}
> ├── bin
> │ └── examples
> ├── build
> │ └── test-data
> ├── CMakeCache.txt
> ├── CMakeFiles
> │ ├── 3.10.2
> │ ├── append-test.dir
> │ ├── async-batch-rpc-retrying-test.dir
> │ ├── async-rpc-retrying-test.dir
> │ ├── bytes-util-test.dir
> │ ├── cell-test.dir
> │ ├── CheckLibraryExists
> │ ├── client-deserializer-test.dir
> │ ├── client-serializer-test.dir
> │ ├── client-test.dir
> │ ├── cmake.check_cache
> │ ├── CMakeDirectoryInformation.cmake
> │ ├── CMakeError.log
> │ ├── CMakeOutput.log
> │ ├── CMakeRuleHashes.txt
> │ ├── CMakeTmp
> │ ├── concurrent-map-test.dir
> │ ├── configuration-test.dir
> │ ├── connection-pool-test.dir
> │ ├── delete-test.dir
> │ ├── exception-test.dir
> │ ├── facebook-folly-proj-complete
> │ ├── facebook-folly-proj.dir
> │ ├── facebook-wangle-proj-complete
> │ ├── facebook-wangle-proj.dir
> │ ├── feature_tests.bin
> │ ├── feature_tests.cxx
> │ ├── filter-test.dir
> │ ├── get-test.dir
> │ ├── hbase
> │ ├── hbaseclient-shared.dir
> │ ├── hbaseclient-static.dir
> │ ├── hbase-configuration-test.dir
> │ ├── increment-test.dir
> │ ├── linter.dir
> │ ├── load-client.dir
> │ ├── location-cache-retry-test.dir
> │ ├── location-cache-test.dir
> │ ├── Makefile2
> │ ├── Makefile.cmake
> │ ├── progress.marks
> │ ├── put-test.dir
> │ ├── region-info-deserializer-test.dir
> │ ├── request-converter-test.dir
> │ ├── result-test.dir
> │ ├── rpc-test.dir
> │ ├── scanner-test.dir
> │ ├── scan-result-cache-test.dir
> │ ├── scan-test.dir
> │ ├── server-name-test.dir
> │ ├── simple-client.dir
> │ ├── table-name-test.dir
> │ ├── TargetDirectories.txt
> │ ├── testutil.dir
> │ ├── time-range-test.dir
> │ ├── user-util-test.dir
> │ ├── zk-deserializer-test.dir
> │ ├── zk-util-test.dir
> │ ├── ZooKeeper-complete
> │ └── ZooKeeper.dir
> ├── cmake_install.cmake
> ├── CTestTestfile.cmake
> ├── dependencies
> │ ├── apache-hbase-build
> │ ├── apache-hbase-download
> │ ├── apache-hbase-src
> │ ├── facebook-folly-proj-install
> │ ├── facebook-folly-proj-src
> │ ├── facebook-wangle-proj-download
> │ ├── facebook-wangle-proj-install
> │ ├── facebook-wangle-proj-src
> │ ├── Protobuf-build
> │ ├── Protobuf-download
> │ ├── Protobuf-src
> │ ├── src
> │ ├──
[GitHub] [hbase-native-client] bharathv merged pull request #9: HBASE-24575: Cleanup the build directory structure
bharathv merged pull request #9: URL: https://github.com/apache/hbase-native-client/pull/9 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-native-client] bharathv commented on a change in pull request #9: HBASE-24575: Cleanup the build directory structure
bharathv commented on a change in pull request #9:
URL: https://github.com/apache/hbase-native-client/pull/9#discussion_r441772862
##
File path: cmake/DownloadWangle.cmake
##
@@ -20,24 +20,24 @@
## BINARY_DIR is the build directory, typically 'build'
function(download_wangle SOURCE_DIR BUILD_DIR)
-
- if (DOWNLOAD_DEPENDENCIES)
- SET (PATCH_FOLLY ${CMAKE_COMMAND} -E copy
- "${CMAKE_CURRENT_SOURCE_DIR}/cmake/folly/local/FindFolly.cmake"
${BUILD_DIR}/facebook-wangle-proj-prefix/src/facebook-wangle-proj/wangle/cmake )
- else()
- SET (PATCH_FOLLY "")
- endif()
-
- ExternalProject_Add(
- facebook-wangle-proj
- URL
"https://github.com/facebook/wangle/archive/v2017.09.04.00.tar.gz";
- PATCH_COMMAND ${PATCH_FOLLY}
- INSTALL_DIR
"${BUILD_DIR}/dependencies/facebook-wangle-proj-install"
-CONFIGURE_COMMAND ${CMAKE_COMMAND} -DBUILD_EXAMPLES=OFF
-DCMAKE_CROSSCOMPILING=ON -DBUILD_TESTS=OFF -DFOLLY_ROOT_DIR=${FOLLY_ROOT_DIR}
-DCMAKE_INSTALL_PREFIX:PATH=${BUILD_DIR}/dependencies/facebook-wangle-proj-install
-
${BUILD_DIR}/facebook-wangle-proj-prefix/src/facebook-wangle-proj/wangle # Tell
CMake to use subdirectory as source.
- )
-
-
- set(WANGLE_ROOT_DIR
"${BUILD_DIR}/dependencies/facebook-wangle-proj-install" CACHE STRING "" FORCE)
+ set(WANGLE_DOWNLOAD_DIR
"${BUILD_DIR}/dependencies/facebook-wangle-proj-download")
Review comment:
No worries.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441782616
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
##
@@ -365,4 +363,17 @@ protected void
deserializeStateData(ProcedureStateSerializer serializer) throws
protected void afterReplay(MasterProcedureEnv env) {
getParent(env).attachRemoteProc(this);
}
+
+ @Override public String getProcName() {
Review comment:
proc name should be short. stringclassdetails should be all parameters.
Sometimes the name and the stringclassdetails are the same if there is nothing
much to show for a procedure.
In this case, I think target server is a fundamental attribute of
RegionRemoteProcedure so thought it should be in the proc name. Could move it
to the string details.Let me do that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441773327
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
##
@@ -192,6 +189,21 @@ protected void afterReplay(MasterProcedureEnv env){
if(worker != null){
env.getMasterServices().getSplitWALManager().addUsedSplitWALWorker(worker);
}
+ }
+
+ @Override protected void toStringClassDetails(StringBuilder builder) {
+builder.append(getProcName());
+if (this.worker != null) {
+ builder.append(", splitWorker=");
+ builder.append(this.worker);
+}
+if (this.retryCounter != null) {
+ builder.append(", retry=");
+ builder.append(this.retryCounter);
+}
+ }
+ @Override public String getProcName() {
+return getClass().getSimpleName() + " " + WAL.stripWALsDirPrefix(getWAL());
Review comment:
hmm... I only have a string... but let me just output the file name
instead.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-24538) Deadlock in mini-cluster-test teardown on setup() failure
[
https://issues.apache.org/jira/browse/HBASE-24538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Vissapragada updated HBASE-24538:
-
Fix Version/s: master
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Deadlock in mini-cluster-test teardown on setup() failure
> -
>
> Key: HBASE-24538
> URL: https://issues.apache.org/jira/browse/HBASE-24538
> Project: HBase
> Issue Type: Sub-task
> Components: Client, native-client
>Affects Versions: 3.0.0-alpha-1
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: master
>
>
> Code seems to be weirdly strucutred. Cleanup the life cycle
> {noformat}
> #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
> #1 0x7f4e2ce84023 in __GI___pthread_mutex_lock (mutex=0x55d922b810e8) at
> ../nptl/pthread_mutex_lock.c:78
> #2 0x55d9207b263a in __gthread_mutex_lock ()
> #3 0x55d9207b3bf4 in std::mutex::lock() ()
> #4 0x55d9207c2d88 in
> std::lock_guard::lock_guard(std::mutex&) () <== Attempts to lock
> mutex again.
> #5 0x55d9208cbf86 in hbase::MiniCluster::Setup() ()
> #6 0x55d9208cc87c in hbase::MiniCluster::env() ()
> #7 0x55d9208cd33b in hbase::MiniCluster::StopCluster() ()
> #8 0x55d9208c8ace in hbase::TestUtil::StopMiniCluster() ()
> #9 0x55d9208c86d8 in hbase::TestUtil::~TestUtil() ()
> #10 0x55d92061bbb0 in
> std::default_delete::operator()(hbase::TestUtil*) const ()
> #11 0x55d920616c17 in std::unique_ptr std::default_delete >::~unique_ptr() ()
> #12 0x7f4e29e96041 in __run_exit_handlers (status=-1,
> listp=0x7f4e2a23e718 <__exit_funcs>,
> run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true)
> at exit.c:108
> #13 0x7f4e29e9613a in __GI_exit (status=) at exit.c:139
> #14 0x55d9208cbb67 in hbase::MiniCluster::CreateVM(JavaVM_**) ()
> #15 0x55d9208cbfb1 in hbase::MiniCluster::Setup() () <== Locks mutex
> #16 0x55d9208cc87c in hbase::MiniCluster::env() ()
> #17 0x55d9208cd227 in hbase::MiniCluster::StartCluster(int) ()
> #18 0x55d9208c879b in hbase::TestUtil::StartMiniCluster(int) ()
> #19 0x55d920613076 in AsyncBatchRpcRetryTest::SetUpTestCase() ()
> #20 0x55d9208eaa48 in testing::TestCase::RunSetUpTestCase() ()
> #21 0x55d9208f7347 in void
> testing::internal::HandleSehExceptionsInMethodIfSupported void>(testing::TestCase*, void (testing::TestCase::*)(), char const*) ()
> #22 0x55d9208f1b19 in void
> testing::internal::HandleExceptionsInMethodIfSupported void>(testing::TestCase*, void (testing::TestCase::*)(), char const*) ()
> #23 0x55d9208d6647 in testing::TestCase::Run() ()
> #24 0x55d9208dd552 in testing::internal::UnitTestImpl::RunAllTests() ()
> #25 0x55d9208f7d29 in bool
> testing::internal::HandleSehExceptionsInMethodIfSupported bool>(testing::internal::UnitTestImpl*, bool
> (testing::internal::UnitTestImpl::*)(), char const*) ()
> #26 0x55d9208f21a5 in bool
> testing::internal::HandleExceptionsInMethodIfSupported bool>(testing::internal::UnitTestImpl*, bool
> (testing::internal::UnitTestImpl::*)(), char const*) ()
> #27 0x55d9208dc116 in testing::UnitTest::Run() ()
> #28 0x55d920904ce7 in RUN_ALL_TESTS() ()
> #29 0x55d920904c76 in main ()
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #1912: HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING
saintstack commented on a change in pull request #1912:
URL: https://github.com/apache/hbase/pull/1912#discussion_r441769423
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
##
@@ -192,6 +189,21 @@ protected void afterReplay(MasterProcedureEnv env){
if(worker != null){
env.getMasterServices().getSplitWALManager().addUsedSplitWALWorker(worker);
}
+ }
+
+ @Override protected void toStringClassDetails(StringBuilder builder) {
+builder.append(getProcName());
+if (this.worker != null) {
+ builder.append(", splitWorker=");
+ builder.append(this.worker);
+}
+if (this.retryCounter != null) {
+ builder.append(", retry=");
+ builder.append(this.retryCounter);
+}
+ }
+ @Override public String getProcName() {
+return getClass().getSimpleName() + " " + WAL.stripWALsDirPrefix(getWAL());
Review comment:
You idea is better. Let me do it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-native-client] bharathv merged pull request #7: HBASE-24538: Simplify the lifecycle of mini cluster
bharathv merged pull request #7: URL: https://github.com/apache/hbase-native-client/pull/7 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-native-client] bharathv commented on pull request #7: HBASE-24538: Simplify the lifecycle of mini cluster
bharathv commented on pull request #7: URL: https://github.com/apache/hbase-native-client/pull/7#issuecomment-645563707 Thanks for the review @phrocker. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-24580) Persist map of regions-to-last-flush-sequenceid across restarts
Michael Stack created HBASE-24580: - Summary: Persist map of regions-to-last-flush-sequenceid across restarts Key: HBASE-24580 URL: https://issues.apache.org/jira/browse/HBASE-24580 Project: HBase Issue Type: Improvement Reporter: Michael Stack A nice suggestion by our [~ndimiduk] was that we persist the last-successful-flush-sequenceid per region kept up by an in-memory map in Master. We could add it to our nice new general local master store. On restart, could rehydrate the map. Would be helpful on cold start in particular, especially if a big backlog of WALs to replay. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
virajjasani commented on a change in pull request #1921:
URL: https://github.com/apache/hbase/pull/1921#discussion_r441756545
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java
##
@@ -148,6 +148,16 @@ public boolean saslConnect(InputStream inS, OutputStream
outS) throws IOExceptio
inStream.readFully(saslToken);
}
}
+
+ try {
+readStatus(inStream);
+ }
+ catch (IOException e){
+if(e instanceof RemoteException){
Review comment:
I think you removed the outer condition instead of `if(e instanceof
RemoteException)`. Let me copy my previous comment:
I know `RemoteException` coming from `readStatus` is imp to us but we don't
want to swallow IOException if `inStream.readInt()` throws one because
`inStream.readInt()` is pre-requisite to know whether the status is
`SaslStatus.SUCCESS`
I am talking about what is happening within `readStatus(inStream)` call.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java
##
@@ -148,6 +148,16 @@ public boolean saslConnect(InputStream inS, OutputStream
outS) throws IOExceptio
inStream.readFully(saslToken);
}
}
+
+ try {
+readStatus(inStream);
+ }
+ catch (IOException e){
+if(e instanceof RemoteException){
Review comment:
I think you removed the outer condition instead of
`if(e instanceof RemoteException)`.
Let me copy my previous comment:
I know `RemoteException` coming from `readStatus` is imp to us but we don't
want to swallow IOException if `inStream.readInt()` throws one because
`inStream.readInt()` is pre-requisite to know whether the status is
`SaslStatus.SUCCESS`
I am talking about what is happening within `readStatus(inStream)` call.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
virajjasani commented on a change in pull request #1921:
URL: https://github.com/apache/hbase/pull/1921#discussion_r441756545
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java
##
@@ -148,6 +148,16 @@ public boolean saslConnect(InputStream inS, OutputStream
outS) throws IOExceptio
inStream.readFully(saslToken);
}
}
+
+ try {
+readStatus(inStream);
+ }
+ catch (IOException e){
+if(e instanceof RemoteException){
Review comment:
I think you removed the outer condition. Let me copy my previous comment:
I know `RemoteException` coming from `readStatus` is imp to us but we don't
want to swallow IOException if `inStream.readInt()` throws one because
`inStream.readInt()` is pre-requisite to know whether the status is
`SaslStatus.SUCCESS`
I am talking about what is happening within `readStatus(inStream)` call.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
virajjasani commented on a change in pull request #1921:
URL: https://github.com/apache/hbase/pull/1921#discussion_r441752901
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java
##
@@ -148,6 +148,16 @@ public boolean saslConnect(InputStream inS, OutputStream
outS) throws IOExceptio
inStream.readFully(saslToken);
}
}
+
+ try {
+readStatus(inStream);
+ }
+ catch (IOException e){
+if(e instanceof RemoteException){
Review comment:
We still have this condition? I was talking about removing this
condition because with this condition, we are swallowing `IOException` other
than `RemoteException`, which we don't want to do. We want to re-throw all
categories of `IOException` here, not just `RemoteException`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
virajjasani commented on a change in pull request #1921:
URL: https://github.com/apache/hbase/pull/1921#discussion_r441752901
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java
##
@@ -148,6 +148,16 @@ public boolean saslConnect(InputStream inS, OutputStream
outS) throws IOExceptio
inStream.readFully(saslToken);
}
}
+
+ try {
+readStatus(inStream);
+ }
+ catch (IOException e){
+if(e instanceof RemoteException){
Review comment:
We still have this condition? I was talking about removing this
condition because with this condition, we are swallowing `IOException` other
than `RemoteException`, which we don't want to do.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1913: HBASE-24577 Doc WALSplitter classes
Apache-HBase commented on pull request #1913: URL: https://github.com/apache/hbase/pull/1913#issuecomment-645552689 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 9s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 59s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 10s | the patch passed | | -0 :warning: | checkstyle | 1m 7s | hbase-server: The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 32m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1913 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux ff480c5257e3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e202c98fb0 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1913/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441750599
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws
IOException {
}
}
-// The key point is create a new writer for each column family, write
edits then close writer.
+// Create a new hfile writer for each column family, write edits then
close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName,
regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
You are right, I just looked into one of the implementors of slf4j
(Log4jLoggerAdapter) and it already covers `isTraceEnabled` check:
```
public void trace(String format, Object arg) {
if (isTraceEnabled()) {
FormattingTuple ft = MessageFormatter.format(format, arg);
logger.log(FQCN, traceCapable ? Level.TRACE : Level.DEBUG,
ft.getMessage(), ft.getThrowable());
}
}
```
We are following explicit guarding for the cases of Trace and Debug level
log at multiple places, all are redundant checks since library already does it
for us.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441746876
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws
IOException {
}
}
-// The key point is create a new writer for each column family, write
edits then close writer.
+// Create a new hfile writer for each column family, write edits then
close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName,
regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
My bad, this is old doc for apache log4j, not slf4j
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
virajjasani commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441745855
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws
IOException {
}
}
-// The key point is create a new writer for each column family, write
edits then close writer.
+// Create a new hfile writer for each column family, write edits then
close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName,
regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
This is to avoid the cost of parameter construction:
http://logging.apache.org/log4j/1.2/manual.html#performance
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (HBASE-24380) Improve WAL splitting log lines to enable sessionization
[ https://issues.apache.org/jira/browse/HBASE-24380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133176#comment-17133176 ] Viraj Jasani edited comment on HBASE-24380 at 6/17/20, 6:10 PM: [~apurtell] If you get some bandwidth, please review: [https://github.com/apache/hbase/pull/1860] (The sample output is available in 2nd comment of this Jira and in the PR description). Edit: Changes are incorporated as per the latest review. was (Author: vjasani): [~apurtell] If you get some bandwidth, please review: [https://github.com/apache/hbase/pull/1860] (The sample output is available in 2nd comment of this Jira and in the PR description). > Improve WAL splitting log lines to enable sessionization > > > Key: HBASE-24380 > URL: https://issues.apache.org/jira/browse/HBASE-24380 > Project: HBase > Issue Type: Improvement > Components: logging, Operability, wal >Reporter: Andrew Kyle Purtell >Assignee: Viraj Jasani >Priority: Minor > > Looking to reconstruct a timeline from write of recovered.edits file back to > start of WAL file split, with a bunch of unrelated activity in the meantime, > there isn't a consistent token that links split file write messages (which > include store path including region hash) to beginning of WAL splitting > activity. Sessonizing by host doesn't work because work can bounce around > through retries. Thread context names in the logs vary and can be like > [nds1-225-fra:60020-7] or [fb472085572ba72e96f1] (trailing digits of region > hash) or [splits-1589016325868] . > We could have WALSplitter get the current time when starting the split of a > WAL file and have it log this timestamp in every line as a splitting session > identifier. > Related, we should track the time of split task execution end to end and > export a metric that captures it. > It might also be worthwhile to wire up more of WAL splitting to TaskMonitor > status logging. If we do this we can also enable status journal logging, so > when a WAL split has completed, a line will appear in the log that has the > list of all status messages recorded during splitting and the time delta in > milliseconds between them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24528) Improve balancer decision observability
[
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138701#comment-17138701
]
Viraj Jasani commented on HBASE-24528:
--
{quote}That means, we could use an in memory queue, disruptor is also fine(but
maybe a bit overkill?), to cache the records we want to save to the system
table. There will be a background task to flush the records. If the queue is
full we just drop the new records.
{quote}
All of these actions are implemented for Admin API getSlowLogResponses() except
for the last one. Instead of dropping new records if the queue is full, we will
be removing oldest inserted record from the head of the queue and add new
record at the tail. EvictingQueue is taking care of this.
Now we need this entire workflow as service (framework) to be consumed by
multiple clients.
HBASE-23938 uses system table for persistence if related config is turned on
for slow log, however enabling writes to system table for each named queue
(created dynamically) might be a challenge. For framework, we can create
namedQueues dynamically and provide Add and Get APIs for respective queues
(queue per use-case) but we might not want to create system tables for
persistence of each namedQueues records.
If we just use one system table for all named queues, we will encounter
indexing issue i.e fetch all records of a specific use-case, specific client.
On the other hand, if we have system table for each use-case, that might be too
many system tables with growing use-cases.
> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> We provide detailed INFO and DEBUG level logging of balancer decision
> factors, outcome, and reassignment planning, as well as similarly detailed
> logging of the resulting assignment manager activity. However, an operator
> may need to perform online and interactive observation, debugging, or
> performance analysis of current balancer activity. Scraping and correlating
> the many log lines resulting from a balancer execution is labor intensive and
> has a lot of latency (order of ~minutes to acquire and index, order of
> ~minutes to correlate).
> The balancer should maintain a rolling window of history, e.g. the last 100
> region move plans, or last 1000 region move plans submitted to the assignment
> manager. This history should include decision factor details and weights and
> costs. The rsgroups balancer may be able to provide fairly simple decision
> factors, like for example "this table was reassigned to that regionserver
> group". The underlying or vanilla stochastic balancer on the other hand,
> after a walk over random assignment plans, will have considered a number of
> cost functions with various inputs (locality, load, etc.) and multipliers,
> including custom cost functions. We can devise an extensible class structure
> that represents explanations for balancer decisions, and for each region move
> plan that is actually submitted to the assignment manager, we can keep the
> explanations of all relevant decision factors alongside the other details of
> the assignment plan like the region name, and the source and destination
> regionservers.
> This history should be available via API for use by new shell commands and
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of
> balancer decision components into human readable output.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack commented on a change in pull request #1913: URL: https://github.com/apache/hbase/pull/1913#discussion_r441725369 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java ## @@ -115,7 +119,18 @@ private final String tmpDirName; + /** + * Split WAL directly to hfiles instead of into intermediary 'recovered.edits' files. + */ + public static final String WAL_SPLIT_TO_HFILE = "hbase.wal.split.to.hfile"; + public static final boolean DEFAULT_WAL_SPLIT_TO_HFILE = false; + + /** + * True if we are to run with bounded amount of writers rather than let the count blossom. + * Bounded writing tends to have higher throughput. + */ public final static String SPLIT_WRITER_CREATION_BOUNDED = "hbase.split.writer.creation.bounded"; Review comment: I note that it defaults to 'false' but man, this should be 'true'. Let me test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack commented on a change in pull request #1913: URL: https://github.com/apache/hbase/pull/1913#discussion_r441720952 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java ## @@ -115,7 +119,18 @@ private final String tmpDirName; + /** + * Split WAL directly to hfiles instead of into intermediary 'recovered.edits' files. + */ + public static final String WAL_SPLIT_TO_HFILE = "hbase.wal.split.to.hfile"; + public static final boolean DEFAULT_WAL_SPLIT_TO_HFILE = false; + + /** + * True if we are to run with bounded amount of writers rather than let the count blossom. + * Bounded writing tends to have higher throughput. + */ public final static String SPLIT_WRITER_CREATION_BOUNDED = "hbase.split.writer.creation.bounded"; Review comment: Let me add this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack commented on a change in pull request #1913: URL: https://github.com/apache/hbase/pull/1913#discussion_r441721127 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java ## @@ -26,20 +26,20 @@ import java.util.concurrent.ConcurrentMap; import java.util.concurrent.ExecutionException; import java.util.concurrent.Future; - import org.apache.hadoop.fs.Path; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.io.MultipleIOException; import org.apache.yetus.audience.InterfaceAudience; import org.slf4j.Logger; import org.slf4j.LoggerFactory; - import org.apache.hbase.thirdparty.com.google.common.collect.Lists; /** * Class that manages the output streams from the log splitting process. - * Every region only has one recovered edits. + * Every region only has one recovered edits file. Review comment: Let me add qualification here. ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java ## @@ -233,7 +247,7 @@ public static boolean splitLogFile(Path walDir, FileStatus logfile, FileSystem w } /** - * log splitting implementation, splits one log file. + * Log splitting implementation, splits one log file. Review comment: Let me change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1913: HBASE-24577 Doc WALSplitter classes
saintstack commented on a change in pull request #1913:
URL: https://github.com/apache/hbase/pull/1913#discussion_r441720655
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
##
@@ -99,12 +102,13 @@ public void append(RegionEntryBuffer buffer) throws
IOException {
}
}
-// The key point is create a new writer for each column family, write
edits then close writer.
+// Create a new hfile writer for each column family, write edits then
close writer.
String regionName = Bytes.toString(buffer.encodedRegionName);
for (Map.Entry cellsEntry : familyCells.entrySet()) {
String familyName = cellsEntry.getKey();
StoreFileWriter writer = createRecoveredHFileWriter(buffer.tableName,
regionName,
familySeqIds.get(familyName), familyName, isMetaTable);
+ LOG.trace("Created {}", writer.getPath());
Review comment:
Why you say? The slf4j doesn't do any work creating log string if we are
not at log TRACE level.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-23927) hbck2 assigns command should accept a file containing a list of region names
[ https://issues.apache.org/jira/browse/HBASE-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138680#comment-17138680 ] Michael Stack commented on HBASE-23927: --- Nice release note [~clarax98007] Does this work for unassigns? Should it? > hbck2 assigns command should accept a file containing a list of region names > > > Key: HBASE-23927 > URL: https://issues.apache.org/jira/browse/HBASE-23927 > Project: HBase > Issue Type: Improvement > Components: hbase-operator-tools, hbck2, Operability, Usability >Affects Versions: hbase-operator-tools-1.0.0 >Reporter: Nick Dimiduk >Assignee: Clara Xiong >Priority: Major > Fix For: hbase-operator-tools-1.1.0 > > > The interface is not very ergonomic. Currently the command accepts a list of > region names on the command line. If you have 100's of regions to assign, > this sucks. We should accept a path to a file that contains these encoded > regions, one per line. That way, this command tails nicely into an operator's > incantation using grep/sed over log files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24482.
---
Fix Version/s: hbase-operator-tools-1.1.0
Release Note: Make it so tests and compile works against hbase-2.3.x as
well as hbase-2.1.x.
Assignee: Michael Stack
Resolution: Fixed
Resolving. Pushed w/ below commit:
{code}
commit 8d716173b6f713d3b375f9eef18189cf2f1a3745 (HEAD -> master, origin/master,
origin/HEAD)
Author: stack
Date: Wed Jun 17 09:49:32 2020 -0700
HBASE-24482 [hbase-operator-tools] build of hbck2 fails with HBase
branch-2.3, due to missing dependencies
Make it so we compile and tests pass against hbase-2.3.x as
well as 2.1.x.
Remove use of @Nullable -- nice-to-have but dodgy provenance.
Use CommonFSUtils instead of FSUtils as latter no longer subclasses
CommonFSUtils whose utility we make use of in our import of hbck1
functionality (TODO: Undo this reliance on IA.Private)
hbase-hbck2/src/main/java/org/apache/hbase/FileSystemFsck.java
hbase-hbck2/src/main/java/org/apache/hbase/FsRegionsMetaRecoverer.java
hbase-hbck2/src/main/java/org/apache/hbase/hbck1/HBaseFsck.java
hbase-hbck2/src/main/java/org/apache/hbase/hbck1/HBaseFsckRepair.java
hbase-hbck2/src/main/java/org/apache/hbase/hbck1/HFileCorruptionChecker.java
hbase-hbck2/src/main/java/org/apache/hbase/hbck1/OfflineMetaRepair.java
hbase-hbck2/src/test/java/org/apache/hbase/TestHBCK2.java
s/CommonFSUtils/FSUtils/g
hbase-hbck2/src/main/java/org/apache/hbase/HBCKMetaTableAccessor.java
Purge @Nullable usage.
hbase-hbck2/src/test/java/org/apache/hbase/TestSchedulingRecoveries.java
Remove test that behaves differently between 2.1.x and 2.3.x hbase.
hbase-tools/src/test/java/org/apache/hbase/TestRegionsMerger.java
Amend test where merging works differently between hbase-2.3.x and
hbase-2.1.x (2.3 does more protective checks).
{code}
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Assignee: Michael Stack
>Priority: Major
> Fix For: hbase-operator-tools-1.1.0
>
> Attachments:
> 0001-HBASE-24482-hbase-operator-tools-build-of-hbck2-fail.patch,
> HBASE-24482_compile_error.log, HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138644#comment-17138644
]
Michael Stack commented on HBASE-24482:
---
Attached patch I pushed (mostly just s/FSUtils/CommonFSUtils/g and purge of the
@Nullable that came in w/ HBCKMetaTableAccessor.
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments:
> 0001-HBASE-24482-hbase-operator-tools-build-of-hbck2-fail.patch,
> HBASE-24482_compile_error.log, HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24482) [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to missing dependencies
[
https://issues.apache.org/jira/browse/HBASE-24482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24482:
--
Attachment: 0001-HBASE-24482-hbase-operator-tools-build-of-hbck2-fail.patch
> [hbase-operator-tools] build of hbck2 fails with HBase branch-2.3, due to
> missing dependencies
> --
>
> Key: HBASE-24482
> URL: https://issues.apache.org/jira/browse/HBASE-24482
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.1.0
>Reporter: Matthew Foley
>Priority: Major
> Attachments:
> 0001-HBASE-24482-hbase-operator-tools-build-of-hbck2-fail.patch,
> HBASE-24482_compile_error.log, HBASE-24482_missing_symbols_part_2.log
>
>
> If we do a local build of the current HBase branch-2.3 as version
> 2.3.0-SNAPSHOT, and 'mvn install' it in the local maven repository,
> then attempt to build hbase-operator-tools master branch, with
> -Dhbase.version=2.3.0-SNAPSHOT,
> the HBCK2 class file HBCKMetaTableAccessor.java (line 25) fails to compile
> with error: "package edu.umd.cs.findbugs.annotations does not exist",
> followed by multiple instances of missing "symbol: class Nullable". (A
> longer log extract is attached.)
> It would appear the default HBase version 2.1.6 made this dependency package
> available in some manner, but the newer HBase does not.
> *Later Edit:* After addressing this missing dependency (see comment of 5 June
> below), it unmasked a bunch more, in the hbck1 code path. Please see second
> attachment.
> If the @Nullable annotation is still needed, it may be sufficient to simply
> add it to the hbase-operator-tools pom.xml file, so maven will load it.
> However, it is actually deprecated in the current documentation for
> edu.umd.cs.findbugs.annotations. Various commentary online suggest that we
> should be using the similar jsr305.jar (*{{javax.annotation)}}* instead; see
> eg
> *
> [https://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use]
> Does anyone know if that was HBase's approach?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1783: HBASE-24436 The store file open and close thread pool should be share…
Apache-HBase commented on pull request #1783: URL: https://github.com/apache/hbase/pull/1783#issuecomment-645478490 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 59s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 5m 47s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 38s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 7m 14s | hbase-server in the patch failed. | | | | 35m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1783 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5cb8a8e520a8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/testReport/ | | Max. process+thread count | 707 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1783: HBASE-24436 The store file open and close thread pool should be share…
Apache-HBase commented on pull request #1783: URL: https://github.com/apache/hbase/pull/1783#issuecomment-645476512 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | master passed | | +1 :green_heart: | checkstyle | 1m 8s | master passed | | +1 :green_heart: | spotbugs | 1m 56s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 5s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 32m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1783 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux a7e3aa158588 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1783: HBASE-24436 The store file open and close thread pool should be share…
Apache-HBase commented on pull request #1783: URL: https://github.com/apache/hbase/pull/1783#issuecomment-645475400 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 6m 53s | hbase-server in the patch failed. | | | | 30m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1783 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d2b1157f648a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/testReport/ | | Max. process+thread count | 924 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1783/10/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
Apache-HBase commented on pull request #1921: URL: https://github.com/apache/hbase/pull/1921#issuecomment-645474524 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | checkstyle | 0m 29s | master passed | | +1 :green_heart: | spotbugs | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | checkstyle | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 1m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 29m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1921 | | JIRA Issue | HBASE-24579 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 4c9ade9256a7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
Apache-HBase commented on pull request #1921: URL: https://github.com/apache/hbase/pull/1921#issuecomment-645473949 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 52s | master passed | | +1 :green_heart: | compile | 0m 29s | master passed | | +1 :green_heart: | shadedjars | 6m 31s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 26s | hbase-client in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 31s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | shadedjars | 6m 28s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 25s | hbase-client in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 25s | hbase-client in the patch passed. | | | | 28m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1921 | | JIRA Issue | HBASE-24579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 02c70559fc1d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/testReport/ | | Max. process+thread count | 200 (vs. ulimit of 12500) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1921: HBASE-24579: Failed SASL authentication does not result in an exception on client side
Apache-HBase commented on pull request #1921: URL: https://github.com/apache/hbase/pull/1921#issuecomment-64547 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | shadedjars | 6m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | shadedjars | 6m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 13s | hbase-client in the patch passed. | | | | 24m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1921 | | JIRA Issue | HBASE-24579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ab96766f3b19 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 192daded61 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/testReport/ | | Max. process+thread count | 219 (vs. ulimit of 12500) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1921/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1883: HBASE-24530 Introduce a split policy similar with SteppingSplitPolicy…
Apache-HBase commented on pull request #1883: URL: https://github.com/apache/hbase/pull/1883#issuecomment-645471513 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | master passed | | +1 :green_heart: | compile | 1m 9s | master passed | | +1 :green_heart: | shadedjars | 6m 17s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 31s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 6m 20s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 189m 38s | hbase-server in the patch passed. | | | | 223m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1883/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1883 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cf967b2b1e13 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b17ba7b81d | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1883/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1883/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1883/2/testReport/ | | Max. process+thread count | 3083 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1883/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23562) [operator tools] Add a RegionsMerge tool that allows for merging multiple adjacent regions until a desired number of regions is reached.
[ https://issues.apache.org/jira/browse/HBASE-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138595#comment-17138595 ] Michael Stack commented on HBASE-23562: --- I like the way the differentiation is teased out here... the difference between RegionNormalizer and this new tool. Would be good to get up into release note I'd say along w/ some notes on this nice new addition. Thanks. > [operator tools] Add a RegionsMerge tool that allows for merging multiple > adjacent regions until a desired number of regions is reached. > > > Key: HBASE-23562 > URL: https://issues.apache.org/jira/browse/HBASE-23562 > Project: HBase > Issue Type: New Feature > Components: hbase-operator-tools >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Minor > Fix For: hbase-operator-tools-1.1.0 > > > There's been a few occasions where different customers had faced the need to > reduce the number of regions for specific tables, either due to mistakenly > pre-split or after a purge in table data. This jira is for adding a simple > merge tool that takes the table name and the desired number of regions, then > perform region merges until the total number of regions for the given table > reached the passed valued. -- This message was sent by Atlassian Jira (v8.3.4#803005)
