[GitHub] [hbase] Apache9 commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
Apache9 commented on a change in pull request #969: HBASE-23618 Add a tool to
dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361831049
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
+
+ private static final String KEY_TMPL = "Sequence=%s, at write timestamp=%s";
Review comment:
Just align with WALPrettyPrinter, it uses the word 'Sequence' to indicate
the sequence id...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004670#comment-17004670
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #225 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/225/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/225//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/225//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/225//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
Apache9 commented on a change in pull request #969: HBASE-23618 Add a tool to
dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361830832
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
Review comment:
Of course you can use WALPrettyPrinter on this file too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
Apache9 commented on a change in pull request #969: HBASE-23618 Add a tool to
dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361830814
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
Review comment:
WALPrettyPrinter will not decode the procedure while this one will.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23622) Reduce number of Checkstyle violations in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004655#comment-17004655
]
Hudson commented on HBASE-23622:
Results for branch branch-2.1
[build #1755 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reduce number of Checkstyle violations in hbase-common
> --
>
> Key: HBASE-23622
> URL: https://issues.apache.org/jira/browse/HBASE-23622
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004654#comment-17004654
]
Hudson commented on HBASE-23621:
Results for branch branch-2.1
[build #1755 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1755//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #970: HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest
Apache-HBase commented on issue #970: HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest URL: https://github.com/apache/hbase/pull/970#issuecomment-569472809 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 29s | master passed | | +1 :green_heart: | compile | 0m 30s | master passed | | +1 :green_heart: | checkstyle | 0m 21s | master passed | | +1 :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | | +0 :ok: | spotbugs | 1m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 2s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 26s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 16s | hbase-rest: The patch generated 0 new + 2 unchanged - 52 fixed = 2 total (was 54) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 47s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | | +1 :green_heart: | findbugs | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 5m 8s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 59m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/970 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ba99a61996a2 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-970/out/precommit/personality/provided.sh | | git revision | master / 2ca1e46d74 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/2/testReport/ | | Max. process+thread count | 1943 (vs. ulimit of 1) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #972: HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common
Apache-HBase commented on issue #972: HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common URL: https://github.com/apache/hbase/pull/972#issuecomment-569469598 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 22s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 3s | master passed | | +1 :green_heart: | compile | 0m 23s | master passed | | +1 :green_heart: | checkstyle | 0m 34s | master passed | | +1 :green_heart: | shadedjars | 4m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | master passed | | +0 :ok: | spotbugs | 0m 59s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 56s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 56s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 25s | hbase-common: The patch generated 0 new + 0 unchanged - 92 fixed = 0 total (was 92) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 8s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | | +1 :green_heart: | findbugs | 0m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 18s | hbase-common in the patch passed. | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 55m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-972/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/972 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2696f70e4ec7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-972/out/precommit/personality/provided.sh | | git revision | master / 2ca1e46d74 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-972/1/testReport/ | | Max. process+thread count | 283 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-972/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004614#comment-17004614
]
Hudson commented on HBASE-23621:
Results for branch branch-2.2
[build #738 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/738/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/738//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/738//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/738//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23622) Reduce number of Checkstyle violations in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004607#comment-17004607
]
Hudson commented on HBASE-23622:
Results for branch branch-2
[build #2399 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reduce number of Checkstyle violations in hbase-common
> --
>
> Key: HBASE-23622
> URL: https://issues.apache.org/jira/browse/HBASE-23622
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004606#comment-17004606
]
Hudson commented on HBASE-23621:
Results for branch branch-2
[build #2399 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2399//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004603#comment-17004603
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #23 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/23/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/23//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/23//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/23//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23626) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23626:
--
Status: Patch Available (was: In Progress)
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23626
> URL: https://issues.apache.org/jira/browse/HBASE-23626
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23625) Reduce number of Checkstyle violations in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23625:
--
Status: Patch Available (was: In Progress)
> Reduce number of Checkstyle violations in hbase-common
> --
>
> Key: HBASE-23625
> URL: https://issues.apache.org/jira/browse/HBASE-23625
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] HorizonNet opened a new pull request #972: HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common
HorizonNet opened a new pull request #972: HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common URL: https://github.com/apache/hbase/pull/972 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] HorizonNet opened a new pull request #971: HBASE-23625 Reduced number of Checkstyle violations in hbase-common
HorizonNet opened a new pull request #971: HBASE-23625 Reduced number of Checkstyle violations in hbase-common URL: https://github.com/apache/hbase/pull/971 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work started] (HBASE-23626) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-23626 started by Jan Hentschel.
-
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23626
> URL: https://issues.apache.org/jira/browse/HBASE-23626
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23626) Reduce number of Checkstyle violations in tests of hbase-common
Jan Hentschel created HBASE-23626:
-
Summary: Reduce number of Checkstyle violations in tests of
hbase-common
Key: HBASE-23626
URL: https://issues.apache.org/jira/browse/HBASE-23626
Project: HBase
Issue Type: Improvement
Reporter: Jan Hentschel
Assignee: Jan Hentschel
In {{hbase-common}} Checkstyle reports a lot of violations. The number of
violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HBASE-23625) Reduce number of Checkstyle violations in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-23625 started by Jan Hentschel.
-
> Reduce number of Checkstyle violations in hbase-common
> --
>
> Key: HBASE-23625
> URL: https://issues.apache.org/jira/browse/HBASE-23625
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23625) Reduce number of Checkstyle violations in hbase-common
Jan Hentschel created HBASE-23625:
-
Summary: Reduce number of Checkstyle violations in hbase-common
Key: HBASE-23625
URL: https://issues.apache.org/jira/browse/HBASE-23625
Project: HBase
Issue Type: Improvement
Reporter: Jan Hentschel
Assignee: Jan Hentschel
In {{hbase-common}} Checkstyle reports a lot of violations. The number of
violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #970: HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest
Apache-HBase commented on issue #970: HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest URL: https://github.com/apache/hbase/pull/970#issuecomment-569456116 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 22s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 33s | master passed | | +1 :green_heart: | compile | 0m 43s | master passed | | +1 :green_heart: | checkstyle | 0m 29s | master passed | | +1 :green_heart: | shadedjars | 6m 25s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | master passed | | +0 :ok: | spotbugs | 1m 12s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 32s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | -1 :x: | checkstyle | 0m 22s | hbase-rest: The patch generated 4 new + 2 unchanged - 52 fixed = 6 total (was 54) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 6m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 25m 42s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed | | +1 :green_heart: | findbugs | 1m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 6m 35s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 79m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/970 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d8b7c9ac06ea 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-970/out/precommit/personality/provided.sh | | git revision | master / 2ca1e46d74 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/1/artifact/out/diff-checkstyle-hbase-rest.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/1/testReport/ | | Max. process+thread count | 1955 (vs. ulimit of 1) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-970/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
Apache-HBase commented on issue #931: HBASE-22285 A normalizer which merges small size regions with adjacen… URL: https://github.com/apache/hbase/pull/931#issuecomment-569447313 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | -1 :x: | hbaseanti | 0m 0s | The patch appears use Hadoop classification instead of HBase. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 1m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 19s | branch-1 passed | | +1 :green_heart: | compile | 1m 0s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | compile | 1m 6s | branch-1 passed with JDK v1.7.0_242 | | +1 :green_heart: | checkstyle | 2m 1s | branch-1 passed | | +1 :green_heart: | shadedjars | 2m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 1m 3s | branch-1 passed with JDK v1.7.0_242 | | +0 :ok: | spotbugs | 2m 39s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 45s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 59s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed with JDK v1.7.0_242 | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | checkstyle | 0m 29s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 1m 28s | hbase-server: The patch generated 0 new + 0 unchanged - 6 fixed = 0 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 0s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 1m 6s | the patch passed with JDK v1.7.0_242 | | -1 :x: | findbugs | 2m 54s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 51s | hbase-common in the patch passed. | | -1 :x: | unit | 125m 39s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. | | | | 175m 43s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Dead store to entries in org.apache.hadoop.hbase.master.normalizer.MergeNormalizer.getMinimumDurationBeforeMerge() At MergeNormalizer.java:org.apache.hadoop.hbase.master.normalizer.MergeNormalizer.getMinimumDurationBeforeMerge() At MergeNormalizer.java:[line 111] | | Failed junit tests | hadoop.hbase.master.TestMasterMetrics | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-931/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/931 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 19d523ff1edd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-931/out/precommit/personality/provided.sh | | git revision | branch-1 / 27fc96e | | Default Java | 1.7.0_242 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242 | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-931/6/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-931/6/artifact/out/patch-unit-hbase-server.txt | | Test Results |
[GitHub] [hbase] saintstack commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
saintstack commented on a change in pull request #969: HBASE-23618 Add a tool
to dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361810975
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
Review comment:
The WALPrettyPrinter doesn't work for these? The below adds extra info?
Class comment to situate this tool? For debugging?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
saintstack commented on a change in pull request #969: HBASE-23618 Add a tool
to dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361810983
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
+
+ private static final String KEY_TMPL = "Sequence=%s, at write timestamp=%s";
Review comment:
What is a 'Sequence'. Looks like its WALEdit sequenceid...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
saintstack commented on a change in pull request #969: HBASE-23618 Add a tool
to dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361811400
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
+
+ private static final String KEY_TMPL = "Sequence=%s, at write timestamp=%s";
+
+ private static final DateTimeFormatter FORMATTER =
+DateTimeFormatter.ISO_OFFSET_DATE_TIME.withZone(ZoneId.systemDefault());
+
+ private String file;
+
+ private PrintStream out;
+
+ public WALProcedurePrettyPrinter() {
+this(System.out);
+ }
+
+ public WALProcedurePrettyPrinter(PrintStream out) {
+this.out = out;
+ }
+
+ @Override
+ protected void addOptions() {
+ }
+
+ @Override
+ protected void processOptions(CommandLine cmd) {
+if (cmd.getArgList().size() != 1) {
+ throw new IllegalArgumentException("Please specify the file to dump");
+}
+file = cmd.getArgList().get(0);
+ }
+
+ @Override
+ protected int doWork() throws Exception {
+Path path = new Path(file);
+FileSystem fs = path.getFileSystem(conf);
+try (WAL.Reader reader = WALFactory.createReader(fs, path, conf)) {
+ for (;;) {
+WAL.Entry entry = reader.next();
+if (entry == null) {
+ return 0;
+}
+WALKey key = entry.getKey();
+WALEdit edit = entry.getEdit();
+long sequenceId = key.getSequenceId();
+long writeTime = key.getWriteTime();
+out.println(
+ String.format(KEY_TMPL, sequenceId,
FORMATTER.format(Instant.ofEpochMilli(writeTime;
+for (Cell cell : edit.getCells()) {
+ Map op = WALPrettyPrinter.toStringMap(cell);
+ if (!Bytes.equals(FAMILY, 0, FAMILY.length, cell.getFamilyArray(),
cell.getFamilyOffset(),
+cell.getFamilyLength())) {
+// We could have cells other than procedure edits, for example, a
flush marker
+WALPrettyPrinter.printCell(out, op, false);
+continue;
+ }
+ long procId = Bytes.toLong(cell.getRowArray(), cell.getRowOffset(),
cell.getRowLength());
+ out.println("pid=" + procId + ", type=" + op.get("type") + ",
column=" +
+op.get("family") + ":" + op.get("qualifier"));
+ if (cell.getType() == Cell.Type.Put) {
+if (cell.getValueLength() > 0) {
+ // should be a normal put
+ Procedure proc =
+
ProcedureUtil.convertToProcedure(ProcedureProtos.Procedure.parser()
+ .parseFrom(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
+ out.println("\t" + proc.toStringDetails());
+} else {
+ //
[GitHub] [hbase] saintstack commented on a change in pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
saintstack commented on a change in pull request #969: HBASE-23618 Add a tool
to dump procedure info in the WAL file
URL: https://github.com/apache/hbase/pull/969#discussion_r361811221
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/WALProcedurePrettyPrinter.java
##
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.procedure2.store.region;
+
+import static
org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.FAMILY;
+
+import java.io.PrintStream;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.util.AbstractHBaseTool;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.hadoop.hbase.wal.WALFactory;
+import org.apache.hadoop.hbase.wal.WALKey;
+import org.apache.hadoop.hbase.wal.WALPrettyPrinter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
+@InterfaceStability.Evolving
+public class WALProcedurePrettyPrinter extends AbstractHBaseTool {
+
+ private static final String KEY_TMPL = "Sequence=%s, at write timestamp=%s";
Review comment:
Usually we write sequence as seqid or sequenceid elsewhere logging WALKey
sequenceids.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #967: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
saintstack commented on a change in pull request #967: HBASE-23621 Reduced the
number of Checkstyle violations in tests of hbase-common
URL: https://github.com/apache/hbase/pull/967#discussion_r361810842
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/RedundantKVGenerator.java
##
@@ -469,95 +463,9 @@ public static ByteBuffer
convertKvToByteBuffer(List keyValues,
return result;
}
- / get/set ***/
- public RedundantKVGenerator setCommonPrefix(byte[] prefix){
-this.commonPrefix = prefix;
-return this;
- }
-
- public RedundantKVGenerator setRandomizer(Random randomizer) {
-this.randomizer = randomizer;
-return this;
- }
-
- public RedundantKVGenerator setNumberOfRowPrefixes(int numberOfRowPrefixes) {
-this.numberOfRowPrefixes = numberOfRowPrefixes;
-return this;
- }
-
- public RedundantKVGenerator setAveragePrefixLength(int averagePrefixLength) {
-this.averagePrefixLength = averagePrefixLength;
-return this;
- }
-
- public RedundantKVGenerator setPrefixLengthVariance(int
prefixLengthVariance) {
-this.prefixLengthVariance = prefixLengthVariance;
-return this;
- }
-
- public RedundantKVGenerator setAverageSuffixLength(int averageSuffixLength) {
-this.averageSuffixLength = averageSuffixLength;
-return this;
- }
-
- public RedundantKVGenerator setSuffixLengthVariance(int
suffixLengthVariance) {
-this.suffixLengthVariance = suffixLengthVariance;
-return this;
- }
-
- public RedundantKVGenerator setNumberOfRows(int numberOfRows) {
-this.numberOfRows = numberOfRows;
-return this;
- }
-
- public RedundantKVGenerator setChanceForSameQualifier(float
chanceForSameQualifier) {
-this.chanceForSameQualifier = chanceForSameQualifier;
-return this;
- }
-
- public RedundantKVGenerator setChanceForSimilarQualifier(float
chanceForSimiliarQualifier) {
-this.chanceForSimilarQualifier = chanceForSimiliarQualifier;
-return this;
- }
-
- public RedundantKVGenerator setAverageQualifierLength(int
averageQualifierLength) {
-this.averageQualifierLength = averageQualifierLength;
-return this;
- }
-
- public RedundantKVGenerator setQualifierLengthVariance(int
qualifierLengthVariance) {
-this.qualifierLengthVariance = qualifierLengthVariance;
-return this;
- }
-
- public RedundantKVGenerator setColumnFamilyLength(int columnFamilyLength) {
-this.columnFamilyLength = columnFamilyLength;
-return this;
- }
-
public RedundantKVGenerator setFamily(byte[] family) {
this.family = family;
this.columnFamilyLength = family.length;
return this;
}
-
- public RedundantKVGenerator setValueLength(int valueLength) {
-this.valueLength = valueLength;
-return this;
- }
-
- public RedundantKVGenerator setChanceForZeroValue(float chanceForZeroValue) {
-this.chanceForZeroValue = chanceForZeroValue;
-return this;
- }
-
- public RedundantKVGenerator setBaseTimestampDivide(int baseTimestampDivide) {
-this.baseTimestampDivide = baseTimestampDivide;
-return this;
- }
-
- public RedundantKVGenerator setTimestampDiffSize(int timestampDiffSize) {
-this.timestampDiffSize = timestampDiffSize;
-return this;
- }
Review comment:
Ok. Good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23238) Additional test and checks for null references on ScannerCallableWithReplicas
[
https://issues.apache.org/jira/browse/HBASE-23238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004562#comment-17004562
]
Michael Stack commented on HBASE-23238:
---
[~vjasani] Merged the PR to branch-1. We should be good here now.
> Additional test and checks for null references on ScannerCallableWithReplicas
> -
>
> Key: HBASE-23238
> URL: https://issues.apache.org/jira/browse/HBASE-23238
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.12
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0, 2.1.8, 2.2.3
>
> Attachments: HBASE-23238.branch-2.patch
>
>
> One of our customers running a 1.2 based version is facing NPE when scanning
> data from a MR job. It happens when the map task is finalising:
> {noformat}
> ...
> 2019-09-10 14:17:22,238 INFO [main] org.apache.hadoop.mapred.MapTask:
> Ignoring exception during close for
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@3a5b7d7e
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.setClose(ScannerCallableWithReplicas.java:99)
> at
> org.apache.hadoop.hbase.client.ClientScanner.close(ClientScanner.java:730)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.close(TableRecordReaderImpl.java:178)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReader.close(TableRecordReader.java:89)
> at
> org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatBase$1.close(MultiTableInputFormatBase.java:112)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:529)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2039)
> ...
> 2019-09-10 14:18:24,601 FATAL [IPC Server handler 5 on 35745]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1566832111959_6047_m_00_3 - exited :
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.setClose(ScannerCallableWithReplicas.java:99)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264)
> at
> org.apache.hadoop.hbase.client.ClientScanner.possiblyNextScanner(ClientScanner.java:248)
> at
> org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:542)
> at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:371)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.nextKeyValue(TableRecordReaderImpl.java:222)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReader.nextKeyValue(TableRecordReader.java:147)
> at
> org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatBase$1.nextKeyValue(MultiTableInputFormatBase.java:139)
> ...
> {noformat}
> After some investigation, we found out that 1.2 based deployments will
> consistently face this problem under the following conditions:
> 1) The sum of all the given row KVs size targeted to be returned in the scan
> is larger than *max result size*;
> 2) At same time, the scan filter has exhausted, so that all remaining KVs
> should be filtered and not returned;
> We could simulate this with the UT being proposed in this PR. When checking
> newer branches, though, I could verify this specific problem is not present
> on newer branches, I believe it was indirectly sorted by changes from
> HBASE-17489.
> Nevertheless, I think it would still be useful to have this extra test and
> checks added as a safeguard measure.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil…
saintstack commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil… URL: https://github.com/apache/hbase/pull/965#issuecomment-569445869 @virajjasani All good. I didn't see these last bits of the conversation. Thanks @virajjasani This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23623) Reduce number of Checkstyle violations in hbase-rest
[
https://issues.apache.org/jira/browse/HBASE-23623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23623:
--
Status: Patch Available (was: In Progress)
> Reduce number of Checkstyle violations in hbase-rest
>
>
> Key: HBASE-23623
> URL: https://issues.apache.org/jira/browse/HBASE-23623
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-rest}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-20673) Reduce the number of Cell implementations; the profusion is distracting to users and JIT
[ https://issues.apache.org/jira/browse/HBASE-20673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004560#comment-17004560 ] Michael Stack commented on HBASE-20673: --- Thank you for taking a look at this one zhuobin zheng. @zhuobin zheng, please make an account in JIRA and then you should be able to attach files. I'll add you as a contributor too after you do this. It is a while since I looked at this issue. Yes, there were large steps in CPU usage attributed to a single instruction that seemed to attributable to complicated dispatch. Would be interested in result of your experiments (yes, repro in test setup is tough). Is the machine that is using all CPUs carrying hbase:meta? > Reduce the number of Cell implementations; the profusion is distracting to > users and JIT > > > Key: HBASE-20673 > URL: https://issues.apache.org/jira/browse/HBASE-20673 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Attachments: 0001-current.patch, 0001-current.patch, hits.20673.png > > > We have a wild blossom of Cell implementations in hbase. Purge the bulk of > them. Make it so we do one type well. JIT gets confused if it has an abstract > or an interface and then the instantiated classes are myriad (megamorphic). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] HorizonNet opened a new pull request #970: HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest
HorizonNet opened a new pull request #970: HBASE-23623 Reduced the number of Checkstyle violations in hbase-rest URL: https://github.com/apache/hbase/pull/970 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23622) Reduce number of Checkstyle violations in hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23622:
--
Fix Version/s: 2.1.9
2.2.3
2.3.0
3.0.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, branch-2, branch-2.2, branch-2.1.
> Reduce number of Checkstyle violations in hbase-common
> --
>
> Key: HBASE-23622
> URL: https://issues.apache.org/jira/browse/HBASE-23622
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil…
virajjasani commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil… URL: https://github.com/apache/hbase/pull/965#issuecomment-569444733 My bad, I should have merged the PR directly to avoid this confusion This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code.
saintstack commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code. URL: https://github.com/apache/hbase/pull/957#issuecomment-569444673 I think the rename to ConnectionRegistry looks good. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code.
saintstack commented on issue #957: HBASE-23604: Clarify AsyncRegistry usage in the code. URL: https://github.com/apache/hbase/pull/957#issuecomment-569444590 I rebased for you. Why the conflict above do you think? And do you want to fix the checkstyle? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil…
virajjasani commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil… URL: https://github.com/apache/hbase/pull/965#issuecomment-56936 @saintstack This is committed to branch-1: https://github.com/apache/hbase/commit/27fc96e076ec320c4d6287573be48ca0d3aff96e May be you tried on top of this and hence didn't apply? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil…
saintstack commented on issue #965: HBASE-23238: Remove 'static'ness of cell counter in LimitKVsReturnFil… URL: https://github.com/apache/hbase/pull/965#issuecomment-569444136 Does #965 apply for your @bharathv It fails for me against branch-1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] HorizonNet merged pull request #968: HBASE-23622 Reduced the number of Checkstyle violations in hbase-common
HorizonNet merged pull request #968: HBASE-23622 Reduced the number of Checkstyle violations in hbase-common URL: https://github.com/apache/hbase/pull/968 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
Apache-HBase commented on issue #969: HBASE-23618 Add a tool to dump procedure info in the WAL file URL: https://github.com/apache/hbase/pull/969#issuecomment-569440956 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 1s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 30s | master passed | | +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | | +0 :ok: | spotbugs | 4m 53s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 51s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 30s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | checkstyle | 1m 29s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 35s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 4m 57s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 165m 57s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 230m 8s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | | hadoop.hbase.master.assignment.TestRegionMoveAndAbandon | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-969/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/969 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 3eda1b619207 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-969/out/precommit/personality/provided.sh | | git revision | master / 0ba84d8e95 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-969/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-969/1/testReport/ | | Max. process+thread count | 4857 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-969/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel resolved HBASE-23621.
---
Fix Version/s: 2.1.9
2.2.3
2.3.0
3.0.0
Resolution: Fixed
Pushed to master, branch-2, branch-2.2, branch-2.1
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-23621:
-
Fix Version/s: (was: 3.0.0)
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004534#comment-17004534
]
Viraj Jasani commented on HBASE-23621:
--
Sure, reopened for backports.
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani reopened HBASE-23621:
--
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004533#comment-17004533
]
Jan Hentschel commented on HBASE-23621:
---
[~vjasani] I'm going to backport it as far as possible to make future backports
easier.
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-23621:
-
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004531#comment-17004531
]
Viraj Jasani commented on HBASE-23621:
--
[~janh] we don't need this on branch-2 and branch-1, correct?
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-23621:
-
Fix Version/s: 3.0.0
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23621) Reduce number of Checkstyle violations in tests of hbase-common
[
https://issues.apache.org/jira/browse/HBASE-23621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-23621:
-
Affects Version/s: 3.0.0
> Reduce number of Checkstyle violations in tests of hbase-common
> ---
>
> Key: HBASE-23621
> URL: https://issues.apache.org/jira/browse/HBASE-23621
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> In {{hbase-common}} Checkstyle reports a lot of violations. The number of
> violations in the tests should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani closed pull request #967: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
virajjasani closed pull request #967: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common URL: https://github.com/apache/hbase/pull/967 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
Apache-HBase commented on issue #931: HBASE-22285 A normalizer which merges small size regions with adjacen… URL: https://github.com/apache/hbase/pull/931#issuecomment-569433457 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 42s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | -1 :x: | hbaseanti | 0m 0s | The patch appears use Hadoop classification instead of HBase. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 1m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 32s | branch-1 passed | | +1 :green_heart: | compile | 1m 4s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | compile | 1m 10s | branch-1 passed with JDK v1.7.0_242 | | +1 :green_heart: | checkstyle | 2m 11s | branch-1 passed | | +1 :green_heart: | shadedjars | 2m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | branch-1 passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 1m 10s | branch-1 passed with JDK v1.7.0_242 | | +0 :ok: | spotbugs | 2m 49s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 57s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 0s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed with JDK v1.7.0_242 | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 1m 40s | hbase-server: The patch generated 0 new + 4 unchanged - 2 fixed = 4 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 20s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javadoc | 1m 7s | the patch passed with JDK v1.7.0_242 | | -1 :x: | findbugs | 3m 13s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 49s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 126m 22s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. | | | | 179m 56s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Dead store to entries in org.apache.hadoop.hbase.master.normalizer.MergeNormalizer.getMinimumDurationBeforeMerge() At MergeNormalizer.java:org.apache.hadoop.hbase.master.normalizer.MergeNormalizer.getMinimumDurationBeforeMerge() At MergeNormalizer.java:[line 131] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-931/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/931 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 82b2a708c4c9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-931/out/precommit/personality/provided.sh | | git revision | branch-1 / 27fc96e | | Default Java | 1.7.0_242 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242 | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-931/5/artifact/out/new-findbugs-hbase-server.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-931/5/testReport/ | | Max. process+thread count | 4214 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | |
[GitHub] [hbase] virajjasani commented on a change in pull request #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
virajjasani commented on a change in pull request #931: HBASE-22285 A
normalizer which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/931#discussion_r361803287
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -22,61 +22,36 @@
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
-
-import org.apache.commons.logging.Log;
-import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.hbase.HBaseIOException;
import org.apache.hadoop.hbase.HRegionInfo;
-import org.apache.hadoop.hbase.RegionLoad;
-import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.TableName;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
-import org.apache.hadoop.hbase.client.Admin.MasterSwitchType;
-import org.apache.hadoop.hbase.master.MasterRpcServices;
-import org.apache.hadoop.hbase.master.MasterServices;
-import org.apache.hadoop.hbase.protobuf.RequestConverter;
-
-import com.google.protobuf.ServiceException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
- * Simple implementation of region normalizer.
- *
- * Logic in use:
- *
- *
- * get all regions of a given table
- * get avg size S of each region (by total size of store files reported
in RegionLoad)
- * If biggest region is bigger than S * 2, it is kindly requested to
split,
- *and normalization stops
- * Otherwise, two smallest region R1 and its smallest neighbor R2 are
kindly requested
- *to merge, if R1 + R1 S, and normalization stops
- * Otherwise, no action is performed
+ * Simple implementation of region normalizer. Logic in use:
+ *
+ * get all regions of a given table
+ * get avg size S of each region (by total size of store files reported in
RegionLoad)
+ * If biggest region is bigger than S * 2, it is kindly requested to
split, and normalization
+ * stops
+ * Otherwise, two smallest region R1 and its smallest neighbor R2 are
kindly requested to merge,
+ * if R1 + R1 S, and normalization stops
+ * Otherwise, no action is performed
*
*
- * Region sizes are coarse and approximate on the order of megabytes.
Additionally,
- * "empty" regions (less than 1MB, with the previous note) are not merged
away. This
- * is by design to prevent normalization from undoing the pre-splitting of a
table.
+ * Region sizes are coarse and approximate on the order of megabytes.
Additionally, "empty" regions
+ * (less than 1MB, with the previous note) are not merged away. This is by
design to prevent
+ * normalization from undoing the pre-splitting of a table.
*/
@InterfaceAudience.Private
-public class SimpleRegionNormalizer implements RegionNormalizer {
-
- private static final Log LOG =
LogFactory.getLog(SimpleRegionNormalizer.class);
+public class SimpleRegionNormalizer extends BaseNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(SimpleRegionNormalizer.class);
private static final int MIN_REGION_COUNT = 3;
Review comment:
Right, also this is mostly same constant. if we want to use configs for
these values(which we don't want I believe), then we could use it separately
but here, may be we can refer to same constant.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mnpoonia commented on a change in pull request #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on a change in pull request #931: HBASE-22285 A normalizer
which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/931#discussion_r361802820
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -22,61 +22,36 @@
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
-
-import org.apache.commons.logging.Log;
-import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.hbase.HBaseIOException;
import org.apache.hadoop.hbase.HRegionInfo;
-import org.apache.hadoop.hbase.RegionLoad;
-import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.TableName;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
-import org.apache.hadoop.hbase.client.Admin.MasterSwitchType;
-import org.apache.hadoop.hbase.master.MasterRpcServices;
-import org.apache.hadoop.hbase.master.MasterServices;
-import org.apache.hadoop.hbase.protobuf.RequestConverter;
-
-import com.google.protobuf.ServiceException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
- * Simple implementation of region normalizer.
- *
- * Logic in use:
- *
- *
- * get all regions of a given table
- * get avg size S of each region (by total size of store files reported
in RegionLoad)
- * If biggest region is bigger than S * 2, it is kindly requested to
split,
- *and normalization stops
- * Otherwise, two smallest region R1 and its smallest neighbor R2 are
kindly requested
- *to merge, if R1 + R1 S, and normalization stops
- * Otherwise, no action is performed
+ * Simple implementation of region normalizer. Logic in use:
+ *
+ * get all regions of a given table
+ * get avg size S of each region (by total size of store files reported in
RegionLoad)
+ * If biggest region is bigger than S * 2, it is kindly requested to
split, and normalization
+ * stops
+ * Otherwise, two smallest region R1 and its smallest neighbor R2 are
kindly requested to merge,
+ * if R1 + R1 S, and normalization stops
+ * Otherwise, no action is performed
*
*
- * Region sizes are coarse and approximate on the order of megabytes.
Additionally,
- * "empty" regions (less than 1MB, with the previous note) are not merged
away. This
- * is by design to prevent normalization from undoing the pre-splitting of a
table.
+ * Region sizes are coarse and approximate on the order of megabytes.
Additionally, "empty" regions
+ * (less than 1MB, with the previous note) are not merged away. This is by
design to prevent
+ * normalization from undoing the pre-splitting of a table.
*/
@InterfaceAudience.Private
-public class SimpleRegionNormalizer implements RegionNormalizer {
-
- private static final Log LOG =
LogFactory.getLog(SimpleRegionNormalizer.class);
+public class SimpleRegionNormalizer extends BaseNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(SimpleRegionNormalizer.class);
private static final int MIN_REGION_COUNT = 3;
Review comment:
I am not sure about this yet. Let me think a bit. The point is that both
normalizer can have different requirement for the MIN_REGION_COUNT. But then
we can't have both normalizer enabled together.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] mnpoonia commented on a change in pull request #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
mnpoonia commented on a change in pull request #931: HBASE-22285 A normalizer
which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/931#discussion_r361802729
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/BaseNormalizer.java
##
@@ -0,0 +1,161 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import com.google.protobuf.ServiceException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.RegionLoad;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.master.MasterRpcServices;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.hadoop.hbase.protobuf.RequestConverter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public abstract class BaseNormalizer implements RegionNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(BaseNormalizer.class);
+ protected MasterServices masterServices;
+ protected MasterRpcServices masterRpcServices;
+
+ /**
+ * Set the master service.
+ * @param masterServices inject instance of MasterServices
+ */
+ @Override
+ public void setMasterServices(MasterServices masterServices) {
+this.masterServices = masterServices;
+ }
+
+ @Override
+ public void setMasterRpcServices(MasterRpcServices masterRpcServices) {
+this.masterRpcServices = masterRpcServices;
+ }
+
+ protected long getRegionSize(HRegionInfo hri) {
+ServerName sn =
+
masterServices.getAssignmentManager().getRegionStates().getRegionServerOfRegion(hri);
+RegionLoad regionLoad =
+
masterServices.getServerManager().getLoad(sn).getRegionsLoad().get(hri.getRegionName());
+if (regionLoad == null) {
+ LOG.debug("Region {} was not found in RegionsLoad",
hri.getRegionNameAsString());
+ return -1;
+}
+return regionLoad.getStorefileSizeMB();
+ }
+
+ protected boolean isMergeEnabled() {
+boolean mergeEnabled = true;
+try {
+ mergeEnabled = masterRpcServices
+ .isSplitOrMergeEnabled(null,
+
RequestConverter.buildIsSplitOrMergeEnabledRequest(Admin.MasterSwitchType.MERGE))
+ .getEnabled();
+} catch (ServiceException se) {
+ LOG.debug("Unable to determine whether merge is enabled", se);
+}
+return mergeEnabled;
+ }
+
+ protected boolean isSplitEnabled() {
+boolean splitEnabled = true;
+try {
+ splitEnabled = masterRpcServices
+ .isSplitOrMergeEnabled(null,
+
RequestConverter.buildIsSplitOrMergeEnabledRequest(Admin.MasterSwitchType.SPLIT))
+ .getEnabled();
+} catch (ServiceException se) {
+ LOG.debug("Unable to determine whether merge is enabled", se);
Review comment:
WARN it is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
virajjasani commented on a change in pull request #931: HBASE-22285 A
normalizer which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/931#discussion_r361799434
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -22,61 +22,36 @@
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
-
-import org.apache.commons.logging.Log;
-import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.hbase.HBaseIOException;
import org.apache.hadoop.hbase.HRegionInfo;
-import org.apache.hadoop.hbase.RegionLoad;
-import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.TableName;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
-import org.apache.hadoop.hbase.client.Admin.MasterSwitchType;
-import org.apache.hadoop.hbase.master.MasterRpcServices;
-import org.apache.hadoop.hbase.master.MasterServices;
-import org.apache.hadoop.hbase.protobuf.RequestConverter;
-
-import com.google.protobuf.ServiceException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
- * Simple implementation of region normalizer.
- *
- * Logic in use:
- *
- *
- * get all regions of a given table
- * get avg size S of each region (by total size of store files reported
in RegionLoad)
- * If biggest region is bigger than S * 2, it is kindly requested to
split,
- *and normalization stops
- * Otherwise, two smallest region R1 and its smallest neighbor R2 are
kindly requested
- *to merge, if R1 + R1 S, and normalization stops
- * Otherwise, no action is performed
+ * Simple implementation of region normalizer. Logic in use:
+ *
+ * get all regions of a given table
+ * get avg size S of each region (by total size of store files reported in
RegionLoad)
+ * If biggest region is bigger than S * 2, it is kindly requested to
split, and normalization
+ * stops
+ * Otherwise, two smallest region R1 and its smallest neighbor R2 are
kindly requested to merge,
+ * if R1 + R1 S, and normalization stops
+ * Otherwise, no action is performed
*
*
- * Region sizes are coarse and approximate on the order of megabytes.
Additionally,
- * "empty" regions (less than 1MB, with the previous note) are not merged
away. This
- * is by design to prevent normalization from undoing the pre-splitting of a
table.
+ * Region sizes are coarse and approximate on the order of megabytes.
Additionally, "empty" regions
+ * (less than 1MB, with the previous note) are not merged away. This is by
design to prevent
+ * normalization from undoing the pre-splitting of a table.
*/
@InterfaceAudience.Private
-public class SimpleRegionNormalizer implements RegionNormalizer {
-
- private static final Log LOG =
LogFactory.getLog(SimpleRegionNormalizer.class);
+public class SimpleRegionNormalizer extends BaseNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(SimpleRegionNormalizer.class);
private static final int MIN_REGION_COUNT = 3;
Review comment:
MergeNormalizer and SimpleRegionNormalizer both can use same constant
`MIN_REGION_COUNT`, may be good to move it to HConstant
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work started] (HBASE-23624) Add a tool to dump the procedure info in HFile
[ https://issues.apache.org/jira/browse/HBASE-23624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23624 started by Duo Zhang. - > Add a tool to dump the procedure info in HFile > -- > > Key: HBASE-23624 > URL: https://issues.apache.org/jira/browse/HBASE-23624 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23624) Add a tool to dump the procedure info in HFile
[ https://issues.apache.org/jira/browse/HBASE-23624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-23624: - Assignee: Duo Zhang > Add a tool to dump the procedure info in HFile > -- > > Key: HBASE-23624 > URL: https://issues.apache.org/jira/browse/HBASE-23624 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
virajjasani commented on a change in pull request #931: HBASE-22285 A
normalizer which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/931#discussion_r361798913
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/BaseNormalizer.java
##
@@ -0,0 +1,161 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import com.google.protobuf.ServiceException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.RegionLoad;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.master.MasterRpcServices;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.hadoop.hbase.protobuf.RequestConverter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public abstract class BaseNormalizer implements RegionNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(BaseNormalizer.class);
+ protected MasterServices masterServices;
+ protected MasterRpcServices masterRpcServices;
+
+ /**
+ * Set the master service.
+ * @param masterServices inject instance of MasterServices
+ */
+ @Override
+ public void setMasterServices(MasterServices masterServices) {
+this.masterServices = masterServices;
+ }
+
+ @Override
+ public void setMasterRpcServices(MasterRpcServices masterRpcServices) {
+this.masterRpcServices = masterRpcServices;
+ }
+
+ protected long getRegionSize(HRegionInfo hri) {
+ServerName sn =
+
masterServices.getAssignmentManager().getRegionStates().getRegionServerOfRegion(hri);
+RegionLoad regionLoad =
+
masterServices.getServerManager().getLoad(sn).getRegionsLoad().get(hri.getRegionName());
+if (regionLoad == null) {
+ LOG.debug("Region {} was not found in RegionsLoad",
hri.getRegionNameAsString());
+ return -1;
+}
+return regionLoad.getStorefileSizeMB();
+ }
+
+ protected boolean isMergeEnabled() {
+boolean mergeEnabled = true;
+try {
+ mergeEnabled = masterRpcServices
+ .isSplitOrMergeEnabled(null,
+
RequestConverter.buildIsSplitOrMergeEnabledRequest(Admin.MasterSwitchType.MERGE))
+ .getEnabled();
+} catch (ServiceException se) {
+ LOG.debug("Unable to determine whether merge is enabled", se);
+}
+return mergeEnabled;
+ }
+
+ protected boolean isSplitEnabled() {
+boolean splitEnabled = true;
+try {
+ splitEnabled = masterRpcServices
+ .isSplitOrMergeEnabled(null,
+
RequestConverter.buildIsSplitOrMergeEnabledRequest(Admin.MasterSwitchType.SPLIT))
+ .getEnabled();
+} catch (ServiceException se) {
+ LOG.debug("Unable to determine whether merge is enabled", se);
Review comment:
same here
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #931: HBASE-22285 A normalizer which merges small size regions with adjacen…
virajjasani commented on a change in pull request #931: HBASE-22285 A
normalizer which merges small size regions with adjacen…
URL: https://github.com/apache/hbase/pull/931#discussion_r361798908
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/BaseNormalizer.java
##
@@ -0,0 +1,161 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import com.google.protobuf.ServiceException;
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.RegionLoad;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.master.MasterRpcServices;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.hadoop.hbase.protobuf.RequestConverter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public abstract class BaseNormalizer implements RegionNormalizer {
+ private static final Logger LOG =
LoggerFactory.getLogger(BaseNormalizer.class);
+ protected MasterServices masterServices;
+ protected MasterRpcServices masterRpcServices;
+
+ /**
+ * Set the master service.
+ * @param masterServices inject instance of MasterServices
+ */
+ @Override
+ public void setMasterServices(MasterServices masterServices) {
+this.masterServices = masterServices;
+ }
+
+ @Override
+ public void setMasterRpcServices(MasterRpcServices masterRpcServices) {
+this.masterRpcServices = masterRpcServices;
+ }
+
+ protected long getRegionSize(HRegionInfo hri) {
+ServerName sn =
+
masterServices.getAssignmentManager().getRegionStates().getRegionServerOfRegion(hri);
+RegionLoad regionLoad =
+
masterServices.getServerManager().getLoad(sn).getRegionsLoad().get(hri.getRegionName());
+if (regionLoad == null) {
+ LOG.debug("Region {} was not found in RegionsLoad",
hri.getRegionNameAsString());
+ return -1;
+}
+return regionLoad.getStorefileSizeMB();
+ }
+
+ protected boolean isMergeEnabled() {
+boolean mergeEnabled = true;
+try {
+ mergeEnabled = masterRpcServices
+ .isSplitOrMergeEnabled(null,
+
RequestConverter.buildIsSplitOrMergeEnabledRequest(Admin.MasterSwitchType.MERGE))
+ .getEnabled();
+} catch (ServiceException se) {
+ LOG.debug("Unable to determine whether merge is enabled", se);
Review comment:
want to convert this to error or warn log?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-22285) A normalizer which merges very small size regions with adjacent regions.(MergeToNormalize)
[ https://issues.apache.org/jira/browse/HBASE-22285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Poonia updated HBASE-22285: Status: Patch Available (was: Open) > A normalizer which merges very small size regions with adjacent > regions.(MergeToNormalize) > -- > > Key: HBASE-22285 > URL: https://issues.apache.org/jira/browse/HBASE-22285 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > > There are scenarios where we have seen around 5% of total regions with a size > of 0 bytes and another 5-6 % regions with size in a few bytes. These kinds of > regions increase with time considering we have TTL over the rows. > After exploring the option of RegionNormalizer and doing some quick runs we > found that that is not suitable considering it also splits the regions and > merges to normalize. What we really want is to split as per Split policy and > merge very small regions with adjacent regions to make sure we reduce 0-byte > regions. > We can plugin this normalizer using the property > "hbase.master.normalizer.class" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23617) Add a stress test tool for region based procedure store
[
https://issues.apache.org/jira/browse/HBASE-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004490#comment-17004490
]
Duo Zhang commented on HBASE-23617:
---
{quote}
For the SYNC_WAIT_MSEC, that a follow-on?
{quote}
No. It is in the WALProcedureStore, which will be removed in the future.
> Add a stress test tool for region based procedure store
> ---
>
> Key: HBASE-23617
> URL: https://issues.apache.org/jira/browse/HBASE-23617
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23618) Add a tool to dump procedure info in the WAL file
[ https://issues.apache.org/jira/browse/HBASE-23618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23618: -- Fix Version/s: 2.3.0 3.0.0 > Add a tool to dump procedure info in the WAL file > - > > Key: HBASE-23618 > URL: https://issues.apache.org/jira/browse/HBASE-23618 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 opened a new pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file
Apache9 opened a new pull request #969: HBASE-23618 Add a tool to dump procedure info in the WAL file URL: https://github.com/apache/hbase/pull/969 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23618) Add a tool to dump procedure info in the WAL file
[ https://issues.apache.org/jira/browse/HBASE-23618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23618: -- Summary: Add a tool to dump procedure info in the WAL file (was: Add a tool to print procedure info offline) > Add a tool to dump procedure info in the WAL file > - > > Key: HBASE-23618 > URL: https://issues.apache.org/jira/browse/HBASE-23618 > Project: HBase > Issue Type: Improvement > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23624) Add a tool to dump the procedure info in HFile
Duo Zhang created HBASE-23624: - Summary: Add a tool to dump the procedure info in HFile Key: HBASE-23624 URL: https://issues.apache.org/jira/browse/HBASE-23624 Project: HBase Issue Type: Improvement Components: proc-v2 Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23619) Use built-in formatting for logging in hbase-zookeeper
[
https://issues.apache.org/jira/browse/HBASE-23619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004487#comment-17004487
]
Hudson commented on HBASE-23619:
Results for branch master
[build #1581 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1581/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1581//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1581//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1581//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use built-in formatting for logging in hbase-zookeeper
> --
>
> Key: HBASE-23619
> URL: https://issues.apache.org/jira/browse/HBASE-23619
> Project: HBase
> Issue Type: Improvement
> Components: Zookeeper
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> String concatenation is used a lot for creating the log messages in the
> {{hbase-zookeeper}} module. The built-in formatting of the logger should be
> used instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23617) Add a stress test tool for region based procedure store
[
https://issues.apache.org/jira/browse/HBASE-23617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004488#comment-17004488
]
Hudson commented on HBASE-23617:
Results for branch master
[build #1581 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1581/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1581//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1581//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1581//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add a stress test tool for region based procedure store
> ---
>
> Key: HBASE-23617
> URL: https://issues.apache.org/jira/browse/HBASE-23617
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23238) Additional test and checks for null references on ScannerCallableWithReplicas
[
https://issues.apache.org/jira/browse/HBASE-23238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004460#comment-17004460
]
Hudson commented on HBASE-23238:
Results for branch branch-1
[build #1182 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1182/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1182//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1182//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1182//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Additional test and checks for null references on ScannerCallableWithReplicas
> -
>
> Key: HBASE-23238
> URL: https://issues.apache.org/jira/browse/HBASE-23238
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.12
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0, 2.1.8, 2.2.3
>
> Attachments: HBASE-23238.branch-2.patch
>
>
> One of our customers running a 1.2 based version is facing NPE when scanning
> data from a MR job. It happens when the map task is finalising:
> {noformat}
> ...
> 2019-09-10 14:17:22,238 INFO [main] org.apache.hadoop.mapred.MapTask:
> Ignoring exception during close for
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@3a5b7d7e
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.setClose(ScannerCallableWithReplicas.java:99)
> at
> org.apache.hadoop.hbase.client.ClientScanner.close(ClientScanner.java:730)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.close(TableRecordReaderImpl.java:178)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReader.close(TableRecordReader.java:89)
> at
> org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatBase$1.close(MultiTableInputFormatBase.java:112)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:529)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2039)
> ...
> 2019-09-10 14:18:24,601 FATAL [IPC Server handler 5 on 35745]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1566832111959_6047_m_00_3 - exited :
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.setClose(ScannerCallableWithReplicas.java:99)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264)
> at
> org.apache.hadoop.hbase.client.ClientScanner.possiblyNextScanner(ClientScanner.java:248)
> at
> org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:542)
> at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:371)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.nextKeyValue(TableRecordReaderImpl.java:222)
> at
> org.apache.hadoop.hbase.mapreduce.TableRecordReader.nextKeyValue(TableRecordReader.java:147)
> at
> org.apache.hadoop.hbase.mapreduce.MultiTableInputFormatBase$1.nextKeyValue(MultiTableInputFormatBase.java:139)
> ...
> {noformat}
> After some investigation, we found out that 1.2 based deployments will
> consistently face this problem under the following conditions:
> 1) The sum of all the given row KVs size targeted to be returned in the scan
> is larger than *max result size*;
> 2) At same time, the scan filter has exhausted, so that all remaining KVs
> should be filtered and not returned;
> We could simulate this with the UT being proposed in this PR. When checking
> newer branches, though, I could verify this specific problem is not present
> on newer branches, I believe it was indirectly sorted by changes from
> HBASE-17489.
> Nevertheless, I think it would still be useful to have this extra test and
> checks added as a safeguard measure.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004421#comment-17004421
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #224 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/224/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/224//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/224//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/224//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
