[jira] [Updated] (HBASE-23723) Ensure MOB compaction works in optimized mode after snapshot clone
[ https://issues.apache.org/jira/browse/HBASE-23723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-23723: Summary: Ensure MOB compaction works in optimized mode after snapshot clone (was: Add tests for MOB compaction on a table created from snapshot) > Ensure MOB compaction works in optimized mode after snapshot clone > -- > > Key: HBASE-23723 > URL: https://issues.apache.org/jira/browse/HBASE-23723 > Project: HBase > Issue Type: Sub-task > Components: Compaction, mob >Reporter: Vladimir Rodionov >Assignee: Sean Busbey >Priority: Blocker > > How does code handle snapshot naming convention for MOB files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23723) Ensure MOB compaction works in optimized mode after snapshot clone
[ https://issues.apache.org/jira/browse/HBASE-23723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-23723: Component/s: snapshots > Ensure MOB compaction works in optimized mode after snapshot clone > -- > > Key: HBASE-23723 > URL: https://issues.apache.org/jira/browse/HBASE-23723 > Project: HBase > Issue Type: Sub-task > Components: Compaction, mob, snapshots >Reporter: Vladimir Rodionov >Assignee: Sean Busbey >Priority: Blocker > > How does code handle snapshot naming convention for MOB files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] busbey opened a new pull request #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone
busbey opened a new pull request #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone URL: https://github.com/apache/hbase/pull/1446 This is broken up into 3 commits that should be squashed before merging. * First refactor the MOB compaction tests so that they do more code reuse, so we can add our test(s) once and cover the various configuration options (tests should still pass after this commit) * Next add test(s) for when a snapshot clone happens to ensure we still properly can compact (the write amplification optimized tests will start to fail here) * Fix how we keep track of mob reference files so that we can tell where they ought to be in the filesystem even after a snapshot clone, essentially by aggregating at the hfile level the amount of information we have when resolving a given cell (tests should again pass) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone
busbey commented on a change in pull request #1446: HBASE-23723 Ensure MOB
compaction works in optimized mode after snapshot clone
URL: https://github.com/apache/hbase/pull/1446#discussion_r404584224
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobCompactionWithDefaults.java
##
@@ -0,0 +1,335 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.mob;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Random;
+import java.util.stream.Collectors;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.CompactionState;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.RegionSplitter;
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Mob file compaction base test.
+ * 1. Enables batch mode for regular MOB compaction,
+ *Sets batch size to 7 regions. (Optional)
+ * 2. Disables periodic MOB compactions, sets minimum age to archive to 10 sec
+ * 3. Creates MOB table with 20 regions
+ * 4. Loads MOB data (randomized keys, 1000 rows), flushes data.
+ * 5. Repeats 4. two more times
+ * 6. Verifies that we have 20 *3 = 60 mob files (equals to number of regions
x 3)
+ * 7. Runs major MOB compaction.
+ * 8. Verifies that number of MOB files in a mob directory is 20 x4 = 80
+ * 9. Waits for a period of time larger than minimum age to archive
+ * 10. Runs Mob cleaner chore
+ * 11 Verifies that number of MOB files in a mob directory is 20.
+ * 12 Runs scanner and checks all 3 * 1000 rows.
+ */
+@SuppressWarnings("deprecation")
+@Category(LargeTests.class)
+public class TestMobCompactionWithDefaults {
Review comment:
This file is a rename of `TestMobCompactionBase` that's then combined with
`TestMobCompactionRegularMode.java`. I'm not sure why the github UI doesn't
show it as that. my local git client can see it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-filesystem] busbey commented on issue #12: HBASE-24124 hbase-filesystem to use guava from hbase-thirdparty
busbey commented on issue #12: HBASE-24124 hbase-filesystem to use guava from hbase-thirdparty URL: https://github.com/apache/hbase-filesystem/pull/12#issuecomment-610218411 So we don't exclude the guava that curator has and shades internally, correct? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (HBASE-24081) Provide documentation for running Yetus with HBase
[
https://issues.apache.org/jira/browse/HBASE-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened HBASE-24081:
-
This broke the RAT check for license headers. please ensure the new file has a
license header.
{code}
$ mvn org.apache.rat:apache-rat-plugin:check
...
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.13:check
(default-cli) on project hbase: Too many files with unapproved license: 1 See
RAT report in: /Users/busbey/projects_upstream/hbase/target/rat.txt -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
busbey-mba13:hbase busbey$ vim target/rat.txt
busbey-mba13:hbase busbey$ cat target/rat.txt | grep '?'
!? dev-support/HOW_TO_YETUS_LOCAL.md
{code}
Please see existing Markdown files in the code repo for examples.
> Provide documentation for running Yetus with HBase
> --
>
> Key: HBASE-24081
> URL: https://issues.apache.org/jira/browse/HBASE-24081
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 1.7.0, 2.4.0
>
>
> A colleague asked how to use Yetus with HBase, so I wrote up a little how-to
> doc. Maybe it's useful to someone else?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone
Apache-HBase commented on issue #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone URL: https://github.com/apache/hbase/pull/1446#issuecomment-610234348 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 42s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 17s | master passed | | +1 :green_heart: | checkstyle | 1m 50s | master passed | | +1 :green_heart: | spotbugs | 3m 11s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 28s | the patch passed | | -0 :warning: | checkstyle | 1m 6s | hbase-server: The patch generated 23 new + 35 unchanged - 1 fixed = 58 total (was 36) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 10m 34s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 2m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 41m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1446 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 8f5b0883325c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 93 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1437: HBASE-24055 Make AsyncFSWAL can run on EC cluster
Apache-HBase commented on issue #1437: HBASE-24055 Make AsyncFSWAL can run on EC cluster URL: https://github.com/apache/hbase/pull/1437#issuecomment-610234976 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 37s | master passed | | +1 :green_heart: | compile | 1m 10s | master passed | | -1 :x: | shadedjars | 0m 15s | branch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 21s | the patch passed | | +1 :green_heart: | compile | 1m 16s | the patch passed | | +1 :green_heart: | javac | 1m 16s | the patch passed | | -1 :x: | shadedjars | 0m 13s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 183m 7s | hbase-server in the patch passed. | | | | 202m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1437/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1437 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0536c4fcf66a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 1.8.0_232 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1437/2/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1437/2/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1437/2/testReport/ | | Max. process+thread count | 3475 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1437/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
[
https://issues.apache.org/jira/browse/HBASE-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077017#comment-17077017
]
Hudson commented on HBASE-24126:
Results for branch branch-2
[build #2588 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588//console].
> Up the container nproc uplimit from 1 to 12500
> --
>
> Key: HBASE-24126
> URL: https://issues.apache.org/jira/browse/HBASE-24126
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24126-Up-the-container-nproc-uplimit-from-1000.patch
>
>
> Let me up the container nproc count from 1. If many tests running in
> parallel, container could breach the 1 limit: i.e. if a bunch of
> heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8
> possible concurrents) and we spin up 1-2000 threads in each test, then could
> hit the 1 limit. Move up the limit some expecially after INFRA just upped
> our limit to 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24077) When encounter RowTooBigException, log the row info.
[
https://issues.apache.org/jira/browse/HBASE-24077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077020#comment-17077020
]
Hudson commented on HBASE-24077:
Results for branch branch-2
[build #2588 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588//console].
> When encounter RowTooBigException, log the row info.
>
>
> Key: HBASE-24077
> URL: https://issues.apache.org/jira/browse/HBASE-24077
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.2.4
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
>
> Current when we encounter a big row and throw RowTooBigException, there is no
> information about the row, it hard to debug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24115) Relocate test-only REST "client" from src/ to test/ and mark Private
[
https://issues.apache.org/jira/browse/HBASE-24115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077019#comment-17077019
]
Hudson commented on HBASE-24115:
Results for branch branch-2
[build #2588 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588//console].
> Relocate test-only REST "client" from src/ to test/ and mark Private
>
>
> Key: HBASE-24115
> URL: https://issues.apache.org/jira/browse/HBASE-24115
> Project: HBase
> Issue Type: Test
> Components: REST, security
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0, 2.3.1, 1.3.7, 1.7.0, 2.4.0, 2.1.10, 1.4.14, 2.2.5
>
>
> Relocate test-only REST "client" from src/ to test/ and annotate as Private.
> The classes o.a.h.h.rest.Remote* were developed to facilitate REST unit tests
> and incorrectly committed to src/ .
> Although this "breaks" compatibility by moving public classes to test jar and
> marking them private, no attention has been paid to these classes with
> respect to performance, convenience, or security. Consensus from various
> discussions over the years is to move them to test/ as was intent of the
> original committer, but misplaced by the same individual.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24081) Provide documentation for running Yetus with HBase
[
https://issues.apache.org/jira/browse/HBASE-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077018#comment-17077018
]
Hudson commented on HBASE-24081:
Results for branch branch-2
[build #2588 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2588//console].
> Provide documentation for running Yetus with HBase
> --
>
> Key: HBASE-24081
> URL: https://issues.apache.org/jira/browse/HBASE-24081
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 1.7.0, 2.4.0
>
>
> A colleague asked how to use Yetus with HBase, so I wrote up a little how-to
> doc. Maybe it's useful to someone else?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone
Apache-HBase commented on issue #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone URL: https://github.com/apache/hbase/pull/1446#issuecomment-610279301 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 37s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | -1 :x: | shadedjars | 0m 13s | branch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 7s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | -1 :x: | shadedjars | 0m 13s | patch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 36s | hbase-server generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 8s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 115m 11s | hbase-server in the patch passed. | | | | 136m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1446 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8e5315937ec6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 1.8.0_232 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/testReport/ | | Max. process+thread count | 4149 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone
Apache-HBase commented on issue #1446: HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone URL: https://github.com/apache/hbase/pull/1446#issuecomment-610281593 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 39s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 20s | master passed | | +1 :green_heart: | compile | 1m 38s | master passed | | -1 :x: | shadedjars | 0m 15s | branch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 6m 7s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | -1 :x: | shadedjars | 0m 13s | patch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 117m 13s | hbase-server in the patch passed. | | | | 140m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1446 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bab6c61537f7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 2020-01-14 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/testReport/ | | Max. process+thread count | 4232 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1446/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
Reidd commented on a change in pull request #1439: HBASE-24121
[Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it
should be
URL: https://github.com/apache/hbase/pull/1439#discussion_r404678758
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
##
@@ -313,6 +314,7 @@ public void onConfigurationChange(Configuration newConf) {
if (scheduler instanceof ConfigurationObserver) {
((ConfigurationObserver) scheduler).onConfigurationChange(newConf);
}
+HBasePolicyProvider.init(newConf, authManager);
Review comment:
Fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
Reidd commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be URL: https://github.com/apache/hbase/pull/1439#issuecomment-610297423 About the test, the functional test of ServiceAuthorizationManager is done in hadoop-common, and the re-config framework is also testify by many other UTs already. WDYT. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd edited a comment on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
Reidd edited a comment on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be URL: https://github.com/apache/hbase/pull/1439#issuecomment-610297423 About the test, the functional test of ServiceAuthorizationManager is done in hadoop-common, and the re-config framework is also testified by many other UTs already. WDYT. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup
Reidd commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup URL: https://github.com/apache/hbase/pull/1435#issuecomment-610309337 Just rolled back to the previous version, the one using setGroups. It is the appropriate & right approach by now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
Apache-HBase commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be URL: https://github.com/apache/hbase/pull/1439#issuecomment-610310031 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 14s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | spotbugs | 2m 19s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 41s | the patch passed | | +1 :green_heart: | checkstyle | 1m 18s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 33s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 2m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 42m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1439 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux e5f28cab3560 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Max. process+thread count | 83 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Reidddddd edited a comment on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup
Reidd edited a comment on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup URL: https://github.com/apache/hbase/pull/1435#issuecomment-610309337 Just rolled back to the previous version, the one using setRSGroups. It is the appropriate & right approach by now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup
Apache-HBase commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup URL: https://github.com/apache/hbase/pull/1435#issuecomment-610331383 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 55s | master passed | | +1 :green_heart: | checkstyle | 2m 50s | master passed | | +1 :green_heart: | spotbugs | 7m 52s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 57s | the patch passed | | -0 :warning: | checkstyle | 0m 28s | hbase-client: The patch generated 1 new + 37 unchanged - 0 fixed = 38 total (was 37) | | -0 :warning: | checkstyle | 1m 7s | hbase-server: The patch generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 22s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 2m 46s | the patch passed | | +1 :green_heart: | spotbugs | 8m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 0s | The patch does not generate ASF License warnings. | | | | 54m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1435 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle cc hbaseprotoc prototool | | uname | Linux 247cbd306d56 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 93 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] nyl3532016 closed pull request #1436: HBASE-23933 Separate a hbase-balancer module
nyl3532016 closed pull request #1436: HBASE-23933 Separate a hbase-balancer module URL: https://github.com/apache/hbase/pull/1436 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24115) Relocate test-only REST "client" from src/ to test/ and mark Private
[
https://issues.apache.org/jira/browse/HBASE-24115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077157#comment-17077157
]
Hudson commented on HBASE-24115:
Results for branch branch-1
[build #1278 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Relocate test-only REST "client" from src/ to test/ and mark Private
>
>
> Key: HBASE-24115
> URL: https://issues.apache.org/jira/browse/HBASE-24115
> Project: HBase
> Issue Type: Test
> Components: REST, security
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0, 2.3.1, 1.3.7, 1.7.0, 2.4.0, 2.1.10, 1.4.14, 2.2.5
>
>
> Relocate test-only REST "client" from src/ to test/ and annotate as Private.
> The classes o.a.h.h.rest.Remote* were developed to facilitate REST unit tests
> and incorrectly committed to src/ .
> Although this "breaks" compatibility by moving public classes to test jar and
> marking them private, no attention has been paid to these classes with
> respect to performance, convenience, or security. Consensus from various
> discussions over the years is to move them to test/ as was intent of the
> original committer, but misplaced by the same individual.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24081) Provide documentation for running Yetus with HBase
[
https://issues.apache.org/jira/browse/HBASE-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077156#comment-17077156
]
Hudson commented on HBASE-24081:
Results for branch branch-1
[build #1278 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1278//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Provide documentation for running Yetus with HBase
> --
>
> Key: HBASE-24081
> URL: https://issues.apache.org/jira/browse/HBASE-24081
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 1.7.0, 2.4.0
>
>
> A colleague asked how to use Yetus with HBase, so I wrote up a little how-to
> doc. Maybe it's useful to someone else?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
Apache-HBase commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be URL: https://github.com/apache/hbase/pull/1439#issuecomment-610349912 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 30s | master passed | | +1 :green_heart: | compile | 1m 14s | master passed | | -1 :x: | shadedjars | 0m 13s | branch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 49s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 29s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | -1 :x: | shadedjars | 0m 12s | patch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 46s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 118m 56s | hbase-server in the patch failed. | | | | 140m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1439 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3625d1fe8736 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 2020-01-14 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/testReport/ | | Max. process+thread count | 3395 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
Apache-HBase commented on issue #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be URL: https://github.com/apache/hbase/pull/1439#issuecomment-610371228 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 17s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | -1 :x: | shadedjars | 0m 11s | branch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 43s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | -1 :x: | shadedjars | 0m 10s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 169m 14s | hbase-server in the patch passed. | | | | 187m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1439 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux abc935bbbd77 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 1.8.0_232 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/testReport/ | | Max. process+thread count | 3614 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1439/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24077) When encounter RowTooBigException, log the row info.
[
https://issues.apache.org/jira/browse/HBASE-24077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077211#comment-17077211
]
Hudson commented on HBASE-24077:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> When encounter RowTooBigException, log the row info.
>
>
> Key: HBASE-24077
> URL: https://issues.apache.org/jira/browse/HBASE-24077
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.2.4
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
>
> Current when we encounter a big row and throw RowTooBigException, there is no
> information about the row, it hard to debug.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[
https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077206#comment-17077206
]
Hudson commented on HBASE-24118:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> [Flakey Tests] TestCloseRegionWhileRSCrash
> --
>
> Key: HBASE-24118
> URL: https://issues.apache.org/jira/browse/HBASE-24118
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch
>
>
> TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because
> moved Region may not online in the end. Remove the last bit of the test...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24115) Relocate test-only REST "client" from src/ to test/ and mark Private
[
https://issues.apache.org/jira/browse/HBASE-24115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077209#comment-17077209
]
Hudson commented on HBASE-24115:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Relocate test-only REST "client" from src/ to test/ and mark Private
>
>
> Key: HBASE-24115
> URL: https://issues.apache.org/jira/browse/HBASE-24115
> Project: HBase
> Issue Type: Test
> Components: REST, security
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0, 2.3.1, 1.3.7, 1.7.0, 2.4.0, 2.1.10, 1.4.14, 2.2.5
>
>
> Relocate test-only REST "client" from src/ to test/ and annotate as Private.
> The classes o.a.h.h.rest.Remote* were developed to facilitate REST unit tests
> and incorrectly committed to src/ .
> Although this "breaks" compatibility by moving public classes to test jar and
> marking them private, no attention has been paid to these classes with
> respect to performance, convenience, or security. Consensus from various
> discussions over the years is to move them to test/ as was intent of the
> original committer, but misplaced by the same individual.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24111) Enable CompactionTool executions on non-HDFS filesystems
[
https://issues.apache.org/jira/browse/HBASE-24111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077203#comment-17077203
]
Hudson commented on HBASE-24111:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Enable CompactionTool executions on non-HDFS filesystems
>
>
> Key: HBASE-24111
> URL: https://issues.apache.org/jira/browse/HBASE-24111
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, mapreduce, tooling
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.4.0, 2.2.5
>
>
> CompactionTool on non-HDFS filesystems does not work because the used
> filesystems are mixed up. To collect the StoreFiles the
> {{Filesystem.get(conf)}} method is used instead of relying on the root dir
> filesystem.
> With YARN the delegation tokens are not handled correctly with a different
> filesystem and the mappers only get delegation token for the staging
> directory's filesystem.
> Another issue is in MapReduce mode when YARN is used. In this case the TMP
> directory ({{hbase.tmp.dir}}) is created under
> {{yarn.nodemanager.local-dirs}} but this directory is created (even on HDFS a
> regular user can't create this directory) on the Storefile's filesystem where
> a local filesystem temp directory shouldn't be used.
> In this ticket I plan to clean up the used filesystems, remove the custom
> temp directory (use the regions' .tmp directory) and collect delegation
> tokens for staging dir + store dir paths' FileSystems.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24119) Polish the protobuf usage in hbase-examples
[
https://issues.apache.org/jira/browse/HBASE-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077204#comment-17077204
]
Hudson commented on HBASE-24119:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Polish the protobuf usage in hbase-examples
> ---
>
> Key: HBASE-24119
> URL: https://issues.apache.org/jira/browse/HBASE-24119
> Project: HBase
> Issue Type: Sub-task
> Components: Protobufs
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24081) Provide documentation for running Yetus with HBase
[
https://issues.apache.org/jira/browse/HBASE-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077210#comment-17077210
]
Hudson commented on HBASE-24081:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Provide documentation for running Yetus with HBase
> --
>
> Key: HBASE-24081
> URL: https://issues.apache.org/jira/browse/HBASE-24081
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 1.7.0, 2.4.0
>
>
> A colleague asked how to use Yetus with HBase, so I wrote up a little how-to
> doc. Maybe it's useful to someone else?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[
https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077201#comment-17077201
]
Hudson commented on HBASE-24118:
Results for branch branch-2.3
[build #20 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Tests] TestCloseRegionWhileRSCrash
> --
>
> Key: HBASE-24118
> URL: https://issues.apache.org/jira/browse/HBASE-24118
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch
>
>
> TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because
> moved Region may not online in the end. Remove the last bit of the test...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24115) Relocate test-only REST "client" from src/ to test/ and mark Private
[
https://issues.apache.org/jira/browse/HBASE-24115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077202#comment-17077202
]
Hudson commented on HBASE-24115:
Results for branch branch-2.3
[build #20 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/20/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Relocate test-only REST "client" from src/ to test/ and mark Private
>
>
> Key: HBASE-24115
> URL: https://issues.apache.org/jira/browse/HBASE-24115
> Project: HBase
> Issue Type: Test
> Components: REST, security
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0, 2.3.1, 1.3.7, 1.7.0, 2.4.0, 2.1.10, 1.4.14, 2.2.5
>
>
> Relocate test-only REST "client" from src/ to test/ and annotate as Private.
> The classes o.a.h.h.rest.Remote* were developed to facilitate REST unit tests
> and incorrectly committed to src/ .
> Although this "breaks" compatibility by moving public classes to test jar and
> marking them private, no attention has been paid to these classes with
> respect to performance, convenience, or security. Consensus from various
> discussions over the years is to move them to test/ as was intent of the
> original committer, but misplaced by the same individual.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24122) Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory'
[
https://issues.apache.org/jira/browse/HBASE-24122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077205#comment-17077205
]
Hudson commented on HBASE-24122:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just
> 'max locked memory'
> -
>
> Key: HBASE-24122
> URL: https://issues.apache.org/jira/browse/HBASE-24122
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
> Attachments:
> 0001-HBASE-24122-Change-machine-ulimit-l-to-ulimit-a-so-d.patch
>
>
> Dump out full ulimit list under the machine dir job output rather than
> one-liner. More utility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24086) Disable output stream capability enforcement when running in standalone mode
[
https://issues.apache.org/jira/browse/HBASE-24086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077208#comment-17077208
]
Hudson commented on HBASE-24086:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Disable output stream capability enforcement when running in standalone mode
>
>
> Key: HBASE-24086
> URL: https://issues.apache.org/jira/browse/HBASE-24086
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> {noformat}
> $
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
> mvn clean install -DskipTests
> $
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home
> ./bin/hbase master start
> {noformat}
> gives
> {noformat}
> 2020-03-30 17:12:43,857 ERROR
> [master/192.168.111.13:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
>
> java.io.IOException: cannot get log writer
>
>
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:118)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:704)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:710)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:839)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:549)
>
>
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:490)
>
>
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:156)
>
>
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:61)
>
>
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:297)
>
>
> at
> org.apache.hadoop.hbase.procedure2.store.region.RegionProc
[jira] [Commented] (HBASE-24106) Update getting started documentation after HBASE-24086
[
https://issues.apache.org/jira/browse/HBASE-24106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077207#comment-17077207
]
Hudson commented on HBASE-24106:
Results for branch master
[build #1691 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1691/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1691//console].
> Update getting started documentation after HBASE-24086
> --
>
> Key: HBASE-24106
> URL: https://issues.apache.org/jira/browse/HBASE-24106
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0
>
>
> HBASE-24086 allows HBase to degrade gracefully to running on a
> {{LocalFileSystem}} without further user configuration. Update the docs
> accordingly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610399199 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610399166 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610399184 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24081) Provide documentation for running Yetus with HBase
[ https://issues.apache.org/jira/browse/HBASE-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077264#comment-17077264 ] Minwoo Kang commented on HBASE-24081: - I had rat error 10 hours ago. I made an [issue|https://issues.apache.org/jira/browse/HBASE-24130] and [Pull Request|[https://github.com/apache/hbase/pull/1445]]. > Provide documentation for running Yetus with HBase > -- > > Key: HBASE-24081 > URL: https://issues.apache.org/jira/browse/HBASE-24081 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0, 1.7.0, 2.4.0 > > > A colleague asked how to use Yetus with HBase, so I wrote up a little how-to > doc. Maybe it's useful to someone else? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24131) [Flakey Tests] TestExportSnapshot takes too long; up against 13min max
[
https://issues.apache.org/jira/browse/HBASE-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077269#comment-17077269
]
Michael Stack commented on HBASE-24131:
---
6 tests all timed out in this run actually. They all ran concurrently. All but
one were permutations on TestExportSnapshot. All failed w/ TEST TIMED OUT
running for longer than our 13 minute max:
{code}
[INFO]
[ERROR] Errors:
[ERROR] org.apache.hadoop.hbase.replication.TestVerifyReplication.null
[ERROR] Run 1:
TestVerifyReplication.testVerifyReplicationWithSnapshotSupport:449 »
TestTimedOut
[ERROR] Run 2: TestVerifyReplication » Appears to be stuck in thread Socket
Reader #1 for po...
[INFO]
[ERROR] org.apache.hadoop.hbase.snapshot.TestExportSnapshot.null
[ERROR] Run 1:
TestExportSnapshot.testExportRetry:267->testExportFileSystemState:227 »
TestTimedOut
[ERROR] Run 2: TestExportSnapshot » Appears to be stuck in thread
RS-EventLoopGroup-5-3
[INFO]
[ERROR]
org.apache.hadoop.hbase.snapshot.TestExportSnapshotWithTemporaryDirectory.null
[ERROR] Run 1:
TestExportSnapshotWithTemporaryDirectory>TestExportSnapshot.testExportRetry:267->TestExportSnapshot.testExportFileSystemState:227
» TestTimedOut
[ERROR] Run 2: TestExportSnapshotWithTemporaryDirectory » Appears to be
stuck in thread RS-E...
[INFO]
[ERROR] org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot.null
[ERROR] Run 1:
TestMobExportSnapshot>TestExportSnapshot.testExportRetry:267->TestExportSnapshot.testExportFileSystemState:227
» TestTimedOut
[ERROR] Run 2: TestMobExportSnapshot » Appears to be stuck in thread
RS-EventLoopGroup-5-3
[INFO]
[ERROR] org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot.null
[ERROR] Run 1:
TestMobSecureExportSnapshot>TestExportSnapshot.testExportRetry:267->TestExportSnapshot.testExportFileSystemState:227
» TestTimedOut
[ERROR] Run 2: TestMobSecureExportSnapshot » Appears to be stuck in thread
RS-EventLoopGroup...
[INFO]
[ERROR] org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot.null
[ERROR] Run 1:
TestSecureExportSnapshot>TestExportSnapshot.testExportRetry:267->TestExportSnapshot.testExportFileSystemState:227
» TestTimedOut
[ERROR] Run 2: TestSecureExportSnapshot » Appears to be stuck in thread
RS-EventLoopGroup-5-...
[INFO]
[INFO]
[ERROR] Tests run: 275, Failures: 0, Errors: 6, Skipped: 29
{code}
Let me break up these tests into smaller pieces.
> [Flakey Tests] TestExportSnapshot takes too long; up against 13min max
> --
>
> Key: HBASE-24131
> URL: https://issues.apache.org/jira/browse/HBASE-24131
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Priority: Major
>
> TestExportSnapshot fails fairly regularly locally. Looking, its test timeout.
> Looking at how long it ran, its 13minutes plus. Looking at recent successful
> branch-2 run, it passed but it took about 7-8minutes. Let me break up the
> test into to pieces.
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
>at
> org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemState(TestExportSnapshot.java:227)
>at
> org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportRetry(TestExportSnapshot.java:267)
> ... I see this in the log:
> > TEST TIMED OUT. PRINTING THREAD DUMP. <
> Test started at:
> 2020-04-06 17:19:21,739 INFO
> ... and the timestamp just above the TIMED OUT was
> 2020-04-06 17:31:01,758
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610408695 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 3s | branch-2 passed | | +1 :green_heart: | compile | 0m 21s | branch-2 passed | | -1 :x: | shadedjars | 0m 11s | branch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 51s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | -1 :x: | shadedjars | 0m 11s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 19s | hbase-common in the patch passed. | | | | 17m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1426 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f378199a01ee 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 4a85f06590 | | Default Java | 1.8.0_232 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/testReport/ | | Max. process+thread count | 225 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610410705 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 1s | branch-2 passed | | +1 :green_heart: | compile | 0m 28s | branch-2 passed | | -1 :x: | shadedjars | 0m 12s | branch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 20s | hbase-common in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 8s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | -1 :x: | shadedjars | 0m 11s | patch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 39s | hbase-common in the patch passed. | | | | 21m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1426 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ae376a3e327f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 4a85f06590 | | Default Java | 2020-01-14 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/testReport/ | | Max. process+thread count | 285 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24127) Switch to `RawLocalFileSystem` as default FS for tests
[
https://issues.apache.org/jira/browse/HBASE-24127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077278#comment-17077278
]
Josh Elser commented on HBASE-24127:
I couldn't tell you why it is the way it is, just that it's been this way since
Hadoop 2.6 times :)
> Switch to `RawLocalFileSystem` as default FS for tests
> --
>
> Key: HBASE-24127
> URL: https://issues.apache.org/jira/browse/HBASE-24127
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Nick Dimiduk
>Priority: Major
>
> In the PR discussion on HBASE-24106, [~elserj]
> [mentioned|https://github.com/apache/hbase/pull/1422#discussion_r403789851]
> that another project has had success with switching its unit test suite to
> use {{RawLocalFileSystem}} from {{LocalFileSystem}}. This is discussed in the
> context of the availability of the {{hsync}} and {{hflush}} stream
> capabilities, which are not available on {{LocalFileSystem}}.
> Perhaps we should also consider doing the same for our default {{standalone}}
> user experience. I don't know if doing so should be a separate task from this
> one, or if completing one is effectively performing the implementation for
> the other.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
Apache-HBase commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610415953 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 25s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 26s | branch-2 passed | | +1 :green_heart: | spotbugs | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 11s | the patch passed | | +1 :green_heart: | checkstyle | 0m 23s | the patch passed | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 10m 37s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 31m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1426 | | Optional Tests | dupname asflicense shellcheck shelldocs spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b5f58075da8b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 4a85f06590 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 93 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1426/23/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24132) Upgrade to Apache ZooKeeper 3.5.7
[ https://issues.apache.org/jira/browse/HBASE-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HBASE-24132: -- Attachment: HBASE-24132_001.patch > Upgrade to Apache ZooKeeper 3.5.7 > - > > Key: HBASE-24132 > URL: https://issues.apache.org/jira/browse/HBASE-24132 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.2.3 >Reporter: Jianfei Jiang >Priority: Major > Attachments: HBASE-24132_001.patch > > > Apache ZooKeeper 3.5.7 has been released, HDFS and other projects have > updated the dependent zookeeper verison. Perhaps Hbase may update as well. > Some of the interfaces are changed in this zookeeper version . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24132) Upgrade to Apache ZooKeeper 3.5.7
Jianfei Jiang created HBASE-24132: - Summary: Upgrade to Apache ZooKeeper 3.5.7 Key: HBASE-24132 URL: https://issues.apache.org/jira/browse/HBASE-24132 Project: HBase Issue Type: Improvement Affects Versions: 2.2.3, 3.0.0 Reporter: Jianfei Jiang Attachments: HBASE-24132_001.patch Apache ZooKeeper 3.5.7 has been released, HDFS and other projects have updated the dependent zookeeper verison. Perhaps Hbase may update as well. Some of the interfaces are changed in this zookeeper version . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24132) Upgrade to Apache ZooKeeper 3.5.7
[ https://issues.apache.org/jira/browse/HBASE-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HBASE-24132: -- Status: Patch Available (was: Open) > Upgrade to Apache ZooKeeper 3.5.7 > - > > Key: HBASE-24132 > URL: https://issues.apache.org/jira/browse/HBASE-24132 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.2.3, 3.0.0 >Reporter: Jianfei Jiang >Priority: Major > Attachments: HBASE-24132_001.patch > > > Apache ZooKeeper 3.5.7 has been released, HDFS and other projects have > updated the dependent zookeeper verison. Perhaps Hbase may update as well. > Some of the interfaces are changed in this zookeeper version . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup
Apache-HBase commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup URL: https://github.com/apache/hbase/pull/1435#issuecomment-610417394 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 50s | master passed | | +1 :green_heart: | compile | 3m 10s | master passed | | -1 :x: | shadedjars | 0m 13s | branch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 45s | the patch passed | | +1 :green_heart: | compile | 3m 6s | the patch passed | | +1 :green_heart: | javac | 3m 6s | the patch passed | | -1 :x: | shadedjars | 0m 10s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 52s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 198m 30s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 23s | hbase-thrift in the patch passed. | | | | 234m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1435 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 602e73f85d8b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 1.8.0_232 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/testReport/ | | Max. process+thread count | 2655 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup
Apache-HBase commented on issue #1435: HBASE-24112 [RSGroup] Support renaming rsgroup URL: https://github.com/apache/hbase/pull/1435#issuecomment-610421484 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 9s | master passed | | +1 :green_heart: | compile | 4m 13s | master passed | | -1 :x: | shadedjars | 0m 12s | branch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 28s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 43s | hbase-server in master failed. | | -0 :warning: | javadoc | 1m 0s | hbase-thrift in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 7m 1s | the patch passed | | +1 :green_heart: | compile | 4m 11s | the patch passed | | +1 :green_heart: | javac | 4m 11s | the patch passed | | -1 :x: | shadedjars | 0m 13s | patch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 29s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 45s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 1m 12s | hbase-thrift in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 17s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 8s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 198m 45s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 5s | hbase-thrift in the patch passed. | | | | 241m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1435 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f42e90b2ddff 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 62718cdb28 | | Default Java | 2020-01-14 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1435/4/testReport/ | | Max. process+thread count | 2798 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-server hb
[jira] [Commented] (HBASE-24072) Nightlies reporting OutOfMemoryError: unable to create new native thread
[
https://issues.apache.org/jira/browse/HBASE-24072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077299#comment-17077299
]
Michael Stack commented on HBASE-24072:
---
HBASE-24126 is about upping container ulimit.
> Nightlies reporting OutOfMemoryError: unable to create new native thread
>
>
> Key: HBASE-24072
> URL: https://issues.apache.org/jira/browse/HBASE-24072
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24072-Nightlies-reporting-OutOfMemoryError-una.patch,
> print_ulimit.patch
>
>
> Seeing this kind of thing in nightly...
> {code}
> java.lang.RuntimeException: java.lang.OutOfMemoryError: unable to create new
> native thread
> at
> org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper.beforeClass(TestMultithreadedTableMapper.java:83)
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
> at
> org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper.beforeClass(TestMultithreadedTableMapper.java:83)
> {code}
> Chatting w/ Nick and Huaxiang, doing the math, we are likely oversubscribing
> our docker container. It is set to 20G (The hosts are 48G). Fork count is
> 0.5C on a 16 CPU machine which is 8 *2.8G our current forked jvm size. Add
> the maven 4G and we could be over the top.
> Play w/ downing the fork size (in earlier study we didn't seem to need this
> much RAM when running a fat long test). Let me also take th ms off the mvn
> allocation to see if that helps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
[ https://issues.apache.org/jira/browse/HBASE-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077302#comment-17077302 ] Michael Stack commented on HBASE-24126: --- Pushed an addendum, attached. Also pushed change to personality: diff --git a/dev-support/hbase-personality.sh b/dev-support/hbase-personality.sh index 58d2aa8266..be305c5443 100755 --- a/dev-support/hbase-personality.sh +++ b/dev-support/hbase-personality.sh @@ -82,7 +82,7 @@ function personality_globals # Yetus 0.7.0 enforces limits. Default proclimit is 1000. # Up it. See HBASE-19902 for how we arrived at this number. #shellcheck disable=SC2034 - PROCLIMIT=1 + PROCLIMIT=12500 > Up the container nproc uplimit from 1 to 12500 > -- > > Key: HBASE-24126 > URL: https://issues.apache.org/jira/browse/HBASE-24126 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Michael Stack >Priority: Major > Attachments: > 0001-HBASE-24126-Up-the-container-nproc-uplimit-from-1000.patch > > > Let me up the container nproc count from 1. If many tests running in > parallel, container could breach the 1 limit: i.e. if a bunch of > heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8 > possible concurrents) and we spin up 1-2000 threads in each test, then could > hit the 1 limit. Move up the limit some expecially after INFRA just upped > our limit to 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack closed pull request #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
saintstack closed pull request #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne…
saintstack commented on issue #1426: HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create ne… URL: https://github.com/apache/hbase/pull/1426#issuecomment-610430572 Shutting down this experiment. Doesn't work. Can't set hbase-personality changes and have them take effect inline w/ the PR run. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 merged pull request #1437: HBASE-24055 Make AsyncFSWAL can run on EC cluster
Apache9 merged pull request #1437: HBASE-24055 Make AsyncFSWAL can run on EC cluster URL: https://github.com/apache/hbase/pull/1437 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
[
https://issues.apache.org/jira/browse/HBASE-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077358#comment-17077358
]
Michael Stack commented on HBASE-24126:
---
To be clear (since my application has been messy), this issue has three commits
against it on branch-2; an original commit followed by two addendums:
{code}
commit 01715fbbf6e00edbb4497bfa72e3c2a5b9937088
Author: stack
Date: Mon Apr 6 18:23:35 2020 -0700
HBASE-24126 Up the container nproc uplimit from 1 to 12500
Addendum number 2; docker doesn't like the ulimit arg as is.
commit 25773c681b35e231152b3bb2347e7f55f03ba2e4
Author: stack
Date: Mon Apr 6 16:29:46 2020 -0700
HBASE-24126 Up the container nproc uplimit from 1 to 12500
ADDENDUM: Add to hbase-personality
commit ed24998b85dec1dddedd636b01d87bcdb0445ef0 (2.up.nproc)
Author: stack
Date: Mon Apr 6 13:44:53 2020 -0700
HBASE-24126 Up the container nproc uplimit from 1 to 12500
{code}
A related push against HBASE-24072 is complimentary making use of the changes
made in this JIRA:
{code}
commit 9c60c5fddbd9b1910fa59c44592249696948041a (HEAD -> 2, origin/branch-2)
Author: stack
Date: Thu Apr 2 18:29:05 2020 -0700
HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new
native thread.
Up the nproc in hbase-personality. Add test that prints out env.
{code}
Will push everywhere after confirmed all works.
> Up the container nproc uplimit from 1 to 12500
> --
>
> Key: HBASE-24126
> URL: https://issues.apache.org/jira/browse/HBASE-24126
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24126-Up-the-container-nproc-uplimit-from-1000.patch
>
>
> Let me up the container nproc count from 1. If many tests running in
> parallel, container could breach the 1 limit: i.e. if a bunch of
> heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8
> possible concurrents) and we spin up 1-2000 threads in each test, then could
> hit the 1 limit. Move up the limit some expecially after INFRA just upped
> our limit to 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
[ https://issues.apache.org/jira/browse/HBASE-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24126: -- Release Note: Start docker with upped ulimit for nproc. It was 1 but made it 12500. > Up the container nproc uplimit from 1 to 12500 > -- > > Key: HBASE-24126 > URL: https://issues.apache.org/jira/browse/HBASE-24126 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Michael Stack >Priority: Major > Attachments: > 0001-HBASE-24126-Up-the-container-nproc-uplimit-from-1000.patch > > > Let me up the container nproc count from 1. If many tests running in > parallel, container could breach the 1 limit: i.e. if a bunch of > heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8 > possible concurrents) and we spin up 1-2000 threads in each test, then could > hit the 1 limit. Move up the limit some expecially after INFRA just upped > our limit to 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24055) Make AsyncFSWAL can run on EC cluster
[ https://issues.apache.org/jira/browse/HBASE-24055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24055. --- Resolution: Fixed Pushed to branch-2.3+. Thanks [~stack] and [~ndimiduk] for reviewing. > Make AsyncFSWAL can run on EC cluster > - > > Key: HBASE-24055 > URL: https://issues.apache.org/jira/browse/HBASE-24055 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > We need to enable replicate when creating the file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24133) Temporary issue... ignore
Michael Stack created HBASE-24133: - Summary: Temporary issue... ignore Key: HBASE-24133 URL: https://issues.apache.org/jira/browse/HBASE-24133 Project: HBase Issue Type: Sub-task Reporter: Michael Stack -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #1447: HBASE-24133 Temporary issue... ignore
saintstack opened a new pull request #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24055) Make AsyncFSWAL can run on EC cluster
[ https://issues.apache.org/jira/browse/HBASE-24055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-24055: -- Fix Version/s: 2.3.0 3.0.0 Hadoop Flags: Reviewed Release Note: Now AsyncFSWAL can also be used against the directory which has EC enabled. Need to make sure you also make use of the hadoop 3.x client as the option is only available in hadoop 3.x. > Make AsyncFSWAL can run on EC cluster > - > > Key: HBASE-24055 > URL: https://issues.apache.org/jira/browse/HBASE-24055 > Project: HBase > Issue Type: Improvement > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > We need to enable replicate when creating the file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24133) Temporary issue... ignore
[ https://issues.apache.org/jira/browse/HBASE-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24133: -- Description: Issue to try PRs against. Ignore. > Temporary issue... ignore > - > > Key: HBASE-24133 > URL: https://issues.apache.org/jira/browse/HBASE-24133 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Major > > Issue to try PRs against. Ignore. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #1445: HBASE-24130 rat plugin complains about having an unlicensed file
ndimiduk merged pull request #1445: HBASE-24130 rat plugin complains about having an unlicensed file URL: https://github.com/apache/hbase/pull/1445 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk merged pull request #1448: Backport" HBASE-24130 rat plugin complains about having an unlicensed file (#1445)" to branch-2
ndimiduk merged pull request #1448: Backport" HBASE-24130 rat plugin complains about having an unlicensed file (#1445)" to branch-2 URL: https://github.com/apache/hbase/pull/1448 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1448: Backport" HBASE-24130 rat plugin complains about having an unlicensed file (#1445)" to branch-2
ndimiduk opened a new pull request #1448: Backport" HBASE-24130 rat plugin complains about having an unlicensed file (#1445)" to branch-2 URL: https://github.com/apache/hbase/pull/1448 Signed-off-by: Jan Hentschel Signed-off-by: Nick Dimiduk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk merged pull request #1449: Backport "HBASE-24130 rat plugin complains about having an unlicensed file (#1445)" to branch-1
ndimiduk merged pull request #1449: Backport "HBASE-24130 rat plugin complains about having an unlicensed file (#1445)" to branch-1 URL: https://github.com/apache/hbase/pull/1449 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1449: HBASE-24130 rat plugin complains about having an unlicensed file (#1445)
ndimiduk opened a new pull request #1449: HBASE-24130 rat plugin complains about having an unlicensed file (#1445) URL: https://github.com/apache/hbase/pull/1449 Signed-off-by: Jan Hentschel Signed-off-by: Nick Dimiduk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-24130) rat plugin complains about having an unlicensed file.
[ https://issues.apache.org/jira/browse/HBASE-24130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24130. -- Fix Version/s: 2.4.0 1.7.0 3.0.0 Resolution: Fixed Thanks for the quick turnaround [~minwoo.kang]! > rat plugin complains about having an unlicensed file. > - > > Key: HBASE-24130 > URL: https://issues.apache.org/jira/browse/HBASE-24130 > Project: HBase > Issue Type: Bug >Reporter: Minwoo Kang >Assignee: Minwoo Kang >Priority: Minor > Fix For: 3.0.0, 1.7.0, 2.4.0 > > > Files with unapproved licenses: > Â Â dev-support/HOW_TO_YETUS_LOCAL.md -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610487408 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 35s | branch-2 passed | | +1 :green_heart: | compile | 0m 23s | branch-2 passed | | -1 :x: | shadedjars | 0m 14s | branch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 11s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | -1 :x: | shadedjars | 0m 12s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 16s | hbase-common in the patch passed. | | | | 15m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 64c5be84fec3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 74a85e26ee | | Default Java | 1.8.0_232 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/testReport/ | | Max. process+thread count | 376 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack merged pull request #1442: hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAss…
saintstack merged pull request #1442: hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAss… URL: https://github.com/apache/hbase/pull/1442 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610489643 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 10s | branch-2 passed | | +1 :green_heart: | compile | 0m 24s | branch-2 passed | | -1 :x: | shadedjars | 0m 12s | branch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-common in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 48s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | -1 :x: | shadedjars | 0m 11s | patch has 7 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 58s | hbase-common in the patch passed. | | | | 20m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5edb16644f9e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 74a85e26ee | | Default Java | 2020-01-14 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/testReport/ | | Max. process+thread count | 272 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-24128) [Flakey Tests] Add retry on thrift cmdline if client fails plus misc debug
[
https://issues.apache.org/jira/browse/HBASE-24128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24128.
---
Fix Version/s: 2.3.0
3.0.0
Assignee: Michael Stack
Resolution: Fixed
Merged to branch 2.3+.
> [Flakey Tests] Add retry on thrift cmdline if client fails plus misc debug
> --
>
> Key: HBASE-24128
> URL: https://issues.apache.org/jira/browse/HBASE-24128
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> {code}
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
> Saw case where Master failed startup because of BindException but it
> came out wrapped
> in an IOE so we did not trip the BindException retry logic. Make the
> catch more general.
> hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
> Add some debug and up timeouts. This test fails frequently for me
> locally.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
> Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
> This test fails locally and up on jenkins.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
> Debug. Have assert say what bad count was.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
> Fails on occasion. Found count is off by a few. Tricky to debug. Adding
> sub-issue to reenable.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
> Debug. Add wait and check before moving to assert.
>
> hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
> Check for null before shutting; can be null if failed start.
>
> hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
> Add retry if client messes up connection. Fails for me locally.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610495544 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 44s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 25s | branch-2 passed | | +1 :green_heart: | spotbugs | 0m 44s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | -0 :warning: | checkstyle | 0m 23s | hbase-common: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 10m 31s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 31m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1447 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux ea44c971a3eb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 74a85e26ee | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 93 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610500572 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610500568 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610500558 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC.
markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC. URL: https://github.com/apache/hbase/pull/1440#discussion_r404972301 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java ## @@ -67,20 +68,49 @@ * Tests may set this down from unlimited. */ public static final String HBASE_NETTY_EVENTLOOP_RPCSERVER_THREADCOUNT_KEY = -"hbase.netty.eventloop.rpcserver.thread.count"; + "hbase.netty.eventloop.rpcserver.thread.count"; private static final int EVENTLOOP_THREADCOUNT_DEFAULT = 0; + protected static final String SERVER_TCP_BACKLOG = "hbase.ipc.server.tcpbacklog"; + protected static final String SERVER_TCP_REUSEADDR = "hbase.ipc.server.tcpreuseaddr"; + protected static final String SERVER_TCP_NODELAY = "hbase.ipc.server.tcpnodelay"; + protected static final String SERVER_TCP_KEEPALIVE = "hbase.ipc.server.tcpkeepalive"; + + protected static final String SERVER_BUFFER_LOW_WATERMARK = "hbase.ipc.server.bufferlowwatermark"; + protected static final String SERVER_BUFFER_HIGH_WATERMARK = + "hbase.ipc.server.bufferhighwatermark"; + + protected static final boolean DEFAULT_SERVER_REUSEADDR = true; + protected static final int DEFAULT_SERVER_TCP_BACKLOG = 1024; Review comment: The TCP_BACKLOG? I think the linux default is often 128 as mentioned in the JIRA? Raising it too high is also often a mistake, so I went with what the original author did. Have another suggestion? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC.
markrmiller commented on a change in pull request #1440: HBASE-23113: Improve
and add additional Netty configuration for RPC.
URL: https://github.com/apache/hbase/pull/1440#discussion_r404976317
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
##
@@ -258,6 +252,8 @@ private void connect() {
.option(ChannelOption.SO_KEEPALIVE, rpcClient.tcpKeepAlive)
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, rpcClient.connectTO)
.handler(new
BufferCallBeforeInitHandler()).localAddress(rpcClient.localAddr)
+.option(ChannelOption.WRITE_BUFFER_WATER_MARK,
+((NettyRpcClient) rpcClient).writeBufferWaterMark)
Review comment:
Good catch, no longer necessary, from an earlier iteration.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC.
markrmiller commented on a change in pull request #1440: HBASE-23113: Improve
and add additional Netty configuration for RPC.
URL: https://github.com/apache/hbase/pull/1440#discussion_r404975868
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
##
@@ -45,24 +46,45 @@
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.CONFIG)
public class NettyRpcClient extends AbstractRpcClient {
+ protected final WriteBufferWaterMark writeBufferWaterMark;
+
+ protected static final String CLIENT_CONNECT_MAX_RETRIES =
"hbase.ipc.client.connect.max.retries";
Review comment:
Copy paste error, should just be WATERMARK stuff.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC.
markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC. URL: https://github.com/apache/hbase/pull/1440#discussion_r404976832 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java ## @@ -67,20 +68,49 @@ * Tests may set this down from unlimited. */ public static final String HBASE_NETTY_EVENTLOOP_RPCSERVER_THREADCOUNT_KEY = -"hbase.netty.eventloop.rpcserver.thread.count"; + "hbase.netty.eventloop.rpcserver.thread.count"; private static final int EVENTLOOP_THREADCOUNT_DEFAULT = 0; + protected static final String SERVER_TCP_BACKLOG = "hbase.ipc.server.tcpbacklog"; + protected static final String SERVER_TCP_REUSEADDR = "hbase.ipc.server.tcpreuseaddr"; + protected static final String SERVER_TCP_NODELAY = "hbase.ipc.server.tcpnodelay"; Review comment: Same copy paste mistake as above - previous PR had this all in the base class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610512553 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 10s | branch-2 passed | | +1 :green_heart: | compile | 0m 22s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 9s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | shadedjars | 4m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 19s | hbase-common in the patch passed. | | | | 23m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 07a1e368d7ee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 9da7f95fa7 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/testReport/ | | Max. process+thread count | 363 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610513905 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 4s | branch-2 passed | | +1 :green_heart: | compile | 0m 24s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 2s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-common in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 30s | hbase-common in the patch passed. | | | | 26m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux eef0e3daa2ff 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 9da7f95fa7 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/testReport/ | | Max. process+thread count | 361 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC.
markrmiller commented on a change in pull request #1440: HBASE-23113: Improve and add additional Netty configuration for RPC. URL: https://github.com/apache/hbase/pull/1440#discussion_r404979877 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java ## @@ -67,20 +68,49 @@ * Tests may set this down from unlimited. */ public static final String HBASE_NETTY_EVENTLOOP_RPCSERVER_THREADCOUNT_KEY = -"hbase.netty.eventloop.rpcserver.thread.count"; + "hbase.netty.eventloop.rpcserver.thread.count"; private static final int EVENTLOOP_THREADCOUNT_DEFAULT = 0; + protected static final String SERVER_TCP_BACKLOG = "hbase.ipc.server.tcpbacklog"; + protected static final String SERVER_TCP_REUSEADDR = "hbase.ipc.server.tcpreuseaddr"; + protected static final String SERVER_TCP_NODELAY = "hbase.ipc.server.tcpnodelay"; + protected static final String SERVER_TCP_KEEPALIVE = "hbase.ipc.server.tcpkeepalive"; + + protected static final String SERVER_BUFFER_LOW_WATERMARK = "hbase.ipc.server.bufferlowwatermark"; + protected static final String SERVER_BUFFER_HIGH_WATERMARK = + "hbase.ipc.server.bufferhighwatermark"; + + protected static final boolean DEFAULT_SERVER_REUSEADDR = true; + protected static final int DEFAULT_SERVER_TCP_BACKLOG = 1024; Review comment: For the WATER_MARK settings, the previous pr is using: protected static final int DEFAULT_SERVER_BUFFER_LOW_WATERMARK = 1024; protected static final int DEFAULT_SERVER_BUFFER_HIGH_WATERMARK = 64 * 1024; If the number of queued bytes goes above the high mark, then channel.isWritable will return false until the number of outstanding bytes falls below the low water mark. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore
Apache-HBase commented on issue #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447#issuecomment-610516513 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 43s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 25s | branch-2 passed | | +1 :green_heart: | spotbugs | 0m 47s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 13s | the patch passed | | -0 :warning: | checkstyle | 0m 23s | hbase-common: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 10m 55s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 32m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1447 | | Optional Tests | dupname asflicense shellcheck shelldocs spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux f6856d42dff4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 9da7f95fa7 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 93 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1447/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
[ https://issues.apache.org/jira/browse/HBASE-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24126: -- Release Note: Start docker with upped ulimit for nproc. It was 1 but made it 12500. Then, set proclimit when we run yetus to be the new 12500 value. (was: Start docker with upped ulimit for nproc. It was 1 but made it 12500.) > Up the container nproc uplimit from 1 to 12500 > -- > > Key: HBASE-24126 > URL: https://issues.apache.org/jira/browse/HBASE-24126 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Michael Stack >Priority: Major > Attachments: > 0001-HBASE-24126-Up-the-container-nproc-uplimit-from-1000.patch > > > Let me up the container nproc count from 1. If many tests running in > parallel, container could breach the 1 limit: i.e. if a bunch of > heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8 > possible concurrents) and we spin up 1-2000 threads in each test, then could > hit the 1 limit. Move up the limit some expecially after INFRA just upped > our limit to 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
[ https://issues.apache.org/jira/browse/HBASE-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077448#comment-17077448 ] Michael Stack commented on HBASE-24126: --- Ok. Finally figured how to up the nproc limit after pushing 4 times -- an original and 3 three addendums -- against this issue on branch-2. In short, we pass an arg. when we do 'docker run' to up ulimit for nproc on the container and then once the container is running, we pass an upped proclimit to yetus so it will use the new max. Made a PR for master that is an aggregate of the four pushes. > Up the container nproc uplimit from 1 to 12500 > -- > > Key: HBASE-24126 > URL: https://issues.apache.org/jira/browse/HBASE-24126 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Michael Stack >Priority: Major > Attachments: > 0001-HBASE-24126-Up-the-container-nproc-uplimit-from-1000.patch > > > Let me up the container nproc count from 1. If many tests running in > parallel, container could breach the 1 limit: i.e. if a bunch of > heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8 > possible concurrents) and we spin up 1-2000 threads in each test, then could > hit the 1 limit. Move up the limit some expecially after INFRA just upped > our limit to 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
saintstack opened a new pull request #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack closed pull request #1447: HBASE-24133 Temporary issue... ignore
saintstack closed pull request #1447: HBASE-24133 Temporary issue... ignore URL: https://github.com/apache/hbase/pull/1447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-24133) Temporary issue... ignore
[ https://issues.apache.org/jira/browse/HBASE-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24133. --- Resolution: Not A Bug Resolving. Done w/ this hanger. > Temporary issue... ignore > - > > Key: HBASE-24133 > URL: https://issues.apache.org/jira/browse/HBASE-24133 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Major > > Issue to try PRs against. Ignore. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24072) Nightlies reporting OutOfMemoryError: unable to create new native thread
[
https://issues.apache.org/jira/browse/HBASE-24072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24072:
--
Release Note: Hadoop hosts have had their ulimit -u raised from 1 to
3 (per user). The build container has had its limit raised from 1 to
12500.
> Nightlies reporting OutOfMemoryError: unable to create new native thread
>
>
> Key: HBASE-24072
> URL: https://issues.apache.org/jira/browse/HBASE-24072
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24072-Nightlies-reporting-OutOfMemoryError-una.patch,
> print_ulimit.patch
>
>
> Seeing this kind of thing in nightly...
> {code}
> java.lang.RuntimeException: java.lang.OutOfMemoryError: unable to create new
> native thread
> at
> org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper.beforeClass(TestMultithreadedTableMapper.java:83)
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
> at
> org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper.beforeClass(TestMultithreadedTableMapper.java:83)
> {code}
> Chatting w/ Nick and Huaxiang, doing the math, we are likely oversubscribing
> our docker container. It is set to 20G (The hosts are 48G). Fork count is
> 0.5C on a 16 CPU machine which is 8 *2.8G our current forked jvm size. Add
> the maven 4G and we could be over the top.
> Play w/ downing the fork size (in earlier study we didn't seem to need this
> much RAM when running a fat long test). Let me also take th ms off the mvn
> allocation to see if that helps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450#issuecomment-610525749 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450#issuecomment-610525831 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450#issuecomment-610525832 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450#issuecomment-610526618 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | | -0 :warning: | shellcheck | 0m 2s | The patch generated 1 new + 9 unchanged - 1 fixed = 10 total (was 10) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1450 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 3a17135ae434 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 80d4a09ddc | | shellcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/artifact/yetus-general-check/output/diff-patch-shellcheck.txt | | Max. process+thread count | 47 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450#issuecomment-610526938 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1450 | | Optional Tests | | | uname | Linux 7f68ff9e6500 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 80d4a09ddc | | Max. process+thread count | 52 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Apache-HBase commented on issue #1450: HBASE-24126 Up the container nproc uplimit from 1 to 12500 URL: https://github.com/apache/hbase/pull/1450#issuecomment-610527388 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 56s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 44s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 3m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1450 | | Optional Tests | | | uname | Linux 0d85475155a2 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 80d4a09ddc | | Max. process+thread count | 47 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1450/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1439: HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be
bharathv commented on a change in pull request #1439: HBASE-24121
[Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it
should be
URL: https://github.com/apache/hbase/pull/1439#discussion_r405024401
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
##
@@ -313,6 +315,10 @@ public void onConfigurationChange(Configuration newConf) {
if (scheduler instanceof ConfigurationObserver) {
((ConfigurationObserver) scheduler).onConfigurationChange(newConf);
}
+synchronized (authManager) {
Review comment:
nit: Add a quick log that we are re-setting the authManager state, probably
helps with debugging races later if any..
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (HBASE-24134) Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11
Michael Stack created HBASE-24134: - Summary: Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11 Key: HBASE-24134 URL: https://issues.apache.org/jira/browse/HBASE-24134 Project: HBase Issue Type: Sub-task Reporter: Michael Stack Docker container we run tests in is hardcoded 20G. Our fork count is 0.25C currently. The jenkins machines are 16CPUs. 0.25C means 4CPUs so 4 forks running at any one time. Our current forked JVM heap size is 2800m if jdk8 or 3200m if jdk11. If we want to move our fork count to 0.5C, then that means 8 forks possibly running at once: i.e. 8 * 2800 = 22.4G... i.e. too big for the container. This issue is about running w/ smaller forked jvm heap size. If 2200m then, its 19.6G which is cutting it close but might allow us sqeeze through (mvn takes 4G!). Lets see. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23779) Up the default fork count to make builds complete faster; make count relative to CPU count
[ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076667#comment-17076667 ] Michael Stack edited comment on HBASE-23779 at 4/7/20, 6:34 PM: Reopening. Was resolved because thought there no elbow room here but have been making progress on this master project off in side issues. Let me bring the work together back here under this issue. High-level, trying to run w/ more parallelism -- forkcount and the mvn -T -- we ran into limits. Experiments trying to up our forkcount to 0.5C (8 cores on apache jenkins) ran into container memory (hard-coded at 20G, box is 48G w/ two executors per box so 24G each... See HBASE-24134) and host/user ulimit upper-bounds (per [~bharathv] in his comments above ... see HBASE-24126). Returning fork count down to 0.25C re-stabilized builds (along w/ test fixes). Minimally would at least like to add -T0.25C to maven before resolving this issue again. Maximally, would like to get build running on jenkins at 0.5C (In local tests build takes twice as long if forkcount+ T arg are halved... say from 0.5C to 0.25C). = In local tests, I can run with 1.0C. On a 40CPU linux machine, it is a bit wobbly. It was wobbly on big GCE instances but I should revisit. On a 16CPU mac, it seems fine as on a 4CPU VM. For Jenkins, 0.5C might be doable. Lets try again now we know more. was (Author: stack): Reopening. Was resolved because thought there no elbow room here but have been making progress on this master project off in side issues. Let me bring the work together back here under this issue. High-level, trying to run w/ more parallelism -- forkcount and the mvn -T -- we ran into limits. Experiments trying to up our forkcount to 0.5C (8 cores on apache jenkins) ran into container memory and host/user ulimit upper-bounds. Returning fork count down to 0.25C re-stabilized builds (along w/ test fixes). Minimally would at least like to add -T0.25C to maven before resolving this issue again. Maximally, would like to get build running on jenkins at 0.5C (In local tests build takes twice as long if forkcount+ T arg are halved... say from 0.5C to 0.25C). In local tests, I can run with 1.0C. On a 40CPU linux machine, it is a bit wobbly. It was wobbly on big GCE instances but I should revisit. On a 16CPU mac, it seems fine as on a 4CPU VM. For Jenkins, 0.5C might be doable. Lets try again now we know more. > Up the default fork count to make builds complete faster; make count relative > to CPU count > -- > > Key: HBASE-23779 > URL: https://issues.apache.org/jira/browse/HBASE-23779 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: addendum2.patch, test_yetus_934.0.patch > > > Tests take a long time. Our fork count running all tests are conservative -- > 1 (small) for first part and 5 for second part (medium and large). Rather > than hardcoding we should set the fork count to be relative to machine size. > Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box. > Looking up at jenkins, it seems like the boxes are 24 cores... at least going > by my random survey. The load reported on a few seems low though this not > representative (looking at machine/uptime). > More parallelism willl probably mean more test failure. Let me take a look > see. -- This message was sent by Atlassian Jira (v8.3.4#803005)
