[GitHub] [hbase] Apache-HBase commented on pull request #3533: HBASE-26144 The HStore.snapshot method is never called in main code
Apache-HBase commented on pull request #3533: URL: https://github.com/apache/hbase/pull/3533#issuecomment-25617 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 57s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | +1 :green_heart: | shadedjars | 8m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 40s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 8m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 185m 8s | hbase-server in the patch passed. | | | | 219m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3533 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux dec9d1bf2b1e 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/3/testReport/ | | Max. process+thread count | 3918 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3531: HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.msl…
Apache-HBase commented on pull request #3531: URL: https://github.com/apache/hbase/pull/3531#issuecomment-25331 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 38s | master passed | | +1 :green_heart: | compile | 1m 16s | master passed | | +1 :green_heart: | shadedjars | 9m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 8m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 143m 2s | hbase-server in the patch failed. | | | | 176m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 780013676d6b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/testReport/ | | Max. process+thread count | 4178 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3460: HBASE-26064 Introduce a StoreFileTracker to abstract the stor…
Apache-HBase commented on pull request #3460: URL: https://github.com/apache/hbase/pull/3460#issuecomment-16264 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | master passed | | +1 :green_heart: | compile | 4m 18s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | spotbugs | 2m 8s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 3m 6s | the patch passed | | +1 :green_heart: | javac | 3m 6s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 3 new + 144 unchanged - 15 fixed = 147 total (was 159) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 51s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | -1 :x: | spotbugs | 2m 21s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 50m 35s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.regionserver.HStore.close() does not release lock on all exception paths At HStore.java:on all exception paths At HStore.java:[line 718] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/12/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3460 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux b4382669fac8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/12/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/12/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/12/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26114) when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
[
https://issues.apache.org/jira/browse/HBASE-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389266#comment-17389266
]
Anoop Sam John commented on HBASE-26114:
We use PR flow to merge changes. I can see u raised one against branch-2.3
Can u pls raise one against master.
IMO we can fix this in master and branch-2 . No need for patch branch fixes as
this is a minor one.
> when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster
> cannot start normally
> ---
>
> Key: HBASE-26114
> URL: https://issues.apache.org/jira/browse/HBASE-26114
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.2.0, 2.4.4
> Environment: HBase 2.2.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_191
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Priority: Minor
> Labels: patch
> Attachments: HBASE-26114.000.patch
>
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> In hbase-default.xml：
>
> {code:java}
>
> hbase.mob.compaction.threads.max
> 1
>
> The max number of threads used in MobCompactor.
>
> {code}
>
> When the value is set to a negative number, such as -1, Hmaster cannot start
> normally.
> The log file will output:
>
> {code:cpp}
> 2021-07-22 18:54:13,758 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.
> (MobCompactionChore.java:51) at
> org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Master server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: * ABORTING master javafuzz,16000,1626951243154: Unhandled
> exception. Starting shutdown. * java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
>
> at
> org.apache.hadoop.hbase.master.MobCompactionChore.(MobCompactionChore.java:51)
>
> at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
>
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-
> 22 18:54:13,760 INFO [master/JavaFuzz:16000:becomeActiveMaster]
> regionserver.HRegionServer: * STOPPING region server
> 'javafuzz,16000,1626951243154' *{code}
>
> In MobUtils.java(package org.apache.hadoop.hbase.mob)
> This method from version 2.2.0 to version 2.4.4 is the same
> {code:java}
> public static ExecutorService createMobCompactorThreadPool(Configuration
> conf) { int maxThreads =
> conf.getInt(MobConstants.MOB_COMPACTION_THREADS_MAX,
> MobConstants.DEFAULT_MOB_COMPACTION_THREADS_MAX);
> if (maxThreads == 0) {
>maxThreads = 1;
> }
> final SynchronousQueue queue = new SynchronousQueue<>();
> ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads, 60,
> TimeUnit.SECONDS, queue,
> Threads.newDaemonThreadFactory("MobCompactor"), new
> RejectedExecutionHandler() {
> @Override
> public void rejectedExecution(Runnable r, ThreadPoolExecutor
> executor) {
> try {
> // waiting for a thread to pick up instead of throwing
> exceptions.
> queue.put(r);
> } catch (InterruptedException e) {
> throw new RejectedExecutionException(e);
> }
> }
> });
>
[jira] [Updated] (HBASE-26114) when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
[
https://issues.apache.org/jira/browse/HBASE-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-26114:
---
Fix Version/s: 3.0.0-alpha-2
2.5.0
> when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster
> cannot start normally
> ---
>
> Key: HBASE-26114
> URL: https://issues.apache.org/jira/browse/HBASE-26114
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.2.0, 2.4.4
> Environment: HBase 2.2.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_191
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Priority: Minor
> Labels: patch
> Fix For: 2.5.0, 3.0.0-alpha-2
>
> Attachments: HBASE-26114.000.patch
>
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> In hbase-default.xml：
>
> {code:java}
>
> hbase.mob.compaction.threads.max
> 1
>
> The max number of threads used in MobCompactor.
>
> {code}
>
> When the value is set to a negative number, such as -1, Hmaster cannot start
> normally.
> The log file will output:
>
> {code:cpp}
> 2021-07-22 18:54:13,758 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.
> (MobCompactionChore.java:51) at
> org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Master server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: * ABORTING master javafuzz,16000,1626951243154: Unhandled
> exception. Starting shutdown. * java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
>
> at
> org.apache.hadoop.hbase.master.MobCompactionChore.(MobCompactionChore.java:51)
>
> at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
>
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-
> 22 18:54:13,760 INFO [master/JavaFuzz:16000:becomeActiveMaster]
> regionserver.HRegionServer: * STOPPING region server
> 'javafuzz,16000,1626951243154' *{code}
>
> In MobUtils.java(package org.apache.hadoop.hbase.mob)
> This method from version 2.2.0 to version 2.4.4 is the same
> {code:java}
> public static ExecutorService createMobCompactorThreadPool(Configuration
> conf) { int maxThreads =
> conf.getInt(MobConstants.MOB_COMPACTION_THREADS_MAX,
> MobConstants.DEFAULT_MOB_COMPACTION_THREADS_MAX);
> if (maxThreads == 0) {
>maxThreads = 1;
> }
> final SynchronousQueue queue = new SynchronousQueue<>();
> ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads, 60,
> TimeUnit.SECONDS, queue,
> Threads.newDaemonThreadFactory("MobCompactor"), new
> RejectedExecutionHandler() {
> @Override
> public void rejectedExecution(Runnable r, ThreadPoolExecutor
> executor) {
> try {
> // waiting for a thread to pick up instead of throwing
> exceptions.
> queue.put(r);
> } catch (InterruptedException e) {
> throw new RejectedExecutionException(e);
> }
> }
> });
> ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
> return pool;
>}{code}
> When MOB_COMPACTION_THREADS_MAX is set to 0, mobUtil
[jira] [Assigned] (HBASE-26114) when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
[
https://issues.apache.org/jira/browse/HBASE-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John reassigned HBASE-26114:
--
Assignee: (was: Anoop Sam John)
> when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster
> cannot start normally
> ---
>
> Key: HBASE-26114
> URL: https://issues.apache.org/jira/browse/HBASE-26114
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.2.0, 2.4.4
> Environment: HBase 2.2.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_191
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Priority: Minor
> Labels: patch
> Attachments: HBASE-26114.000.patch
>
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> In hbase-default.xml：
>
> {code:java}
>
> hbase.mob.compaction.threads.max
> 1
>
> The max number of threads used in MobCompactor.
>
> {code}
>
> When the value is set to a negative number, such as -1, Hmaster cannot start
> normally.
> The log file will output:
>
> {code:cpp}
> 2021-07-22 18:54:13,758 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.
> (MobCompactionChore.java:51) at
> org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Master server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: * ABORTING master javafuzz,16000,1626951243154: Unhandled
> exception. Starting shutdown. * java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
>
> at
> org.apache.hadoop.hbase.master.MobCompactionChore.(MobCompactionChore.java:51)
>
> at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
>
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-
> 22 18:54:13,760 INFO [master/JavaFuzz:16000:becomeActiveMaster]
> regionserver.HRegionServer: * STOPPING region server
> 'javafuzz,16000,1626951243154' *{code}
>
> In MobUtils.java(package org.apache.hadoop.hbase.mob)
> This method from version 2.2.0 to version 2.4.4 is the same
> {code:java}
> public static ExecutorService createMobCompactorThreadPool(Configuration
> conf) { int maxThreads =
> conf.getInt(MobConstants.MOB_COMPACTION_THREADS_MAX,
> MobConstants.DEFAULT_MOB_COMPACTION_THREADS_MAX);
> if (maxThreads == 0) {
>maxThreads = 1;
> }
> final SynchronousQueue queue = new SynchronousQueue<>();
> ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads, 60,
> TimeUnit.SECONDS, queue,
> Threads.newDaemonThreadFactory("MobCompactor"), new
> RejectedExecutionHandler() {
> @Override
> public void rejectedExecution(Runnable r, ThreadPoolExecutor
> executor) {
> try {
> // waiting for a thread to pick up instead of throwing
> exceptions.
> queue.put(r);
> } catch (InterruptedException e) {
> throw new RejectedExecutionException(e);
> }
> }
> });
> ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
> return pool;
>}{code}
> When MOB_COMPACTION_THREADS_MAX is set to 0, mobUtil will set it to 1. But
> the program does not take into
[jira] [Assigned] (HBASE-26114) when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
[
https://issues.apache.org/jira/browse/HBASE-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John reassigned HBASE-26114:
--
Assignee: Anoop Sam John
> when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster
> cannot start normally
> ---
>
> Key: HBASE-26114
> URL: https://issues.apache.org/jira/browse/HBASE-26114
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.2.0, 2.4.4
> Environment: HBase 2.2.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_191
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Assignee: Anoop Sam John
>Priority: Minor
> Labels: patch
> Attachments: HBASE-26114.000.patch
>
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> In hbase-default.xml：
>
> {code:java}
>
> hbase.mob.compaction.threads.max
> 1
>
> The max number of threads used in MobCompactor.
>
> {code}
>
> When the value is set to a negative number, such as -1, Hmaster cannot start
> normally.
> The log file will output:
>
> {code:cpp}
> 2021-07-22 18:54:13,758 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.
> (MobCompactionChore.java:51) at
> org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Master server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: * ABORTING master javafuzz,16000,1626951243154: Unhandled
> exception. Starting shutdown. * java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
>
> at
> org.apache.hadoop.hbase.master.MobCompactionChore.(MobCompactionChore.java:51)
>
> at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
>
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-
> 22 18:54:13,760 INFO [master/JavaFuzz:16000:becomeActiveMaster]
> regionserver.HRegionServer: * STOPPING region server
> 'javafuzz,16000,1626951243154' *{code}
>
> In MobUtils.java(package org.apache.hadoop.hbase.mob)
> This method from version 2.2.0 to version 2.4.4 is the same
> {code:java}
> public static ExecutorService createMobCompactorThreadPool(Configuration
> conf) { int maxThreads =
> conf.getInt(MobConstants.MOB_COMPACTION_THREADS_MAX,
> MobConstants.DEFAULT_MOB_COMPACTION_THREADS_MAX);
> if (maxThreads == 0) {
>maxThreads = 1;
> }
> final SynchronousQueue queue = new SynchronousQueue<>();
> ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads, 60,
> TimeUnit.SECONDS, queue,
> Threads.newDaemonThreadFactory("MobCompactor"), new
> RejectedExecutionHandler() {
> @Override
> public void rejectedExecution(Runnable r, ThreadPoolExecutor
> executor) {
> try {
> // waiting for a thread to pick up instead of throwing
> exceptions.
> queue.put(r);
> } catch (InterruptedException e) {
> throw new RejectedExecutionException(e);
> }
> }
> });
> ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
> return pool;
>}{code}
> When MOB_COMPACTION_THREADS_MAX is set to 0, mobUtil will set it to 1. But
> the
[GitHub] [hbase] Apache-HBase commented on pull request #3539: [HBASE-26114][master]when “hbase.mob.compaction.threads.max” is set t…
Apache-HBase commented on pull request #3539: URL: https://github.com/apache/hbase/pull/3539#issuecomment-11733 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 5s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 5m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 137m 28s | hbase-server in the patch failed. | | | | 164m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3539/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3539 | | JIRA Issue | HBASE-26114 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a5ef29fd757e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / fd1a3fcf7b | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3539/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3539/1/testReport/ | | Max. process+thread count | 4026 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3539/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-06476 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 2m 19s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 48s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | shadedjars | 6m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 331m 41s | root in the patch passed. | | | | 373m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d469db62d4d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/testReport/ | | Max. process+thread count | 4304 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3460: HBASE-26064 Introduce a StoreFileTracker to abstract the stor…
Apache9 commented on pull request #3460: URL: https://github.com/apache/hbase/pull/3460#issuecomment-02510 @saintstack @wchevreuil @z-york I've modified the patch. PTAL. Now most storefile relate operations are moved to StoreEngine, and we will not expose StoreFileTracker to upper layer any more. In HStore we just need to call the methods in StoreEngine, and inside StoreEngine we will update StoreFileTracker and then StoreFileManager. The store lock is also moved into StoreEngine to allow we do fine-grained lock in StoreEngine. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3540: HBASE-26098 Support passing a customized Configuration object when cr…
Apache-HBase commented on pull request #3540: URL: https://github.com/apache/hbase/pull/3540#issuecomment-888793402 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 57s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 55s | master passed | | +1 :green_heart: | compile | 0m 24s | master passed | | +1 :green_heart: | shadedjars | 10m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 33s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | shadedjars | 10m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 39s | hbase-testing-util in the patch passed. | | | | 36m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3540 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d356f1a57317 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/testReport/ | | Max. process+thread count | 1219 (vs. ulimit of 3) | | modules | C: hbase-testing-util U: hbase-testing-util | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3540: HBASE-26098 Support passing a customized Configuration object when cr…
Apache-HBase commented on pull request #3540: URL: https://github.com/apache/hbase/pull/3540#issuecomment-888785246 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 51s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 0m 22s | master passed | | +1 :green_heart: | shadedjars | 8m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | +1 :green_heart: | shadedjars | 8m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 34s | hbase-testing-util in the patch passed. | | | | 31m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3540 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 45efda0f9a58 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/testReport/ | | Max. process+thread count | 1225 (vs. ulimit of 3) | | modules | C: hbase-testing-util U: hbase-testing-util | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1673: HBASE-24339 Backport HBASE-23968 to branch-1
Apache-HBase commented on pull request #1673: URL: https://github.com/apache/hbase/pull/1673#issuecomment-888782510 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 13s | https://github.com/apache/hbase/pull/1673 does not apply to branch-1. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/1673 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1673/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3531: HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.msl…
Apache-HBase commented on pull request #3531: URL: https://github.com/apache/hbase/pull/3531#issuecomment-888781742 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 55s | master passed | | +1 :green_heart: | compile | 1m 13s | master passed | | +1 :green_heart: | shadedjars | 10m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 45s | the patch passed | | +1 :green_heart: | compile | 1m 30s | the patch passed | | +1 :green_heart: | javac | 1m 30s | the patch passed | | +1 :green_heart: | shadedjars | 12m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 17m 35s | hbase-server in the patch failed. | | | | 59m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 076f75ad9405 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/testReport/ | | Max. process+thread count | 780 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3540: HBASE-26098 Support passing a customized Configuration object when cr…
Apache-HBase commented on pull request #3540: URL: https://github.com/apache/hbase/pull/3540#issuecomment-888781643 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | master passed | | +1 :green_heart: | compile | 0m 34s | master passed | | +1 :green_heart: | checkstyle | 0m 16s | master passed | | +1 :green_heart: | spotbugs | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 37m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3540 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 0cd5a7fe5af2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-testing-util U: hbase-testing-util | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3540/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888781325 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 34s | branch-2 passed | | +1 :green_heart: | compile | 3m 7s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 55s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 3m 6s | the patch passed | | +1 :green_heart: | javac | 3m 6s | the patch passed | | +1 :green_heart: | shadedjars | 8m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 252m 56s | root in the patch passed. | | | | 303m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6edc7ad63877 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/testReport/ | | Max. process+thread count | 3994 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3531: HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.msl…
Apache-HBase commented on pull request #3531: URL: https://github.com/apache/hbase/pull/3531#issuecomment-888780528 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 38s | master passed | | +1 :green_heart: | compile | 3m 55s | master passed | | +1 :green_heart: | checkstyle | 1m 11s | master passed | | +1 :green_heart: | spotbugs | 2m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 5s | the patch passed | | +1 :green_heart: | compile | 3m 37s | the patch passed | | +1 :green_heart: | javac | 3m 37s | the patch passed | | +1 :green_heart: | checkstyle | 1m 14s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 42s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 55m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3531 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux f2ff44830038 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3539: [HBASE-26114][master]when “hbase.mob.compaction.threads.max” is set t…
Apache-HBase commented on pull request #3539: URL: https://github.com/apache/hbase/pull/3539#issuecomment-888770088 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | branch-2.3 passed | | +1 :green_heart: | compile | 3m 4s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 7s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 0s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 11s | the patch passed | | +1 :green_heart: | compile | 3m 5s | the patch passed | | +1 :green_heart: | javac | 3m 5s | the patch passed | | +1 :green_heart: | checkstyle | 1m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 23s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 45m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3539/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3539 | | JIRA Issue | HBASE-26114 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 02497d4e55d1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / fd1a3fcf7b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 95 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3539/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ddupg commented on pull request #3477: HBASE-26084 Add owner of replication queue for ReplicationQueueInfo
ddupg commented on pull request #3477: URL: https://github.com/apache/hbase/pull/3477#issuecomment-888768407 @Apache9 Thank you for reviewing. > What does the owner mean here? I think we have the region server name in the queue name on zk? The owner is the region server holding replication queue. And yes, the region server name is included in zk path. But replication server is first to get region server holding queue and queueid, and then use them to form the zk path of queue. For compatibility, replication server does not actually move the queue to its own zk directory, so replication server needs to know which zk directory of regionserver the queues that it consumes are in. On the regionserver, the holder and consumer of the queue are its own, so there is no need to record the owner extra, which is different for the replication server. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3533: HBASE-26144 The HStore.snapshot method is never called in main code
Apache-HBase commented on pull request #3533: URL: https://github.com/apache/hbase/pull/3533#issuecomment-888763936 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 1s | master passed | | +1 :green_heart: | compile | 3m 10s | master passed | | +1 :green_heart: | checkstyle | 1m 3s | master passed | | +1 :green_heart: | spotbugs | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | compile | 3m 8s | the patch passed | | +1 :green_heart: | javac | 3m 8s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | hbase-server: The patch generated 0 new + 35 unchanged - 6 fixed = 35 total (was 41) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 47m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3533 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8080f1ad1498 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5f0950558f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #3540: HBASE-26098 Support passing a customized Configuration object when cr…
Apache9 opened a new pull request #3540: URL: https://github.com/apache/hbase/pull/3540 …eating TestingHBaseCluster -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] comnetwork commented on a change in pull request #3533: HBASE-26144 The HStore.snapshot method is never called in main code
comnetwork commented on a change in pull request #3533:
URL: https://github.com/apache/hbase/pull/3533#discussion_r678787345
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
##
@@ -1251,6 +1234,9 @@ private boolean updateStorefiles(List sfs,
long snapshotId) throws I
// the lock.
this.lock.writeLock().unlock();
}
+if (snapshotId > 0) {
+ this.memstore.clearSnapshot(snapshotId);
Review comment:
Maybe we could add some comments here why we can clear snapshot outside
the writeLock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389189#comment-17389189 ] Mallikarjun edited comment on HBASE-26147 at 7/29/21, 2:41 AM: --- [~bbeaudreault] This feature is going to be super useful. This comment is not directly on feature as such. # Command name could be inline with existing balancer. May be `balancer 'dry_run'` or something similar to `force`. # Rsgroup support should also be considered in this PR. was (Author: rda3mon): [~bbeaudreault] This comment is not directly on feature as such. # Command name could be inline with existing balancer. May be `balancer 'dry_run'` or something similar to `force`. # Rsgroup support should also be considered in this PR. > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389189#comment-17389189 ] Mallikarjun commented on HBASE-26147: - [~bbeaudreault] This comment is not directly on feature as such. # Command name could be inline with existing balancer. May be `balancer 'dry_run'` or something similar to `force`. # Rsgroup support should also be considered in this PR. > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26114) when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally
[
https://issues.apache.org/jira/browse/HBASE-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HBASE-26114:
Fix Version/s: (was: 2.4.6)
(was: 2.2.8)
(was: 2.3.6)
> when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster
> cannot start normally
> ---
>
> Key: HBASE-26114
> URL: https://issues.apache.org/jira/browse/HBASE-26114
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.2.0, 2.4.4
> Environment: HBase 2.2.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_191
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Priority: Minor
> Labels: patch
> Attachments: HBASE-26114.000.patch
>
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> In hbase-default.xml：
>
> {code:java}
>
> hbase.mob.compaction.threads.max
> 1
>
> The max number of threads used in MobCompactor.
>
> {code}
>
> When the value is set to a negative number, such as -1, Hmaster cannot start
> normally.
> The log file will output:
>
> {code:cpp}
> 2021-07-22 18:54:13,758 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
> at org.apache.hadoop.hbase.master.MobCompactionChore.
> (MobCompactionChore.java:51) at
> org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: Master server abort: loaded coprocessors are:
> [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2021-07-22 18:54:13,760 ERROR [master/JavaFuzz:16000:becomeActiveMaster]
> master.HMaster: * ABORTING master javafuzz,16000,1626951243154: Unhandled
> exception. Starting shutdown. * java.lang.IllegalArgumentException
> at
> java.util.concurrent.ThreadPoolExecutor.(ThreadPoolExecutor.java:1314)
> at
> org.apache.hadoop.hbase.mob.MobUtils.createMobCompactorThreadPool(MobUtils.java:880)
>
> at
> org.apache.hadoop.hbase.master.MobCompactionChore.(MobCompactionChore.java:51)
>
> at org.apache.hadoop.hbase.master.HMaster.initMobCleaner(HMaster.java:1278)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1161)
>
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2112)
>
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:580)
> at java.lang.Thread.run(Thread.java:748)
> 2021-07-
> 22 18:54:13,760 INFO [master/JavaFuzz:16000:becomeActiveMaster]
> regionserver.HRegionServer: * STOPPING region server
> 'javafuzz,16000,1626951243154' *{code}
>
> In MobUtils.java(package org.apache.hadoop.hbase.mob)
> This method from version 2.2.0 to version 2.4.4 is the same
> {code:java}
> public static ExecutorService createMobCompactorThreadPool(Configuration
> conf) { int maxThreads =
> conf.getInt(MobConstants.MOB_COMPACTION_THREADS_MAX,
> MobConstants.DEFAULT_MOB_COMPACTION_THREADS_MAX);
> if (maxThreads == 0) {
>maxThreads = 1;
> }
> final SynchronousQueue queue = new SynchronousQueue<>();
> ThreadPoolExecutor pool = new ThreadPoolExecutor(1, maxThreads, 60,
> TimeUnit.SECONDS, queue,
> Threads.newDaemonThreadFactory("MobCompactor"), new
> RejectedExecutionHandler() {
> @Override
> public void rejectedExecution(Runnable r, ThreadPoolExecutor
> executor) {
> try {
> // waiting for a thread to pick up instead of throwing
> exceptions.
> queue.put(r);
> } catch (InterruptedException e) {
> throw new RejectedExecutionException(e);
> }
> }
> });
> ((ThreadPoolExecutor) pool).allowCoreThreadTimeOut(true);
> return pool;
>}{code}
> When MOB_COMPACTION_THREADS_MAX is set to 0, mobUtil
[GitHub] [hbase] liever18 opened a new pull request #3539: [HBASE-26114][master]when “hbase.mob.compaction.threads.max” is set t…
liever18 opened a new pull request #3539:
URL: https://github.com/apache/hbase/pull/3539
[https://issues.apache.org/jira/projects/HBASE/issues/HBASE-26114]
**Description of PR**
fix bug HBASE-26114
When the value is set to a negative number, such as -1, Hmaster cannot start
normally. When MOB_COMPACTION_THREADS_MAX is set to 0, mobUtil will set it to
1. But the program does not take into account that it is set to a negative
number.
**changge**
modified
if (maxThreads == 0) {
to
if (maxThreads <= 0) {
to improve the handling mechanism for the user to set
hbase.mob.compaction.threads.max as an abnormal value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26146) Allow custom opts for hbck in hbase bin
[
https://issues.apache.org/jira/browse/HBASE-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389185#comment-17389185
]
Hudson commented on HBASE-26146:
Results for branch branch-2.4
[build #170 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/170/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/170/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/170/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/170/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/170/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow custom opts for hbck in hbase bin
> ---
>
> Key: HBASE-26146
> URL: https://issues.apache.org/jira/browse/HBASE-26146
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.6
>
>
> https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
> execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
> to the command line. This is problematic in some cases depending on what
> regionserver opts have been set. For instance, one might configure a jmx port
> for the regionserver but then hbck will fail due to a port conflict if run on
> the same host as a regionserver. Another example would be that a regionserver
> might define an {{-Xms}} value which is significantly more than hbck requires.
>
> We should make it possible for users to define their own HBASE_HBCK_OPTS
> which take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26096) Cleanup the deprecated methods in HBTU related classes and format code
[ https://issues.apache.org/jira/browse/HBASE-26096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26096. --- Fix Version/s: 3.0.0-alpha-2 Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~haxiaolin] and [~niuyulin] for reviewing. > Cleanup the deprecated methods in HBTU related classes and format code > -- > > Key: HBASE-26096 > URL: https://issues.apache.org/jira/browse/HBASE-26096 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-2 > > > As it is now IA.LimitedPrivate, we should clean it up before the final 3.0.0 > release, to make the interface cleaner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #3503: HBASE-26096 Cleanup the deprecated methods in HBTU related classes and format code
Apache9 merged pull request #3503: URL: https://github.com/apache/hbase/pull/3503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3532: HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned
Apache-HBase commented on pull request #3532: URL: https://github.com/apache/hbase/pull/3532#issuecomment-888726920 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 3s | branch-2 passed | | +1 :green_heart: | compile | 3m 47s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 53s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 25s | the patch passed | | +1 :green_heart: | compile | 3m 33s | the patch passed | | +1 :green_heart: | javac | 3m 33s | the patch passed | | +1 :green_heart: | shadedjars | 8m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 0s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 35s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 2m 55s | hbase-client in the patch passed. | | -1 :x: | unit | 217m 50s | hbase-server in the patch failed. | | | | 267m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3532 | | JIRA Issue | HBASE-26122 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f2842ff80844 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/testReport/ | | Max. process+thread count | 2395 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-protocol hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389152#comment-17389152 ] Bryan Beaudreault commented on HBASE-26147: --- Thank you both for the comments. [~ndimiduk] that sounds great. I also wrote a separate java process which effectively ran the full balancer in a separate process and printed a bunch of info. It was useful for some balancer iteration I needed to do, but was super hacky so I eventually threw it away. I like the idea of combining dry run mode with a visualization tool. Dry run mode itself is useful for some cases (just seeing how many regions move and cost reduction), but in other cases you really need better data. I'd be interested to see how we can combine our two tools once they both land in master. [~claraxiong] Yea, I thought about that and love the idea. My current workflow is to edit hbase-site.xml and then use update_config to load it into the master, then run dry_run_balance, rinse and repeat. This is clunky and you need to remember to revert your hbase-site.xml changes and re-run update_config before re-enabling the balancer. I think a separate Jira could be added to allow dry_run_balance to take arguments for temporarily modifying the cost functions on the fly. Thanks for the pointer to 25973, I didn't have that one in my fork when I tested this. I've added it now. > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888707813 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 21s | branch-2 passed | | +1 :green_heart: | compile | 8m 50s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 23s | branch-2 passed | | +1 :green_heart: | spotbugs | 17m 28s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | compile | 8m 44s | the patch passed | | -0 :warning: | javac | 8m 44s | root generated 2 new + 1885 unchanged - 2 fixed = 1887 total (was 1887) | | +1 :green_heart: | checkstyle | 2m 18s | root: The patch generated 0 new + 736 unchanged - 27 fixed = 736 total (was 763) | | +1 :green_heart: | rubocop | 0m 4s | The patch generated 0 new + 3 unchanged - 8 fixed = 3 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 16s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 11m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 19m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 2m 56s | The patch does not generate ASF License warnings. | | | | 93m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml rubocop | | uname | Linux 09f3cf1d8289 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/artifact/yetus-general-check/output/diff-compile-javac-root.txt | | Max. process+thread count | 142 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/6/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389142#comment-17389142 ] Clara Xiong commented on HBASE-26147: - https://issues.apache.org/jira/browse/HBASE-25973?jql=project%20%3D%20HBASE%20AND%20text%20~%20balancer%20ORDER%20BY%20updated%20DESC provide a thorough breakdown of costs and comparison with minCostNeedBalance. > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389138#comment-17389138 ] Clara Xiong commented on HBASE-26147: - This could be very useful. I had been thinking about it for our tuning need on a large cluster with heavy load. Thank you. I wonder if it can be enhanced to take config changes to show what the run would be . > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389133#comment-17389133 ] Nick Dimiduk commented on HBASE-26147: -- Not a comment on this feature as such, but on the problem space. For various reasons, I recently had to implement a "manual" balancer external to the HBase master. It is implemented as a python script that scrapes the output of cluster status and calculates a balancer plan given our requirements. The interesting part, and maybe relevant to your interests, is the next bit. With a plan calculated, it then simulates application of that plan to the known cluster state. Finally it prints out charts showing the cluster status before and after the balance is applied. Specifically, stacked bar charts that are total store file size by table, by host. I then used these visualizations to tune the parameters of my manual balancer until I was satisfied with the result. I started implementing the visualization portion of the tool on HBASE-25865. So, IMHO, having a dry-run feature is very helpful if it can be used together with such a visualization tool for making sense of the current state and the new proposed target state. > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3532: HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned
Apache-HBase commented on pull request #3532: URL: https://github.com/apache/hbase/pull/3532#issuecomment-888693918 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 2m 34s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 30s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 17s | the patch passed | | +1 :green_heart: | compile | 2m 33s | the patch passed | | +1 :green_heart: | javac | 2m 33s | the patch passed | | +1 :green_heart: | shadedjars | 6m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 42s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 26s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 2m 45s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 142m 24s | hbase-server in the patch passed. | | | | 177m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3532 | | JIRA Issue | HBASE-26122 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a40c48a82da4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/testReport/ | | Max. process+thread count | 4223 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-protocol hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888670864 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 35s | branch-2 passed | | +1 :green_heart: | compile | 2m 14s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 40s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | shadedjars | 6m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 58s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 315m 56s | root in the patch failed. | | | | 357m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 041eca4f9473 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/testReport/ | | Max. process+thread count | 4533 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3532: HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned
Apache-HBase commented on pull request #3532: URL: https://github.com/apache/hbase/pull/3532#issuecomment-888650044 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 6m 45s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 25s | branch-2 passed | | +1 :green_heart: | spotbugs | 7m 58s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 33s | the patch passed | | +1 :green_heart: | compile | 6m 48s | the patch passed | | +1 :green_heart: | cc | 6m 48s | the patch passed | | +1 :green_heart: | javac | 6m 48s | the patch passed | | -0 :warning: | checkstyle | 1m 16s | hbase-server: The patch generated 5 new + 272 unchanged - 0 fixed = 277 total (was 272) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 16s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 37s | the patch passed | | +1 :green_heart: | spotbugs | 8m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 68m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3532 | | JIRA Issue | HBASE-26122 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux cfc3c72edc3e 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-protocol hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3532/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888645036 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 44s | branch-2 passed | | +1 :green_heart: | compile | 3m 10s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 7m 17s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | +1 :green_heart: | compile | 3m 13s | the patch passed | | +1 :green_heart: | javac | 3m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 7m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 244m 36s | root in the patch passed. | | | | 297m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b23128fdef28 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/testReport/ | | Max. process+thread count | 4029 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-25815) RSGroupBasedLoadBalancer online status never updates after being set to true for the first time
[ https://issues.apache.org/jira/browse/HBASE-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caroline Zhou reassigned HBASE-25815: - Assignee: Caroline Zhou > RSGroupBasedLoadBalancer online status never updates after being set to true > for the first time > --- > > Key: HBASE-25815 > URL: https://issues.apache.org/jira/browse/HBASE-25815 > Project: HBase > Issue Type: Bug >Reporter: Caroline Zhou >Assignee: Caroline Zhou >Priority: Minor > > Once the RSGroupBasedLoadBalancer is “online” (it has found the hbase:meta > and hbase:rsgroup tables), it will never update the status again. That means > if hbase:meta or hbase:rsgroup ever go offline, the balancer doesn’t update > its status to “offline,” so some of the code paths will go through the > “online” code path even though the catalog tables aren’t available to be read > from or written to (in particular, anything that calls > RSGroupInfoManagerImpl#flushConfig). > Also, in the RSGroupInfoManagerImpl#flushConfig code path, the call to write > to hbase:rsgroup comes before the update to the rsGroupMap and tableMap which > are stored in memory (see order of [these lines of > code|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java#L664-L670]), > so if hbase:rsgroup goes offline after the RSGroupBasedLoadBalancer is > already marked as “online,” exceptions thrown while trying to write to an > offline hbase:rsgroup table prevent the in-memory rsGroupMap and tableMap > from being updated. In terms of the order just mentioned, in-memory state > should be updated first. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[ https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389031#comment-17389031 ] Bryan Beaudreault commented on HBASE-26147: --- There is probably more work we could do to make dry run even more useful in the future, for example we could output what regions would have moved. I've been using dry run internally for a while and honestly knowing the moves is not super useful, especially in large clusters where a balance could mean hundreds of moves. What _would_ probably be useful would be to print the full cost breakdown of the chosen plan, or the full cost breakdown of the best-but-not-chosen plan in the case where no plan was good enough. That would be useful even in normal mode honestly, so I would prefer to make those sorts of improvements in a separate issue. Just having the ability to run in dry run mode is a great first step. > Add dry run mode to hbase balancer > -- > > Key: HBASE-26147 > URL: https://issues.apache.org/jira/browse/HBASE-26147 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > It's often rather hard to know how the cost function changes you're making > will affect the balance of the cluster, and currently the only way to know is > to run it. If the cost decisions are not good, you may have just moved many > regions towards a non-ideal balance. Region moves themselves are not free for > clients, and the resulting balance may cause a regression. > We should add a mode to the balancer so that it can be invoked without > actually executing any plans. This will allow an administrator to iterate on > their cost functions and used the balancer's logging to see how their changes > would affect the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26147) Add dry run mode to hbase balancer
[
https://issues.apache.org/jira/browse/HBASE-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389028#comment-17389028
]
Bryan Beaudreault commented on HBASE-26147:
---
Here's an example of running the balancer in dry run mode using the hbase shell.
*{{Initial status -- cluster is well balanced}}*
{{hbase:003:0> status 'simple'}}
{{active master: master1:6 1627497753352}}
{{1 backup masters}}
{{ master2:6 1627497808648}}
{{4 live servers}}
{{ regionserver1:60020 1627500965175}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=3}}
{{ regionserver2:60020 1627498037868}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=3, snip...}}
{{ regionserver3:60020 1627498045321}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=4, snip...}}
{{ regionserver4:60020 1627498059869}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=3, snip...}}
{{0 dead servers}}
{{Aggregate load: 0, regions: 13}}
{{Took 0.0667 seconds}}
*{{Turn off balancer so it doesn't mess with test}}*
{{hbase:004:0> balance_switch 'false'}}
{{Previous balancer state : true}}
{{Took 0.0208 seconds}}
{{=> true}}
*{{Force an imbalance on the cluster}}*
{{hbase:005:0> move '95c3921b4757077f16b1804278222072',
'regionserver2,60020,1627498037868'}}
{{Took 1.0711 seconds}}
{{hbase:006:0> move '45db520388ef4f427ba4718e3103585e',
'regionserver2,60020,1627498037868'}}
{{Took 1.0316 seconds}}
{{hbase:007:0> move '75ff1c829866bee6cced10e2b603a1cd',
'regionserver2,60020,1627498037868'}}
{{Took 1.0398 seconds}}
*{{Check status -- regionserver2 has too many regions}}*
{{hbase:008:0> status 'simple'}}
{{active master: master1:6 1627497753352}}
{{1 backup masters}}
{{ master2:6 1627497808648}}
{{4 live servers}}
{{ regionserver1:60020 1627500965175}}
{{ requestsPerSecond=1.0, numberOfOnlineRegions=3, snip...}}
{{ regionserver2:60020 1627498037868}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=6, snip...}}
{{ regionserver3:60020 1627498045321}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=3, snip...}}
{{ regionserver4:60020 1627498059869}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=1, snip...}}
{{0 dead servers}}
{{Aggregate load: 1, regions: 13}}
{{Took 0.0110 seconds}}
*{{Run balancer in dry run mode}}*
{{hbase:009:0> dry_run_balancer}}
{{true}}
{{Took 0.1302 seconds}}
{{=> true}}
*{{Check logs on active master, balancer would have moved 2 regions}}*
{{2021-07-28 20:01:27,254 INFO
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer: Start Generate
Balance plan for cluster.}}
{{2021-07-28 20:01:27,257 DEBUG
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer:
RegionCountSkewCostFunction sees a total of 4 servers and 13 regions.}}
{{2021-07-28 20:01:27,257 DEBUG
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: We need to load
balance cluster; total cost=141.75213675213675, sum multiplier=685.0;
cost/multiplier to need a balance is 0.05}}
{{2021-07-28 20:01:27,258 INFO
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: start
StochasticLoadBalancer.balancer, initCost=141.75213675213675,
functionCost=RegionCountSkewCostFunction : (500.0, 0.);
PrimaryRegionCountSkewCostFunction : (500.0, 0.0); MoveCostFunction :}}
{{ (100.0, 0.0); ServerLocalityCostFunction : (25.0, 0.0);
TableSkewCostFunction : (35.0, 0.5897435897435898);
RegionReplicaHostCostFunction : (10.0, 0.0); RegionReplicaRackCostFunction
: (1.0, 0.0); ReadRequestCostFunction : (10.0, 1.0);
WriteRequestCostFunction : (5.0, 0.0); MemStoreSizeCostFunction : (5.0, 0.0);
StoreFileCostFunction : (5.0, 0.0); comp}}
{{utedMaxSteps: 41600}}
{{2021-07-28 20:01:27,380 INFO
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Finished
computing new load balance plan. Computation took PT0.122S to try 41600
different iterations. Found a solution that moves 2 regions; Going from a
computed cost of 141.75213675213675 to a new cost of}}
{{38.84615384615385}}
*{{Check status -- cluster is still imbalanced, dry run mode took no actions}}*
{{hbase:010:0> status 'simple'}}
{{active master: master1:6 1627497753352}}
{{1 backup masters}}
{{ master2:6 1627497808648}}
{{4 live servers}}
{{ regionserver1:60020 1627500965175}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=3, snip...}}
{{ regionserver2:60020 1627498037868}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=6, snip...}}
{{ regionserver3:60020 1627498045321}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=3, snip...}}
{{ regionserver4:60020 1627498059869}}
{{ requestsPerSecond=0.0, numberOfOnlineRegions=1, snip...}}
{{0 dead servers}}
{{Aggregate load: 0, regions: 13}}
{{Took 0.0135 seconds}}
*{{Re-enable balancer}}*
{{hbase:011:0> balance_switch 'true'}}
{{Previous balancer state : false}}
{{Took 0.0095 seconds}}
{{=> false}}
*{{Balance the cluster}}*
{{hbase:012:0> balancer}}
{{true}}
{{Took 1.3375
[GitHub] [hbase] Apache-HBase commented on pull request #3536: HBASE-26147: Add dry_run_balancer and related Admin interfaces for running the balancer without executing any region moves
Apache-HBase commented on pull request #3536: URL: https://github.com/apache/hbase/pull/3536#issuecomment-888588749 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 41s | branch-2 passed | | +1 :green_heart: | compile | 3m 16s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 16s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 12s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 39s | the patch passed | | +1 :green_heart: | compile | 3m 17s | the patch passed | | +1 :green_heart: | javac | 3m 17s | the patch passed | | +1 :green_heart: | shadedjars | 6m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 43s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 47s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 156m 21s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 6m 37s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 17s | hbase-shell in the patch passed. | | | | 209m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3536 | | JIRA Issue | HBASE-26147 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d04e8c71b767 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/testReport/ | | Max. process+thread count | 3456 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3536: HBASE-26147: Add dry_run_balancer and related Admin interfaces for running the balancer without executing any region moves
Apache-HBase commented on pull request #3536: URL: https://github.com/apache/hbase/pull/3536#issuecomment-888576286 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 16s | branch-2 passed | | +1 :green_heart: | compile | 3m 27s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 28s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 3m 27s | the patch passed | | +1 :green_heart: | javac | 3m 27s | the patch passed | | +1 :green_heart: | shadedjars | 6m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 52s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 52s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 132m 59s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 6m 47s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 8s | hbase-shell in the patch passed. | | | | 188m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3536 | | JIRA Issue | HBASE-26147 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 434d45fa8f3d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/testReport/ | | Max. process+thread count | 4124 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-26122) Limit max result size of individual Gets
[
https://issues.apache.org/jira/browse/HBASE-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388971#comment-17388971
]
Bryan Beaudreault edited comment on HBASE-26122 at 7/28/21, 7:10 PM:
-
Here's a quick example of this working in the hbase shell:
*Create table*
{{hbase:005:0> create 't1', 'f1'}}
{{Created table t1}}
{{Took 1.1306 seconds}}
*{{Insert test data}}*
{{hbase:012:0> put 't1', 'r1', 'f1:c1', 'a'}}
{{Took 0.0416 seconds}}
{{hbase:014:0> put 't1', 'r1', 'f1:c2', 'b'}}
{{Took 0.0059 seconds}}
{{hbase:015:0> put 't1', 'r1', 'f1:c3', 'c'}}
{{Took 0.0097 seconds}}
*Get without setMaxResultSize, returns full row and {{mayHaveMoreCellsInRow =
false}}*
{{hbase:037:0> g = Get.new('r1'.to_s.to_java_bytes)}}
{{=> #}}
{{hbase:038:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:039:0> result.mayHaveMoreCellsInRow}}
{{=> false}}
{{hbase:040:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0,
r1/f1:c3/1627498280413/Put/vlen=1/seqid=0}"}}
*{{Get with setMaxResultSize, returns first two columns and
}}{{mayHaveMoreCellsInRow = true}}*
{{hbase:059:0> g = Get.new('r1'.to_s.to_java_bytes).setMaxResultSize(100)}}
{{=> #}}
{{hbase:060:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:061:0> result.mayHaveMoreCellsInRow}}
{{=> true}}
{{hbase:062:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0}"}}
was (Author: bbeaudreault):
Here's a quick example of this working in the hbase shell:
*Create table*
{{hbase:005:0> create 't1', 'f1'}}
{{Created table t1}}
{{Took 1.1306 seconds}}
*{{Insert test data}}*
{{hbase:012:0> put 't1', 'r1', 'f1:c1', 'a'}}
{{Took 0.0416 seconds}}
{{hbase:014:0> put 't1', 'r1', 'f1:c2', 'b'}}
{{Took 0.0059 seconds}}
{{hbase:015:0> put 't1', 'r1', 'f1:c3', 'c'}}
{{Took 0.0097 seconds}}
*Get without setMaxResultSize, returns full row and {{mayHaveMoreCellsInRow =
false}}*{{}}
{{hbase:037:0> g = Get.new('r1'.to_s.to_java_bytes)}}
{{=> #}}
{{hbase:038:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:039:0> result.mayHaveMoreCellsInRow}}
{{=> false}}
{{hbase:040:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0,
r1/f1:c3/1627498280413/Put/vlen=1/seqid=0}"}}
*{{Get with setMaxResultSize, returns first two columns and
}}{{mayHaveMoreCellsInRow = true}}*{{}}
{{hbase:059:0> g = Get.new('r1'.to_s.to_java_bytes).setMaxResultSize(100)}}
{{=> #}}
{{hbase:060:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:061:0> result.mayHaveMoreCellsInRow}}
{{=> true}}
{{hbase:062:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0}"}}
> Limit max result size of individual Gets
>
>
> Key: HBASE-26122
> URL: https://issues.apache.org/jira/browse/HBASE-26122
> Project: HBase
> Issue Type: New Feature
> Components: Client, regionserver
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>
> Scans have the ability to have a configured max result size, which causes
> them to return a partial result once the limit has been reached. MultiGets
> also can throw MultiActionResultTooLarge if the response size is over a
> configured quota. Neither of these really accounts for a single Get of a
> too-large row. Such too-large Gets can cause substantial GC pressure or worse
> if sent at volume.
> Currently one can work around this by converting their Get to a single row
> Scan, but this requires a developer to proactively know about and prepare for
> the issue by using a Scan upfront or wait for the RegionServer to choke on a
> large request and only then rewrite the Get for future requests.
> We should implement the same response size limits for for Get as for Scan,
> whereby the server returns a partial result to the client for handling.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-26122) Limit max result size of individual Gets
[
https://issues.apache.org/jira/browse/HBASE-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388971#comment-17388971
]
Bryan Beaudreault edited comment on HBASE-26122 at 7/28/21, 7:10 PM:
-
Here's a quick example of this working in the hbase shell:
*Create table*
{{hbase:005:0> create 't1', 'f1'}}
{{Created table t1}}
{{Took 1.1306 seconds}}
*{{Insert test data}}*
{{hbase:012:0> put 't1', 'r1', 'f1:c1', 'a'}}
{{Took 0.0416 seconds}}
{{hbase:014:0> put 't1', 'r1', 'f1:c2', 'b'}}
{{Took 0.0059 seconds}}
{{hbase:015:0> put 't1', 'r1', 'f1:c3', 'c'}}
{{Took 0.0097 seconds}}
*Get without setMaxResultSize, returns full row and {{mayHaveMoreCellsInRow =
false}}*{{}}
{{hbase:037:0> g = Get.new('r1'.to_s.to_java_bytes)}}
{{=> #}}
{{hbase:038:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:039:0> result.mayHaveMoreCellsInRow}}
{{=> false}}
{{hbase:040:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0,
r1/f1:c3/1627498280413/Put/vlen=1/seqid=0}"}}
*{{Get with setMaxResultSize, returns first two columns and
}}{{mayHaveMoreCellsInRow = true}}*{{}}
{{hbase:059:0> g = Get.new('r1'.to_s.to_java_bytes).setMaxResultSize(100)}}
{{=> #}}
{{hbase:060:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:061:0> result.mayHaveMoreCellsInRow}}
{{=> true}}
{{hbase:062:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0}"}}
was (Author: bbeaudreault):
Here's a quick example of this working in the hbase shell:
*Create table*
{{hbase:005:0> create 't1', 'f1', SPLITS => ['10', '20', '30', '40', '50',
'60', '70', '80']}}
{{Created table t1}}
{{Took 1.1306 seconds}}
*{{Insert test data}}*
{{hbase:012:0> put 't1', 'r1', 'f1:c1', 'a'}}
{{Took 0.0416 seconds}}
{{hbase:014:0> put 't1', 'r1', 'f1:c2', 'b'}}
{{Took 0.0059 seconds}}
{{hbase:015:0> put 't1', 'r1', 'f1:c3', 'c'}}
{{Took 0.0097 seconds}}
*Get without setMaxResultSize, returns full row and {{mayHaveMoreCellsInRow =
false}}*{{}}
{{hbase:037:0> g = Get.new('r1'.to_s.to_java_bytes)}}
{{=> #}}
{{hbase:038:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:039:0> result.mayHaveMoreCellsInRow}}
{{=> false}}
{{hbase:040:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0,
r1/f1:c3/1627498280413/Put/vlen=1/seqid=0}"}}
*{{Get with setMaxResultSize, returns first two columns and
}}{{mayHaveMoreCellsInRow = true}}*{{}}
{{hbase:059:0> g = Get.new('r1'.to_s.to_java_bytes).setMaxResultSize(100)}}
{{=> #}}
{{hbase:060:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:061:0> result.mayHaveMoreCellsInRow}}
{{=> true}}
{{hbase:062:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0}"}}
> Limit max result size of individual Gets
>
>
> Key: HBASE-26122
> URL: https://issues.apache.org/jira/browse/HBASE-26122
> Project: HBase
> Issue Type: New Feature
> Components: Client, regionserver
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>
> Scans have the ability to have a configured max result size, which causes
> them to return a partial result once the limit has been reached. MultiGets
> also can throw MultiActionResultTooLarge if the response size is over a
> configured quota. Neither of these really accounts for a single Get of a
> too-large row. Such too-large Gets can cause substantial GC pressure or worse
> if sent at volume.
> Currently one can work around this by converting their Get to a single row
> Scan, but this requires a developer to proactively know about and prepare for
> the issue by using a Scan upfront or wait for the RegionServer to choke on a
> large request and only then rewrite the Get for future requests.
> We should implement the same response size limits for for Get as for Scan,
> whereby the server returns a partial result to the client for handling.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26122) Limit max result size of individual Gets
[
https://issues.apache.org/jira/browse/HBASE-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388971#comment-17388971
]
Bryan Beaudreault commented on HBASE-26122:
---
Here's a quick example of this working in the hbase shell:
*Create table*
{{hbase:005:0> create 't1', 'f1', SPLITS => ['10', '20', '30', '40', '50',
'60', '70', '80']}}
{{Created table t1}}
{{Took 1.1306 seconds}}
*{{Insert test data}}*
{{hbase:012:0> put 't1', 'r1', 'f1:c1', 'a'}}
{{Took 0.0416 seconds}}
{{hbase:014:0> put 't1', 'r1', 'f1:c2', 'b'}}
{{Took 0.0059 seconds}}
{{hbase:015:0> put 't1', 'r1', 'f1:c3', 'c'}}
{{Took 0.0097 seconds}}
*Get without setMaxResultSize, returns full row and {{mayHaveMoreCellsInRow =
false}}*{{}}
{{hbase:037:0> g = Get.new('r1'.to_s.to_java_bytes)}}
{{=> #}}
{{hbase:038:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:039:0> result.mayHaveMoreCellsInRow}}
{{=> false}}
{{hbase:040:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0,
r1/f1:c3/1627498280413/Put/vlen=1/seqid=0}"}}
*{{Get with setMaxResultSize, returns first two columns and
}}{{mayHaveMoreCellsInRow = true}}*{{}}
{{hbase:059:0> g = Get.new('r1'.to_s.to_java_bytes).setMaxResultSize(100)}}
{{=> #}}
{{hbase:060:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)}}
{{=> #}}
{{hbase:061:0> result.mayHaveMoreCellsInRow}}
{{=> true}}
{{hbase:062:0> result.toString}}
{{=> "keyvalues=\{r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0}"}}
> Limit max result size of individual Gets
>
>
> Key: HBASE-26122
> URL: https://issues.apache.org/jira/browse/HBASE-26122
> Project: HBase
> Issue Type: New Feature
> Components: Client, regionserver
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>
> Scans have the ability to have a configured max result size, which causes
> them to return a partial result once the limit has been reached. MultiGets
> also can throw MultiActionResultTooLarge if the response size is over a
> configured quota. Neither of these really accounts for a single Get of a
> too-large row. Such too-large Gets can cause substantial GC pressure or worse
> if sent at volume.
> Currently one can work around this by converting their Get to a single row
> Scan, but this requires a developer to proactively know about and prepare for
> the issue by using a Scan upfront or wait for the RegionServer to choke on a
> large request and only then rewrite the Get for future requests.
> We should implement the same response size limits for for Get as for Scan,
> whereby the server returns a partial result to the client for handling.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25392) Direct insert compacted HFiles into data directory.
[
https://issues.apache.org/jira/browse/HBASE-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-25392.
--
Resolution: Fixed
> Direct insert compacted HFiles into data directory.
> ---
>
> Key: HBASE-25392
> URL: https://issues.apache.org/jira/browse/HBASE-25392
> Project: HBase
> Issue Type: Sub-task
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Wellington Chevreuil
>Priority: Major
>
> {color:#00}when performing compaction, write directly to the data
> directory {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25392) Direct insert compacted HFiles into data directory.
[
https://issues.apache.org/jira/browse/HBASE-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-25392:
-
Release Note:
Introduces the DirectStoreCompactor, an extension of DefaultCompactor, which
writes compactions result files straight into the store directory, rather than
the temp dir approach implemented by DefaultCompactor.
In commitCompaction, it does not perform any renames. Should always be used
together with PersistedStoreEngine proposed in HBASE-25395 and
DirectStoreFlusher proposed in HBASE-25391, as it's up for StoreEngine
implementations keep tracking of valid store files.
To enable this, the following properties must be set in RS configuration:
hbase.storefile.tracking.persist.enabledtrue
hbase.hstore.engine.class
org.apache.hadoop.hbase.regionserver.PersistedStoreEngine
hbase.regionserver.store.flush.context.classorg.apache.hadoop.hbase.regionserver.DirectStoreFlushContext
hbase.hstore.defaultengine.storeflusher.classorg.apache.hadoop.hbase.regionserver.DirectStoreFlusher
hbase.hstore.defaultengine.compactor.class
org.apache.hadoop.hbase.regionserver.compactions.DirectStoreCompactor
> Direct insert compacted HFiles into data directory.
> ---
>
> Key: HBASE-25392
> URL: https://issues.apache.org/jira/browse/HBASE-25392
> Project: HBase
> Issue Type: Sub-task
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Wellington Chevreuil
>Priority: Major
>
> {color:#00}when performing compaction, write directly to the data
> directory {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888526518 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | branch-2 passed | | +1 :green_heart: | compile | 8m 46s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 24s | branch-2 passed | | +1 :green_heart: | spotbugs | 17m 28s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | compile | 8m 48s | the patch passed | | -0 :warning: | javac | 8m 48s | root generated 4 new + 1885 unchanged - 2 fixed = 1889 total (was 1887) | | +1 :green_heart: | checkstyle | 2m 19s | root: The patch generated 0 new + 736 unchanged - 27 fixed = 736 total (was 763) | | +1 :green_heart: | rubocop | 0m 3s | The patch generated 0 new + 3 unchanged - 8 fixed = 3 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 16s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 11m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 19m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 2m 58s | The patch does not generate ASF License warnings. | | | | 93m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml rubocop | | uname | Linux 57297ea9521b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/artifact/yetus-general-check/output/diff-compile-javac-root.txt | | Max. process+thread count | 141 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26120) New replication gets stuck or data loss when multiwal groups more than 10
[
https://issues.apache.org/jira/browse/HBASE-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388935#comment-17388935
]
Hudson commented on HBASE-26120:
Results for branch branch-2.3
[build #264 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/264/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/264/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/264/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/264/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/264/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> New replication gets stuck or data loss when multiwal groups more than 10
> -
>
> Key: HBASE-26120
> URL: https://issues.apache.org/jira/browse/HBASE-26120
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 1.7.1, 2.4.5
>Reporter: Jasee Tao
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 2.3.6, 2.4.5, 1.7.2
>
>
> {code:java}
> void preLogRoll(Path newLog) throws IOException {
> recordLog(newLog);
> String logName = newLog.getName();
> String logPrefix = DefaultWALProvider.getWALPrefixFromWALName(logName);
> synchronized (latestPaths) {
> Iterator iterator = latestPaths.iterator();
> while (iterator.hasNext()) {
> Path path = iterator.next();
> if (path.getName().contains(logPrefix)) {
> iterator.remove();
> break;
> }
> }
> this.latestPaths.add(newLog);
> }
> }
> {code}
> ReplicationSourceManager use _latestPaths_ to track each walgroup's last
> WALlog and all of them will be enqueue for replication when new replication
> peer added。
> If we set hbase.wal.regiongrouping.numgroups > 10, says 12, the name of
> WALlog group will be _regionserver.null0.timestamp_ to
> _regionserver.null11.timestamp_。*_String.contains_* is used in _preoLogRoll_
> to replace old logs in same group, leads when _regionserver.null1.ts_ comes,
> _regionserver.null11.ts_ may be replaced, and *_latestPaths_ growing with
> wrong logs*.
> Replication then partly stuckd as _regionsserver.null1.ts_ not exists on
> hdfs, and data may not be replicated to slave as _regionserver.null11.ts_ not
> in replication queue at startup.
> Because of
> [ZOOKEEPER-706|https://issues.apache.org/jira/browse/ZOOKEEPER-706], if there
> is too many logs in zk _/hbase/replication/rs/regionserver/peer_, remove_peer
> may not delete this znode, and other regionserver can't not pick up this
> queue for replication failover.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26146) Allow custom opts for hbck in hbase bin
[
https://issues.apache.org/jira/browse/HBASE-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388919#comment-17388919
]
Hudson commented on HBASE-26146:
Results for branch branch-2
[build #308 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308//console].
> Allow custom opts for hbck in hbase bin
> ---
>
> Key: HBASE-26146
> URL: https://issues.apache.org/jira/browse/HBASE-26146
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.6
>
>
> https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
> execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
> to the command line. This is problematic in some cases depending on what
> regionserver opts have been set. For instance, one might configure a jmx port
> for the regionserver but then hbck will fail due to a port conflict if run on
> the same host as a regionserver. Another example would be that a regionserver
> might define an {{-Xms}} value which is significantly more than hbck requires.
>
> We should make it possible for users to define their own HBASE_HBCK_OPTS
> which take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26120) New replication gets stuck or data loss when multiwal groups more than 10
[
https://issues.apache.org/jira/browse/HBASE-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388918#comment-17388918
]
Hudson commented on HBASE-26120:
Results for branch branch-2
[build #308 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/308//console].
> New replication gets stuck or data loss when multiwal groups more than 10
> -
>
> Key: HBASE-26120
> URL: https://issues.apache.org/jira/browse/HBASE-26120
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 1.7.1, 2.4.5
>Reporter: Jasee Tao
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 2.3.6, 2.4.5, 1.7.2
>
>
> {code:java}
> void preLogRoll(Path newLog) throws IOException {
> recordLog(newLog);
> String logName = newLog.getName();
> String logPrefix = DefaultWALProvider.getWALPrefixFromWALName(logName);
> synchronized (latestPaths) {
> Iterator iterator = latestPaths.iterator();
> while (iterator.hasNext()) {
> Path path = iterator.next();
> if (path.getName().contains(logPrefix)) {
> iterator.remove();
> break;
> }
> }
> this.latestPaths.add(newLog);
> }
> }
> {code}
> ReplicationSourceManager use _latestPaths_ to track each walgroup's last
> WALlog and all of them will be enqueue for replication when new replication
> peer added。
> If we set hbase.wal.regiongrouping.numgroups > 10, says 12, the name of
> WALlog group will be _regionserver.null0.timestamp_ to
> _regionserver.null11.timestamp_。*_String.contains_* is used in _preoLogRoll_
> to replace old logs in same group, leads when _regionserver.null1.ts_ comes,
> _regionserver.null11.ts_ may be replaced, and *_latestPaths_ growing with
> wrong logs*.
> Replication then partly stuckd as _regionsserver.null1.ts_ not exists on
> hdfs, and data may not be replicated to slave as _regionserver.null11.ts_ not
> in replication queue at startup.
> Because of
> [ZOOKEEPER-706|https://issues.apache.org/jira/browse/ZOOKEEPER-706], if there
> is too many logs in zk _/hbase/replication/rs/regionserver/peer_, remove_peer
> may not delete this znode, and other regionserver can't not pick up this
> queue for replication failover.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3536: HBASE-26147: Add dry_run_balancer and related Admin interfaces for running the balancer without executing any region moves
Apache-HBase commented on pull request #3536: URL: https://github.com/apache/hbase/pull/3536#issuecomment-888504476 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 23s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 58s | branch-2 passed | | +1 :green_heart: | compile | 7m 14s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 56s | branch-2 passed | | +1 :green_heart: | spotbugs | 7m 46s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 7m 11s | the patch passed | | +1 :green_heart: | cc | 7m 11s | the patch passed | | +1 :green_heart: | javac | 7m 11s | the patch passed | | -0 :warning: | checkstyle | 0m 35s | hbase-client: The patch generated 2 new + 151 unchanged - 0 fixed = 153 total (was 151) | | -0 :warning: | checkstyle | 1m 16s | hbase-server: The patch generated 4 new + 107 unchanged - 0 fixed = 111 total (was 107) | | -0 :warning: | rubocop | 0m 3s | The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 52s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 46s | the patch passed | | +1 :green_heart: | spotbugs | 8m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 71m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3536 | | JIRA Issue | HBASE-26147 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux ce774313b86d 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 86 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3536/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3538: HBASE-26045 Master control the global throughtput of all compaction servers
Apache-HBase commented on pull request #3538: URL: https://github.com/apache/hbase/pull/3538#issuecomment-888454003 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-25714 Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 25s | HBASE-25714 passed | | +1 :green_heart: | compile | 3m 28s | HBASE-25714 passed | | +1 :green_heart: | shadedjars | 8m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 7s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 5s | the patch passed | | +1 :green_heart: | compile | 3m 29s | the patch passed | | +1 :green_heart: | javac | 3m 29s | the patch passed | | +1 :green_heart: | shadedjars | 9m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 52s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 18s | hbase-client in the patch passed. | | -1 :x: | unit | 155m 15s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 4m 14s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 26s | hbase-shell in the patch passed. | | | | 211m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3538 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5d39b79c2fbc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / 85f02919da | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/testReport/ | | Max. process+thread count | 3760 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3460: HBASE-26064 Introduce a StoreFileTracker to abstract the stor…
Apache-HBase commented on pull request #3460: URL: https://github.com/apache/hbase/pull/3460#issuecomment-888452836 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 5s | master passed | | +1 :green_heart: | compile | 4m 3s | master passed | | +1 :green_heart: | checkstyle | 1m 22s | master passed | | +1 :green_heart: | spotbugs | 2m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 38s | the patch passed | | -1 :x: | compile | 1m 25s | hbase-server in the patch failed. | | -0 :warning: | javac | 1m 25s | hbase-server in the patch failed. | | -0 :warning: | checkstyle | 1m 24s | hbase-server: The patch generated 1 new + 144 unchanged - 15 fixed = 145 total (was 159) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 22m 46s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | -1 :x: | spotbugs | 2m 20s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 56m 1s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.regionserver.HStore.close() does not release lock on all exception paths At HStore.java:on all exception paths At HStore.java:[line 718] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3460 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux d31fed8cefb7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3538: HBASE-26045 Master control the global throughtput of all compaction servers
Apache-HBase commented on pull request #3538: URL: https://github.com/apache/hbase/pull/3538#issuecomment-888445618 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-25714 Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 40s | HBASE-25714 passed | | +1 :green_heart: | compile | 3m 55s | HBASE-25714 passed | | +1 :green_heart: | shadedjars | 8m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 27s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 24s | the patch passed | | +1 :green_heart: | compile | 3m 48s | the patch passed | | +1 :green_heart: | javac | 3m 48s | the patch passed | | +1 :green_heart: | shadedjars | 8m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 1s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 18s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 142m 19s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 40s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 17s | hbase-shell in the patch passed. | | | | 199m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3538 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f83a3a0d36ed 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / 85f02919da | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/testReport/ | | Max. process+thread count | 3694 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3460: HBASE-26064 Introduce a StoreFileTracker to abstract the stor…
Apache-HBase commented on pull request #3460: URL: https://github.com/apache/hbase/pull/3460#issuecomment-888441742 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 20s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 8m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server generated 1 new + 86 unchanged - 0 fixed = 87 total (was 86) | ||| _ Other Tests _ | | -1 :x: | unit | 8m 47s | hbase-server in the patch failed. | | | | 39m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3460 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6a2b52e6d386 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-11.0.10+9 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/testReport/ | | Max. process+thread count | 851 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3460: HBASE-26064 Introduce a StoreFileTracker to abstract the stor…
Apache-HBase commented on pull request #3460: URL: https://github.com/apache/hbase/pull/3460#issuecomment-888441562 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 8m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 8m 40s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 36s | hbase-server generated 1 new + 21 unchanged - 0 fixed = 22 total (was 21) | ||| _ Other Tests _ | | -1 :x: | unit | 8m 37s | hbase-server in the patch failed. | | | | 39m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3460 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4be9b4fe05e9 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/testReport/ | | Max. process+thread count | 991 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3460/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26120) New replication gets stuck or data loss when multiwal groups more than 10
[
https://issues.apache.org/jira/browse/HBASE-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388867#comment-17388867
]
Hudson commented on HBASE-26120:
Results for branch branch-1
[build #154 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/154/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/154//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/154//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/154//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> New replication gets stuck or data loss when multiwal groups more than 10
> -
>
> Key: HBASE-26120
> URL: https://issues.apache.org/jira/browse/HBASE-26120
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 1.7.1, 2.4.5
>Reporter: Jasee Tao
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 2.3.6, 2.4.5, 1.7.2
>
>
> {code:java}
> void preLogRoll(Path newLog) throws IOException {
> recordLog(newLog);
> String logName = newLog.getName();
> String logPrefix = DefaultWALProvider.getWALPrefixFromWALName(logName);
> synchronized (latestPaths) {
> Iterator iterator = latestPaths.iterator();
> while (iterator.hasNext()) {
> Path path = iterator.next();
> if (path.getName().contains(logPrefix)) {
> iterator.remove();
> break;
> }
> }
> this.latestPaths.add(newLog);
> }
> }
> {code}
> ReplicationSourceManager use _latestPaths_ to track each walgroup's last
> WALlog and all of them will be enqueue for replication when new replication
> peer added。
> If we set hbase.wal.regiongrouping.numgroups > 10, says 12, the name of
> WALlog group will be _regionserver.null0.timestamp_ to
> _regionserver.null11.timestamp_。*_String.contains_* is used in _preoLogRoll_
> to replace old logs in same group, leads when _regionserver.null1.ts_ comes,
> _regionserver.null11.ts_ may be replaced, and *_latestPaths_ growing with
> wrong logs*.
> Replication then partly stuckd as _regionsserver.null1.ts_ not exists on
> hdfs, and data may not be replicated to slave as _regionserver.null11.ts_ not
> in replication queue at startup.
> Because of
> [ZOOKEEPER-706|https://issues.apache.org/jira/browse/ZOOKEEPER-706], if there
> is too many logs in zk _/hbase/replication/rs/regionserver/peer_, remove_peer
> may not delete this znode, and other regionserver can't not pick up this
> queue for replication failover.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26092) JVM core dump in the replication path
[
https://issues.apache.org/jira/browse/HBASE-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388837#comment-17388837
]
Huaxiang Sun commented on HBASE-26092:
--
Thanks for update, [~anoop.hbase]. I will take a look later (check rs logs as
well to see if I can find more context).
> JVM core dump in the replication path
> -
>
> Key: HBASE-26092
> URL: https://issues.apache.org/jira/browse/HBASE-26092
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.3.5
>Reporter: Huaxiang Sun
>Priority: Critical
>
> When replication is turned on, we found the following code dump in the region
> server.
> I checked the code dump for replication. I think I got some ideas. For
> replication, when RS receives walEdits from remote cluster, it needs to send
> them out to final RS. In this case, NettyRpcConnection is deployed, calls are
> queued while it refers to ByteBuffer in the context of replicationHandler
> (returned to the pool once it returns). Code dump will happen since the
> byteBuffer has been reused. Needs ref count in this asynchronous processing.
>
> Feel free to take it, otherwise, I will try to work on a patch later.
>
>
> {code:java}
> Stack: [0x7fb1bf039000,0x7fb1bf13a000], sp=0x7fb1bf138560, free
> space=1021k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> J 28175 C2
> org.apache.hadoop.hbase.ByteBufferKeyValue.write(Ljava/io/OutputStream;Z)I
> (21 bytes) @ 0x7fd2663c [0x7fd263c0+0x27c]
> J 14912 C2
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.writeRequest(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Lorg/apache/hadoop/hbase/ipc/Call;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (370 bytes) @ 0x7fdbbb94b590 [0x7fdbbb949c00+0x1990]
> J 14911 C2
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.write(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (30 bytes) @ 0x7fdbb972d1d4 [0x7fdbb972d1a0+0x34]
> J 30476 C2
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.write(Ljava/lang/Object;ZLorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (149 bytes) @ 0x7fdbbd4e7084 [0x7fdbbd4e6900+0x784]
> J 14914 C2 org.apache.hadoop.hbase.ipc.NettyRpcConnection$6$1.run()V (22
> bytes) @ 0x7fdbbb9344ec [0x7fdbbb934280+0x26c]
> J 23528 C2
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(J)Z
> (106 bytes) @ 0x7fdbbcbb0efc [0x7fdbbcbb0c40+0x2bc]
> J 15987% C2
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run()V (461
> bytes) @ 0x7fdbbbaf1580 [0x7fdbbbaf1360+0x220]
> j
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run()V+44
> j
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run()V+11
> j
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run()V+4
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3533: HBASE-26144 The HStore.snapshot method is never called in main code
Apache-HBase commented on pull request #3533: URL: https://github.com/apache/hbase/pull/3533#issuecomment-888364674 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 9m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 34s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 9m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 218m 31s | hbase-server in the patch failed. | | | | 254m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3533 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ca7eefea1477 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/testReport/ | | Max. process+thread count | 3417 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3538: HBASE-26045 Master control the global throughtput of all compaction servers
Apache-HBase commented on pull request #3538: URL: https://github.com/apache/hbase/pull/3538#issuecomment-888361474 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-25714 Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 21s | HBASE-25714 passed | | +1 :green_heart: | compile | 9m 9s | HBASE-25714 passed | | +1 :green_heart: | checkstyle | 3m 33s | HBASE-25714 passed | | +1 :green_heart: | spotbugs | 10m 40s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 7m 7s | the patch passed | | +1 :green_heart: | cc | 7m 7s | the patch passed | | +1 :green_heart: | javac | 7m 7s | the patch passed | | -0 :warning: | checkstyle | 0m 28s | hbase-client: The patch generated 1 new + 38 unchanged - 0 fixed = 39 total (was 38) | | -0 :warning: | checkstyle | 1m 11s | hbase-server: The patch generated 7 new + 6 unchanged - 0 fixed = 13 total (was 6) | | -0 :warning: | rubocop | 0m 29s | The patch generated 20 new + 656 unchanged - 0 fixed = 676 total (was 656) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 20m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | hbaseprotoc | 2m 57s | the patch passed | | +1 :green_heart: | spotbugs | 10m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. | | | | 89m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3538 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux 6bcc747bc9d2 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / 85f02919da | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3538/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26146) Allow custom opts for hbck in hbase bin
[
https://issues.apache.org/jira/browse/HBASE-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388785#comment-17388785
]
Hudson commented on HBASE-26146:
Results for branch master
[build #354 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/354/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/354/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/354/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow custom opts for hbck in hbase bin
> ---
>
> Key: HBASE-26146
> URL: https://issues.apache.org/jira/browse/HBASE-26146
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.6
>
>
> https://issues.apache.org/jira/browse/HBASE-15145 made it so that when you
> execute {{hbase hbck}}, the regionserver or JAAS opts are added automatically
> to the command line. This is problematic in some cases depending on what
> regionserver opts have been set. For instance, one might configure a jmx port
> for the regionserver but then hbck will fail due to a port conflict if run on
> the same host as a regionserver. Another example would be that a regionserver
> might define an {{-Xms}} value which is significantly more than hbck requires.
>
> We should make it possible for users to define their own HBASE_HBCK_OPTS
> which take precedence over the server opts added by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3533: HBASE-26144 The HStore.snapshot method is never called in main code
Apache-HBase commented on pull request #3533: URL: https://github.com/apache/hbase/pull/3533#issuecomment-888307383 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 59s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | +1 :green_heart: | shadedjars | 8m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 8m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 141m 25s | hbase-server in the patch passed. | | | | 175m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3533 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5d7049e25853 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/testReport/ | | Max. process+thread count | 4313 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 opened a new pull request #3538: HBASE-26045 Master control the global throughtput of all compaction servers
nyl3532016 opened a new pull request #3538: URL: https://github.com/apache/hbase/pull/3538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888260711 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 2m 18s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 42s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | compile | 2m 21s | the patch passed | | +1 :green_heart: | javac | 2m 21s | the patch passed | | +1 :green_heart: | shadedjars | 6m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 1s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 330m 4s | root in the patch failed. | | | | 372m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3d3a31e3fd6b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/testReport/ | | Max. process+thread count | 4605 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26092) JVM core dump in the replication path
[
https://issues.apache.org/jira/browse/HBASE-26092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388720#comment-17388720
]
Anoop Sam John commented on HBASE-26092:
The replicateWALEntry request is reached at an RS and read into a Pooled BB.
This is handled by ReplicationHandler thread. In ReplicationSink, this will
create mutation batches and call table.batch() (1 or more calls). So this
call is a sync call. The call wait until it is being processed fully. Means
this RS writes to the original region and that write is completed. After that
only this ReplicationThread's works getting over and finally calling cleanup
so as to release the BB.
Here NettyRPCClient in place. I checked the code in details and tried to do
some hacks in code for the netty EventLoop thread to sleep sometime before
actually sending the req, (NettyRpcConnection # sendRequest0) and with adding
more logs. Its clear from these experiment, to me that the release of BB happen
after this table.batch() is been completed.
So I dont think we have a BB leak as in other case.
> JVM core dump in the replication path
> -
>
> Key: HBASE-26092
> URL: https://issues.apache.org/jira/browse/HBASE-26092
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.3.5
>Reporter: Huaxiang Sun
>Priority: Critical
>
> When replication is turned on, we found the following code dump in the region
> server.
> I checked the code dump for replication. I think I got some ideas. For
> replication, when RS receives walEdits from remote cluster, it needs to send
> them out to final RS. In this case, NettyRpcConnection is deployed, calls are
> queued while it refers to ByteBuffer in the context of replicationHandler
> (returned to the pool once it returns). Code dump will happen since the
> byteBuffer has been reused. Needs ref count in this asynchronous processing.
>
> Feel free to take it, otherwise, I will try to work on a patch later.
>
>
> {code:java}
> Stack: [0x7fb1bf039000,0x7fb1bf13a000], sp=0x7fb1bf138560, free
> space=1021k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> J 28175 C2
> org.apache.hadoop.hbase.ByteBufferKeyValue.write(Ljava/io/OutputStream;Z)I
> (21 bytes) @ 0x7fd2663c [0x7fd263c0+0x27c]
> J 14912 C2
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.writeRequest(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Lorg/apache/hadoop/hbase/ipc/Call;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (370 bytes) @ 0x7fdbbb94b590 [0x7fdbbb949c00+0x1990]
> J 14911 C2
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.write(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (30 bytes) @ 0x7fdbb972d1d4 [0x7fdbb972d1a0+0x34]
> J 30476 C2
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.write(Ljava/lang/Object;ZLorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (149 bytes) @ 0x7fdbbd4e7084 [0x7fdbbd4e6900+0x784]
> J 14914 C2 org.apache.hadoop.hbase.ipc.NettyRpcConnection$6$1.run()V (22
> bytes) @ 0x7fdbbb9344ec [0x7fdbbb934280+0x26c]
> J 23528 C2
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(J)Z
> (106 bytes) @ 0x7fdbbcbb0efc [0x7fdbbcbb0c40+0x2bc]
> J 15987% C2
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run()V (461
> bytes) @ 0x7fdbbbaf1580 [0x7fdbbbaf1360+0x220]
> j
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run()V+44
> j
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run()V+11
> j
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run()V+4
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26148) Backport HBASE-26146 to branch-2.4
[ https://issues.apache.org/jira/browse/HBASE-26148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388705#comment-17388705 ] Bryan Beaudreault commented on HBASE-26148: --- Ok thanks, I'll do that next time. > Backport HBASE-26146 to branch-2.4 > -- > > Key: HBASE-26148 > URL: https://issues.apache.org/jira/browse/HBASE-26148 > Project: HBase > Issue Type: Sub-task >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888227629 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 6s | branch-2 passed | | +1 :green_heart: | compile | 3m 44s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 7m 52s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 24s | the patch passed | | +1 :green_heart: | compile | 3m 29s | the patch passed | | +1 :green_heart: | javac | 3m 29s | the patch passed | | +1 :green_heart: | shadedjars | 8m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 8m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 255m 19s | root in the patch passed. | | | | 312m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 89f949f79e34 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/testReport/ | | Max. process+thread count | 4062 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3533: HBASE-26144 The HStore.snapshot method is never called in main code
Apache-HBase commented on pull request #3533: URL: https://github.com/apache/hbase/pull/3533#issuecomment-888224683 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 3m 7s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 5s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 39s | the patch passed | | +1 :green_heart: | compile | 3m 7s | the patch passed | | +1 :green_heart: | javac | 3m 7s | the patch passed | | +1 :green_heart: | checkstyle | 1m 4s | hbase-server: The patch generated 0 new + 35 unchanged - 6 fixed = 35 total (was 41) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 47m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3533 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux f3f1bac4d343 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3533/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3531: HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.msl…
Apache-HBase commented on pull request #3531: URL: https://github.com/apache/hbase/pull/3531#issuecomment-888148470 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 34s | master passed | | +1 :green_heart: | compile | 1m 14s | master passed | | +1 :green_heart: | shadedjars | 8m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 34s | the patch passed | | +1 :green_heart: | compile | 1m 22s | the patch passed | | +1 :green_heart: | javac | 1m 22s | the patch passed | | +1 :green_heart: | shadedjars | 9m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 142m 49s | hbase-server in the patch passed. | | | | 179m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 964cd6446fc5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/2/testReport/ | | Max. process+thread count | 4373 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil merged pull request #3389: HBASE-25392 Direct insert compacted HFiles into data directory.
wchevreuil merged pull request #3389: URL: https://github.com/apache/hbase/pull/3389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888101334 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | branch-2 passed | | +1 :green_heart: | compile | 10m 10s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 39s | branch-2 passed | | +1 :green_heart: | spotbugs | 22m 5s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 11m 46s | the patch passed | | -0 :warning: | javac | 11m 46s | root generated 3 new + 1887 unchanged - 0 fixed = 1890 total (was 1887) | | -0 :warning: | checkstyle | 2m 53s | root: The patch generated 25 new + 742 unchanged - 21 fixed = 767 total (was 763) | | -0 :warning: | rubocop | 0m 6s | The patch generated 4 new + 3 unchanged - 8 fixed = 7 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 14s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 15m 37s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 26m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 3m 4s | The patch does not generate ASF License warnings. | | | | 118m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml rubocop | | uname | Linux 68a432edeb0d 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-general-check/output/diff-compile-javac-root.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 126 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3531: HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.msl…
Apache-HBase commented on pull request #3531: URL: https://github.com/apache/hbase/pull/3531#issuecomment-888069719 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 6s | master passed | | +1 :green_heart: | compile | 3m 43s | master passed | | +1 :green_heart: | checkstyle | 1m 13s | master passed | | +1 :green_heart: | spotbugs | 2m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 21s | the patch passed | | +1 :green_heart: | compile | 3m 40s | the patch passed | | +1 :green_heart: | javac | 3m 40s | the patch passed | | +1 :green_heart: | checkstyle | 1m 12s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 52s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 57m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3531 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux e8d9692caaeb 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 332d9d13b1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3531/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3529: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2
Apache-HBase commented on pull request #3529: URL: https://github.com/apache/hbase/pull/3529#issuecomment-888034297 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | branch-2 passed | | +1 :green_heart: | compile | 2m 33s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 6s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 2m 32s | the patch passed | | +1 :green_heart: | javac | 2m 32s | the patch passed | | +1 :green_heart: | shadedjars | 7m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 416m 59s | root in the patch passed. | | | | 462m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3529 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 95d44e6c18b7 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / d470b25e05 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/3/testReport/ | | Max. process+thread count | 3553 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-asyncfs hbase-server hbase-mapreduce hbase-shell hbase-it hbase-shaded hbase-shaded/hbase-shaded-client hbase-external-blockcache hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3529/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
