[GitHub] [hbase] Apache-HBase commented on pull request #4834: HBASE-27433 DefaultMobStoreCompactor should delete MobStoreFile clean…
Apache-HBase commented on PR #4834: URL: https://github.com/apache/hbase/pull/4834#issuecomment-1283497096 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 24s | master passed | | +1 :green_heart: | compile | 0m 40s | master passed | | +1 :green_heart: | shadedjars | 3m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 12s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | shadedjars | 3m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 214m 40s | hbase-server in the patch passed. | | | | 231m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4834 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 16492514dc1f 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5cacece790 | | Default Java | Temurin-1.8.0_345-b01 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/testReport/ | | Max. process+thread count | 2388 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4556: HBASE-26913 Replication Observability Framework
Apache9 commented on PR #4556: URL: https://github.com/apache/hbase/pull/4556#issuecomment-1283493979 > @Apache9 Do you think you might have some bandwidth sometime this week or so to review this PR? Will take a look soon. And do we have a new design doc? IIRC, the previous problem is how to make our WAL accept an entry which is not belonged to any regions, or we attach the special WAL entry to an existing entry. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4834: HBASE-27433 DefaultMobStoreCompactor should delete MobStoreFile clean…
Apache-HBase commented on PR #4834: URL: https://github.com/apache/hbase/pull/4834#issuecomment-1283485559 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 53s | master passed | | +1 :green_heart: | compile | 0m 44s | master passed | | +1 :green_heart: | shadedjars | 4m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | shadedjars | 4m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 199m 31s | hbase-server in the patch failed. | | | | 218m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4834 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7af76fdab14c 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5cacece790 | | Default Java | Eclipse Adoptium-11.0.16.1+1 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/testReport/ | | Max. process+thread count | 2679 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4834: HBASE-27433 DefaultMobStoreCompactor should delete MobStoreFile clean…
Apache-HBase commented on PR #4834: URL: https://github.com/apache/hbase/pull/4834#issuecomment-1283346823 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 6s | master passed | | +1 :green_heart: | compile | 2m 14s | master passed | | +1 :green_heart: | checkstyle | 0m 30s | master passed | | +1 :green_heart: | spotless | 0m 39s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 15s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 8s | the patch passed | | +1 :green_heart: | compile | 2m 14s | the patch passed | | +1 :green_heart: | javac | 2m 14s | the patch passed | | +1 :green_heart: | checkstyle | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 7m 48s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 26m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4834 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 922b9014750c 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5cacece790 | | Default Java | Temurin-1.8.0_345-b01 | | Max. process+thread count | 67 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #4556: HBASE-26913 Replication Observability Framework
virajjasani commented on PR #4556: URL: https://github.com/apache/hbase/pull/4556#issuecomment-1283095253 @Apache9 Do you think you might have some bandwidth sometime this week or so to review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27426) Region server abort with failing to list region servers.
[
https://issues.apache.org/jira/browse/HBASE-27426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619775#comment-17619775
]
Hudson commented on HBASE-27426:
Results for branch branch-2
[build #665 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Region server abort with failing to list region servers.
>
>
> Key: HBASE-27426
> URL: https://issues.apache.org/jira/browse/HBASE-27426
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Reporter: Victor Li
>Assignee: Victor Li
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.16
>
>
> During unit tests, clusters were created and destroyed, the region server
> could abort during shutdown with failing to list region servers.
>
> Logs:
>
> 2022-10-11 18:44:43,393 INFO [main] utility.HBaseClusterSetup(160): Starting
> mini cluster: singleCluster
> [master/victorli-wsl:0:becomeActiveMaster] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - not a secure deployment,
> proceeding
> 2022-10-11 18:45:07,234 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(177): Started mini cluster: singleCluster
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.
> 2022-10-11 18:45:45,980 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(132): Stopping mini cluster: singleCluster
> [RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close started.
> [zk-event-processor-pool-0] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer - * ABORTING region
> server victorli-wsl,35113,1665539090704: failed to list region servers *
> [RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close finished.
> [M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close started.
> [M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close finished.
> 2022-10-11 18:45:57,208 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(144): Stopped mini cluster: singleCluster
>
> Stack:
> [zk-event-processor-pool-0] WARN org.apache.hadoop.hbase.zookeeper.ZKUtil -
> regionserver:35113-0x101b6fed2a1, quorum=127.0.0.1:53028, baseZNode=/9
> Unable to list children of znode /9/rs
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1601)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1573)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2420)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:341)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:190)
> at
> org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.loadRegionServerList(RegionServerAddressTracker.java:54)
> at
> org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.nodeChildrenChanged(RegionServerAddressTracker.java:71)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.lambda$processEvent$0(ZKWatcher.java:586)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbas
[jira] [Commented] (HBASE-27159) Emit source metrics for BlockCacheExpressHitPercent
[
https://issues.apache.org/jira/browse/HBASE-27159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619776#comment-17619776
]
Hudson commented on HBASE-27159:
Results for branch branch-2
[build #665 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/665/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Emit source metrics for BlockCacheExpressHitPercent
> ---
>
> Key: HBASE-27159
> URL: https://issues.apache.org/jira/browse/HBASE-27159
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, metrics
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.16
>
>
> [https://github.com/apache/hbase/blob/d447fa01ba36a11d57927b78cce1bbca361b1d52/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java#L346-L400]
> {code:java}
> public double getHitCachingRatio() {
> double requestCachingCount = getRequestCachingCount();
> if (requestCachingCount == 0) {
> return 0;
> }
> return getHitCachingCount() / requestCachingCount;
> } {code}
> This code is responsible for the metric {{{}BlockCacheExpressHitPercent{}}}.
> The metric represents the percentage of requests which were cacheable, but
> not found in the cache. Unfortunately, since the counters are process-level
> counters, the ratio is for the lifetime of the process. This makes it less
> useful for looking at cache behavior during a smaller time period.
> The underlying counters are {{hitCachingCount}} and {{{}missCachingCount{}}}.
> Having access to the underlying counters allows for offline computation of
> the same metric for any given time period. But these counters are not emitted
> today from {{{}MetricsRegionServerWrapperImpl.java{}}}.
> Compare this to {{hitCount}} and {{missCount}} which are emitted as metrics
> {{blockCacheHitCount}} and {{{}blockCacheMissCount{}}}. But these are raw
> counts for the cache, which include requests that are not cacheable. The
> cacheable metrics are more interesting, since it can be common to miss on a
> request which is not cacheable.
> Interestingly, these metrics are emitted regularly as part of a log line in
> {{{}StatisticsThread.logStats{}}}.
> We should emit blockCache{{{}HitCachingCount{}}} and
> {{blockCacheMissCachingCount}} along with the current metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27159) Emit source metrics for BlockCacheExpressHitPercent
[
https://issues.apache.org/jira/browse/HBASE-27159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619770#comment-17619770
]
Hudson commented on HBASE-27159:
Results for branch branch-2.5
[build #228 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Emit source metrics for BlockCacheExpressHitPercent
> ---
>
> Key: HBASE-27159
> URL: https://issues.apache.org/jira/browse/HBASE-27159
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, metrics
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.16
>
>
> [https://github.com/apache/hbase/blob/d447fa01ba36a11d57927b78cce1bbca361b1d52/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java#L346-L400]
> {code:java}
> public double getHitCachingRatio() {
> double requestCachingCount = getRequestCachingCount();
> if (requestCachingCount == 0) {
> return 0;
> }
> return getHitCachingCount() / requestCachingCount;
> } {code}
> This code is responsible for the metric {{{}BlockCacheExpressHitPercent{}}}.
> The metric represents the percentage of requests which were cacheable, but
> not found in the cache. Unfortunately, since the counters are process-level
> counters, the ratio is for the lifetime of the process. This makes it less
> useful for looking at cache behavior during a smaller time period.
> The underlying counters are {{hitCachingCount}} and {{{}missCachingCount{}}}.
> Having access to the underlying counters allows for offline computation of
> the same metric for any given time period. But these counters are not emitted
> today from {{{}MetricsRegionServerWrapperImpl.java{}}}.
> Compare this to {{hitCount}} and {{missCount}} which are emitted as metrics
> {{blockCacheHitCount}} and {{{}blockCacheMissCount{}}}. But these are raw
> counts for the cache, which include requests that are not cacheable. The
> cacheable metrics are more interesting, since it can be common to miss on a
> request which is not cacheable.
> Interestingly, these metrics are emitted regularly as part of a log line in
> {{{}StatisticsThread.logStats{}}}.
> We should emit blockCache{{{}HitCachingCount{}}} and
> {{blockCacheMissCachingCount}} along with the current metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27432) TestUsersOperationsWithSecureHadoop fails affter HBASE-27411
[
https://issues.apache.org/jira/browse/HBASE-27432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619767#comment-17619767
]
Hudson commented on HBASE-27432:
Results for branch branch-2.5
[build #228 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestUsersOperationsWithSecureHadoop fails affter HBASE-27411
>
>
> Key: HBASE-27432
> URL: https://issues.apache.org/jira/browse/HBASE-27432
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.1
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.1
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27411) Update and clean up bcprov-jdk15on dependency
[
https://issues.apache.org/jira/browse/HBASE-27411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619768#comment-17619768
]
Hudson commented on HBASE-27411:
Results for branch branch-2.5
[build #228 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Update and clean up bcprov-jdk15on dependency
> --
>
> Key: HBASE-27411
> URL: https://issues.apache.org/jira/browse/HBASE-27411
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0-alpha-4
>Reporter: Nick Dimiduk
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> {noformat}
> $ env
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
> mvn -T0.5C clean install -DskipTests
>
> [INFO] Scanning for projects...
>
>
> [WARNING]
>
>
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hbase:hbase-server:jar:3.0.0-alpha-4-SNAPSHOT
>
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.bouncycastle:bcprov-jdk15on:jar -> duplicate declaration of
> version (?) @ line 331, column 17
> [WARNING]
>
>
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
>
> [WARNING]
>
>
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
>
> [WARNING]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27426) Region server abort with failing to list region servers.
[
https://issues.apache.org/jira/browse/HBASE-27426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619769#comment-17619769
]
Hudson commented on HBASE-27426:
Results for branch branch-2.5
[build #228 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/228/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Region server abort with failing to list region servers.
>
>
> Key: HBASE-27426
> URL: https://issues.apache.org/jira/browse/HBASE-27426
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Reporter: Victor Li
>Assignee: Victor Li
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.16
>
>
> During unit tests, clusters were created and destroyed, the region server
> could abort during shutdown with failing to list region servers.
>
> Logs:
>
> 2022-10-11 18:44:43,393 INFO [main] utility.HBaseClusterSetup(160): Starting
> mini cluster: singleCluster
> [master/victorli-wsl:0:becomeActiveMaster] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - not a secure deployment,
> proceeding
> 2022-10-11 18:45:07,234 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(177): Started mini cluster: singleCluster
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.
> 2022-10-11 18:45:45,980 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(132): Stopping mini cluster: singleCluster
> [RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close started.
> [zk-event-processor-pool-0] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer - * ABORTING region
> server victorli-wsl,35113,1665539090704: failed to list region servers *
> [RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close finished.
> [M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close started.
> [M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close finished.
> 2022-10-11 18:45:57,208 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(144): Stopped mini cluster: singleCluster
>
> Stack:
> [zk-event-processor-pool-0] WARN org.apache.hadoop.hbase.zookeeper.ZKUtil -
> regionserver:35113-0x101b6fed2a1, quorum=127.0.0.1:53028, baseZNode=/9
> Unable to list children of znode /9/rs
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1601)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1573)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2420)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:341)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:190)
> at
> org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.loadRegionServerList(RegionServerAddressTracker.java:54)
> at
> org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.nodeChildrenChanged(RegionServerAddressTracker.java:71)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.lambda$processEvent$0(ZKWatcher.java:586)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at
[jira] [Commented] (HBASE-27159) Emit source metrics for BlockCacheExpressHitPercent
[
https://issues.apache.org/jira/browse/HBASE-27159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619755#comment-17619755
]
Hudson commented on HBASE-27159:
Results for branch master
[build #703 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Emit source metrics for BlockCacheExpressHitPercent
> ---
>
> Key: HBASE-27159
> URL: https://issues.apache.org/jira/browse/HBASE-27159
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, metrics
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.16
>
>
> [https://github.com/apache/hbase/blob/d447fa01ba36a11d57927b78cce1bbca361b1d52/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java#L346-L400]
> {code:java}
> public double getHitCachingRatio() {
> double requestCachingCount = getRequestCachingCount();
> if (requestCachingCount == 0) {
> return 0;
> }
> return getHitCachingCount() / requestCachingCount;
> } {code}
> This code is responsible for the metric {{{}BlockCacheExpressHitPercent{}}}.
> The metric represents the percentage of requests which were cacheable, but
> not found in the cache. Unfortunately, since the counters are process-level
> counters, the ratio is for the lifetime of the process. This makes it less
> useful for looking at cache behavior during a smaller time period.
> The underlying counters are {{hitCachingCount}} and {{{}missCachingCount{}}}.
> Having access to the underlying counters allows for offline computation of
> the same metric for any given time period. But these counters are not emitted
> today from {{{}MetricsRegionServerWrapperImpl.java{}}}.
> Compare this to {{hitCount}} and {{missCount}} which are emitted as metrics
> {{blockCacheHitCount}} and {{{}blockCacheMissCount{}}}. But these are raw
> counts for the cache, which include requests that are not cacheable. The
> cacheable metrics are more interesting, since it can be common to miss on a
> request which is not cacheable.
> Interestingly, these metrics are emitted regularly as part of a log line in
> {{{}StatisticsThread.logStats{}}}.
> We should emit blockCache{{{}HitCachingCount{}}} and
> {{blockCacheMissCachingCount}} along with the current metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27426) Region server abort with failing to list region servers.
[
https://issues.apache.org/jira/browse/HBASE-27426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619754#comment-17619754
]
Hudson commented on HBASE-27426:
Results for branch master
[build #703 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Region server abort with failing to list region servers.
>
>
> Key: HBASE-27426
> URL: https://issues.apache.org/jira/browse/HBASE-27426
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Reporter: Victor Li
>Assignee: Victor Li
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.16
>
>
> During unit tests, clusters were created and destroyed, the region server
> could abort during shutdown with failing to list region servers.
>
> Logs:
>
> 2022-10-11 18:44:43,393 INFO [main] utility.HBaseClusterSetup(160): Starting
> mini cluster: singleCluster
> [master/victorli-wsl:0:becomeActiveMaster] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - not a secure deployment,
> proceeding
> 2022-10-11 18:45:07,234 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(177): Started mini cluster: singleCluster
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.
> [Listener at localhost/45185] INFO
> org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.
> 2022-10-11 18:45:45,980 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(132): Stopping mini cluster: singleCluster
> [RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close started.
> [zk-event-processor-pool-0] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer - * ABORTING region
> server victorli-wsl,35113,1665539090704: failed to list region servers *
> [RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close finished.
> [M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close started.
> [M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher -
> ZKWatcher close finished.
> 2022-10-11 18:45:57,208 INFO [Listener at localhost/45185]
> utility.HBaseClusterSetup(144): Stopped mini cluster: singleCluster
>
> Stack:
> [zk-event-processor-pool-0] WARN org.apache.hadoop.hbase.zookeeper.ZKUtil -
> regionserver:35113-0x101b6fed2a1, quorum=127.0.0.1:53028, baseZNode=/9
> Unable to list children of znode /9/rs
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1601)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1573)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2420)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:341)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:190)
> at
> org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.loadRegionServerList(RegionServerAddressTracker.java:54)
> at
> org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.nodeChildrenChanged(RegionServerAddressTracker.java:71)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.lambda$processEvent$0(ZKWatcher.java:586)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.processEvent(ZKWatcher.java:554)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.lambda$process$1(ZKWatcher.ja
[jira] [Commented] (HBASE-27344) Update ref guide about the 2.5.x release line
[
https://issues.apache.org/jira/browse/HBASE-27344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619753#comment-17619753
]
Hudson commented on HBASE-27344:
Results for branch master
[build #703 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/703/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Update ref guide about the 2.5.x release line
> -
>
> Key: HBASE-27344
> URL: https://issues.apache.org/jira/browse/HBASE-27344
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> The hadoop support matrix, release manager, etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] shahrs87 commented on a diff in pull request #4556: HBASE-26913 Replication Observability Framework
shahrs87 commented on code in PR #4556:
URL: https://github.com/apache/hbase/pull/4556#discussion_r998548510
##
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationMarkerChore.java:
##
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication.regionserver;
+
+import static
org.apache.hadoop.hbase.replication.master.ReplicationSinkTrackerTableCreator.REPLICATION_SINK_TRACKER_TABLE_NAME;
+
+import java.io.IOException;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.ScheduledChore;
+import org.apache.hadoop.hbase.Stoppable;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl;
+import org.apache.hadoop.hbase.regionserver.RegionServerServices;
+import org.apache.hadoop.hbase.regionserver.wal.WALUtil;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This chore is responsible to create replication marker rows with special
WALEdit with family as
+ * {@link org.apache.hadoop.hbase.wal.WALEdit#METAFAMILY} and column qualifier
as
+ * {@link WALEdit#REPLICATION_MARKER} and empty value. If config key
+ * {@link #REPLICATION_MARKER_ENABLED_KEY} is set to true, then we will create
1 marker row every
+ * {@link #REPLICATION_MARKER_CHORE_DURATION_KEY} ms
+ * {@link
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader}
will populate
+ * the Replication Marker edit with region_server_name, wal_name and
wal_offset encoded in
+ * {@link
org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.ReplicationMarkerDescriptor}
+ * object. {@link
org.apache.hadoop.hbase.replication.regionserver.Replication} will change the
+ * REPLICATION_SCOPE for this edit to GLOBAL so that it can replicate. On the
sink cluster,
+ * {@link org.apache.hadoop.hbase.replication.regionserver.ReplicationSink}
will convert the
+ * ReplicationMarkerDescriptor into a Put mutation to
REPLICATION_SINK_TRACKER_TABLE_NAME_STR table.
+ */
+@InterfaceAudience.Private
+public class ReplicationMarkerChore extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(ReplicationMarkerChore.class);
+ private static final MultiVersionConcurrencyControl MVCC = new
MultiVersionConcurrencyControl();
+ public static final RegionInfo REGION_INFO =
+RegionInfoBuilder.newBuilder(REPLICATION_SINK_TRACKER_TABLE_NAME).build();
+ private static final String DELIMITER = "_";
+ private final Configuration conf;
+ private final RegionServerServices rsServices;
+ private WAL wal;
+
+ public static final String REPLICATION_MARKER_ENABLED_KEY =
+"hbase.regionserver.replication.marker.enabled";
+ public static final boolean REPLICATION_MARKER_ENABLED_DEFAULT = false;
+
+ public static final String REPLICATION_MARKER_CHORE_DURATION_KEY =
+"hbase.regionserver.replication.marker.chore.duration";
+ public static final int REPLICATION_MARKER_CHORE_DURATION_DEFAULT = 30 *
1000; // 30 seconds
+
+ public ReplicationMarkerChore(final Stoppable stopper, final
RegionServerServices rsServices,
+int period, Configuration conf) {
+super("ReplicationTrackerChore", stopper, period);
+this.conf = conf;
+this.rsServices = rsServices;
+ }
+
+ @Override
+ protected void chore() {
+if (wal == null) {
+ try {
+wal = rsServices.getWAL(null);
+ } catch (IOException ioe) {
+LOG.warn("Unable to get WAL ", ioe);
+// Shouldn't happen. Ignore and wait for the next chore run.
+return;
+ }
+}
+String serverName = rsServices.getServerName().getServerName();
+long timeStamp = EnvironmentEdgeManager.currentTime();
+// We only have timestamp in ReplicationMarkerDescriptor and the remaining
properties walname,
+// regionserver name and wal offset at Replicat
[jira] [Commented] (HBASE-27414) Search order for locations in HFileLink
[ https://issues.apache.org/jira/browse/HBASE-27414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619679#comment-17619679 ] ruanhui commented on HBASE-27414: - hi [~huaxiangsun] would you mind if I prepare a patch for this ? > Search order for locations in HFileLink > > > Key: HBASE-27414 > URL: https://issues.apache.org/jira/browse/HBASE-27414 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: Huaxiang Sun >Priority: Minor > > Found that search order for locations is following the order of these > locations added to HFileLink object. > > setLocations(originPath, tempPath, mobPath, archivePath); > archivePath is the last one to be searched. For most cases, hfile exists in > archivePath, so we can move archivePath to the first parameter to avoid > unnecessary NN query. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27423) Upgrade Jackson for CVE-2022-42003/42004
[ https://issues.apache.org/jira/browse/HBASE-27423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27423: -- Component/s: security > Upgrade Jackson for CVE-2022-42003/42004 > > > Key: HBASE-27423 > URL: https://issues.apache.org/jira/browse/HBASE-27423 > Project: HBase > Issue Type: Bug > Components: security >Reporter: Andrew Kyle Purtell >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.2, 2.4.16 > > > Jackson 2.13.4 fixes CVE-2022-42003 and databind 2.14.0-rc1 fixes > CVE-2022-42004. > Move jackson.version to 2.13.4. > Move jackson.databind.version to 2.14.0-rc1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27423) Upgrade Jackson for CVE-2022-42003/42004
[ https://issues.apache.org/jira/browse/HBASE-27423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619450#comment-17619450 ] Duo Zhang commented on HBASE-27423: --- OK, got it. Let's wait for 2.14.0. Thanks. > Upgrade Jackson for CVE-2022-42003/42004 > > > Key: HBASE-27423 > URL: https://issues.apache.org/jira/browse/HBASE-27423 > Project: HBase > Issue Type: Bug >Reporter: Andrew Kyle Purtell >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.2, 2.4.16 > > > Jackson 2.13.4 fixes CVE-2022-42003 and databind 2.14.0-rc1 fixes > CVE-2022-42004. > Move jackson.version to 2.13.4. > Move jackson.databind.version to 2.14.0-rc1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-27430) Should disable replication log cleaner when migrating replication queue data
[ https://issues.apache.org/jira/browse/HBASE-27430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-27430: - Assignee: Duo Zhang > Should disable replication log cleaner when migrating replication queue data > > > Key: HBASE-27430 > URL: https://issues.apache.org/jira/browse/HBASE-27430 > Project: HBase > Issue Type: Sub-task > Components: master, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4834: HBASE-27433 DefaultMobStoreCompactor should delete MobStoreFile clean…
Apache-HBase commented on PR #4834: URL: https://github.com/apache/hbase/pull/4834#issuecomment-1281997299 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | master passed | | +1 :green_heart: | compile | 0m 38s | master passed | | +1 :green_heart: | shadedjars | 3m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 217m 34s | hbase-server in the patch failed. | | | | 235m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4834 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bf677bfbbe7d 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5cacece790 | | Default Java | Temurin-1.8.0_345-b01 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/testReport/ | | Max. process+thread count | 2345 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4834: HBASE-27433 DefaultMobStoreCompactor should delete MobStoreFile clean…
Apache-HBase commented on PR #4834: URL: https://github.com/apache/hbase/pull/4834#issuecomment-1281975018 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 50s | master passed | | +1 :green_heart: | compile | 0m 45s | master passed | | +1 :green_heart: | shadedjars | 4m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | shadedjars | 4m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 200m 21s | hbase-server in the patch passed. | | | | 218m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4834 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5715569a69a8 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5cacece790 | | Default Java | Eclipse Adoptium-11.0.16.1+1 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/testReport/ | | Max. process+thread count | 2678 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4834/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
