[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000683126 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 13s | master passed | | +1 :green_heart: | compile | 2m 19s | master passed | | +1 :green_heart: | shadedjars | 9m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 5s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 4s | the patch passed | | +1 :green_heart: | compile | 2m 21s | the patch passed | | +1 :green_heart: | javac | 2m 21s | the patch passed | | +1 :green_heart: | shadedjars | 9m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 46s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 206m 20s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 9m 12s | hbase-thrift in the patch passed. | | | | 257m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f04e2653f4c9 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/testReport/ | | Max. process+thread count | 2756 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3972: HBASE-26596 region_mover should gracefully ignore null response from RSGroupAdmin#getRSGroupOfServer
virajjasani commented on a change in pull request #3972:
URL: https://github.com/apache/hbase/pull/3972#discussion_r774895851
##
File path: bin/region_mover.rb
##
@@ -493,11 +493,22 @@ def getFilename(options, targetServer, port)
return filename
end
+def getServersRSGroup(rsgroup_admin, hostname, port)
+ rsgroup = rsgroup_admin.getRSGroupOfServer(Address.fromParts(hostname,
+ java.lang.Integer.parseInt(port)))
+ # If the rsgroup is nil, that means this server belongs to no rsgroup.
+ # It should be already offline.
+ # Here we directly log and exit.
+ return rsgroup unless rsgroup.nil?
+ $LOG.info('The server ' + hostname + 'belongs to no rsgroup. Exit regions
moving.')
Review comment:
From the overall system viewpoint, this might be seen as a normal
behaviour that region_mover exited gracefully as expected, however from
region_mover's own viewpoint, yes it makes sense to convert it to WARN level
because the same behaviour is possible only when either the given server is in
zombie state or maybe some other issues persist in RSGroup APIs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3972: HBASE-26596 region_mover should gracefully ignore null response from RSGroupAdmin#getRSGroupOfServer
virajjasani commented on a change in pull request #3972:
URL: https://github.com/apache/hbase/pull/3972#discussion_r774894839
##
File path: bin/region_mover.rb
##
@@ -493,11 +493,22 @@ def getFilename(options, targetServer, port)
return filename
end
+def getServersRSGroup(rsgroup_admin, hostname, port)
+ rsgroup = rsgroup_admin.getRSGroupOfServer(Address.fromParts(hostname,
+ java.lang.Integer.parseInt(port)))
+ # If the rsgroup is nil, that means this server belongs to no rsgroup.
+ # It should be already offline.
+ # Here we directly log and exit.
+ return rsgroup unless rsgroup.nil?
+ $LOG.info('The server ' + hostname + 'belongs to no rsgroup. Exit regions
moving.')
+ exit 0
Review comment:
> It's okay to exit with 0 if the server has no rsgroup AND has no
regions.
> I don't know whether a call to `getOnlineRegions` would succeed.
Hmm, I agree with first statement and that second statement would also be
true.
How about we check something like this?
```
rsgroup = rsgroupAdmin.getRSGroupOfServer
if rsgroup.nil?
try
regions = admin.getOnlineRegions(server)
except
LOG.warn(.)
exit 0
if not regions.nil? and regions.size == 0:
exit 0
else
LOG.warn('Server is not part of any RSGroup and still hosting some
regions')
exit 5
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-26052) Release 3.0.0-alpha-1
[ https://issues.apache.org/jira/browse/HBASE-26052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26052: -- Component/s: community > Release 3.0.0-alpha-1 > - > > Key: HBASE-26052 > URL: https://issues.apache.org/jira/browse/HBASE-26052 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26111) Release 3.0.0-alpha-2
[ https://issues.apache.org/jira/browse/HBASE-26111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26111. --- Resolution: Fixed Done. > Release 3.0.0-alpha-2 > - > > Key: HBASE-26111 > URL: https://issues.apache.org/jira/browse/HBASE-26111 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26621) Set version as 3.0.0-alpha-3-SNAPSHOT in master
[ https://issues.apache.org/jira/browse/HBASE-26621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26621. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~psomogyi] for reviewing. > Set version as 3.0.0-alpha-3-SNAPSHOT in master > --- > > Key: HBASE-26621 > URL: https://issues.apache.org/jira/browse/HBASE-26621 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache9 merged pull request #3978: HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master
Apache9 merged pull request #3978: URL: https://github.com/apache/hbase/pull/3978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000671134 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 26s | master passed | | +1 :green_heart: | compile | 3m 10s | master passed | | +1 :green_heart: | shadedjars | 12m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 7s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 6m 2s | the patch passed | | +1 :green_heart: | compile | 2m 35s | the patch passed | | +1 :green_heart: | javac | 2m 35s | the patch passed | | +1 :green_heart: | shadedjars | 9m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 43s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 162m 54s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 6m 45s | hbase-thrift in the patch passed. | | | | 217m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1db1c6719581 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/testReport/ | | Max. process+thread count | 3639 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26599) Netty exclusion through ZooKeeper not effective as intended
[
https://issues.apache.org/jira/browse/HBASE-26599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464898#comment-17464898
]
Duo Zhang commented on HBASE-26599:
---
IIRC zookeeper depends on netty 3.x? I do not know if they have upgrade to
netty 4.x in newer version, though...
> Netty exclusion through ZooKeeper not effective as intended
> ---
>
> Key: HBASE-26599
> URL: https://issues.apache.org/jira/browse/HBASE-26599
> Project: HBase
> Issue Type: Bug
> Components: dependencies
>Affects Versions: 2.4.8
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
>
> Picking up where [~psomogyi] has been digging this week. We've been seeing an
> issue where MiniDFS-based tests fail to start due to missing netty classes.
> HBASE-25969 seems to have intended to remove transitive Netty but was
> ineffective (at least for hadoop.profile=3.0). The dependency exclusion was
> for {{io.netty:netty}} and {{io.netty:netty-all}} but ZooKeeper 3.5.7
> transitively depends on {{netty-handler}} and
> {{netty-transport-native-epoll}} (per
> [https://search.maven.org/artifact/org.apache.zookeeper/zookeeper/3.5.7/jar)]
> The funny part is that we _should_ have seen failures in any hbase unit test
> using MiniDFS because we excluded netty and netty-all in HBASE-25969, but
> because we missed the exclusions, we still keep running.
> The intent of HBASE-25969 was good, but I think we need to revisit the
> execution. We need netty dependencies on the scope=test classpath. We just
> want to keep them off the scope=compile classpath (out of our client and
> server jars).
> disclaimer: I have not yet looked at 2.5.x or master yet to see if this also
> affects them.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] sunhelly commented on a change in pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
sunhelly commented on a change in pull request #3800:
URL: https://github.com/apache/hbase/pull/3800#discussion_r774865512
##
File path:
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/monitor/StreamSlowMonitor.java
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.asyncfs.monitor;
+
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_EXCLUDE_DATANODE_TTL;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_EXCLUDE_DATANODE_TTL_KEY;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT_KEY;
+
+import java.util.Deque;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.TimeUnit;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.conf.ConfigurationObserver;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheBuilder;
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader;
+import org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache;
+
+/**
+ * Class for monitor the wal file flush performance.
+ * Each active wal file has a StreamSlowMonitor.
+ */
+@InterfaceAudience.Private
+public class StreamSlowMonitor implements ConfigurationObserver {
+ private static final Logger LOG =
LoggerFactory.getLogger(StreamSlowMonitor.class);
+
+ /**
+ * Configure for the min count for a datanode detected slow.
+ * If a datanode is detected slow times up to this count, then it will be
added to the exclude
+ * datanode cache by {@link
ExcludeDatanodeManager#tryAddExcludeDN(DatanodeInfo, String)}
+ * of this regionsever.
+ */
+ private static final String WAL_SLOW_DETECT_MIN_COUNT_KEY =
+"hbase.regionserver.async.wal.min.slow.detect.count";
+ private static final int DEFAULT_WAL_SLOW_DETECT_MIN_COUNT = 3;
+
+ /**
+ * Configure for the TTL of the data that a datanode detected slow.
+ */
+ private static final String WAL_SLOW_DETECT_DATA_TTL_KEY =
+"hbase.regionserver.async.wal.slow.detect.data.ttl.ms";
+ private static final long DEFAULT_WAL_SLOW_DETECT_DATA_TTL = 10 * 60 * 1000;
// 10min in ms
+
+ /**
+ * Configure for the slow packet process time, a duration from send to ACK.
+ * It is preferred that this value should not be less than 1s.
+ */
+ private static final String DATANODE_SLOW_PACKET_PROCESS_TIME_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.process.time.millis";
+ private static final long DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME = 6000;
//6s in ms
+
+ /**
+ * Configure for the packet flush speed.
+ * e.g. If the configured slow slow packet process time is larger than 1s,
then here 0.1kbs means
+ * 100B should be processed in less than 1s.
+ * If the configured slow slow packet process time is larger than 10s, then
here 0.1kbs means
+ * 1KB should be processed in less than 10s.
+ */
+ private static final String DATANODE_SLOW_PACKET_FLUSH_SPEED_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.flush.speed.kbs";
+ private static final double DEFAULT_DATANODE_SLOW_PACKET_FLUSH_SPEED = 0.1;
+
+ private final String name;
+ // this is a map of datanodeInfo->queued slow PacketAckData
+ private final LoadingCache>
datanodeSlowDataQueue;
+ private final ExcludeDatanodeManager excludeDatanodeManager;
+
+ private int minSlowDetectCount;
+ private long slowDataTtl;
+ private long slowPacketAckMs;
+ private double slowPacketAckSpeed;
+
+ public StreamSlowMonitor(Configuration conf, String name,
+ ExcludeDatanodeManager excludeDatanodeManager) {
+setConf(conf);
+this.name = name;
+this.excludeDatanodeManager = excludeDatanodeManager;
+this.datanodeSlowDataQueue = CacheBuilder.ne
[jira] [Created] (HBASE-26624) [hbase-operator-tools] Introduce a HBCK2 tool to fix the store file tracking
Duo Zhang created HBASE-26624: - Summary: [hbase-operator-tools] Introduce a HBCK2 tool to fix the store file tracking Key: HBASE-26624 URL: https://issues.apache.org/jira/browse/HBASE-26624 Project: HBase Issue Type: Sub-task Components: hbase-operator-tools, hbck2 Reporter: Duo Zhang We should provide a HBCK2 tool to recover the store file tracking if it is broken. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] sunhelly commented on a change in pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
sunhelly commented on a change in pull request #3800:
URL: https://github.com/apache/hbase/pull/3800#discussion_r774865512
##
File path:
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/monitor/StreamSlowMonitor.java
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.asyncfs.monitor;
+
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_EXCLUDE_DATANODE_TTL;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_EXCLUDE_DATANODE_TTL_KEY;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT_KEY;
+
+import java.util.Deque;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.TimeUnit;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.conf.ConfigurationObserver;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheBuilder;
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader;
+import org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache;
+
+/**
+ * Class for monitor the wal file flush performance.
+ * Each active wal file has a StreamSlowMonitor.
+ */
+@InterfaceAudience.Private
+public class StreamSlowMonitor implements ConfigurationObserver {
+ private static final Logger LOG =
LoggerFactory.getLogger(StreamSlowMonitor.class);
+
+ /**
+ * Configure for the min count for a datanode detected slow.
+ * If a datanode is detected slow times up to this count, then it will be
added to the exclude
+ * datanode cache by {@link
ExcludeDatanodeManager#tryAddExcludeDN(DatanodeInfo, String)}
+ * of this regionsever.
+ */
+ private static final String WAL_SLOW_DETECT_MIN_COUNT_KEY =
+"hbase.regionserver.async.wal.min.slow.detect.count";
+ private static final int DEFAULT_WAL_SLOW_DETECT_MIN_COUNT = 3;
+
+ /**
+ * Configure for the TTL of the data that a datanode detected slow.
+ */
+ private static final String WAL_SLOW_DETECT_DATA_TTL_KEY =
+"hbase.regionserver.async.wal.slow.detect.data.ttl.ms";
+ private static final long DEFAULT_WAL_SLOW_DETECT_DATA_TTL = 10 * 60 * 1000;
// 10min in ms
+
+ /**
+ * Configure for the slow packet process time, a duration from send to ACK.
+ * It is preferred that this value should not be less than 1s.
+ */
+ private static final String DATANODE_SLOW_PACKET_PROCESS_TIME_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.process.time.millis";
+ private static final long DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME = 6000;
//6s in ms
+
+ /**
+ * Configure for the packet flush speed.
+ * e.g. If the configured slow slow packet process time is larger than 1s,
then here 0.1kbs means
+ * 100B should be processed in less than 1s.
+ * If the configured slow slow packet process time is larger than 10s, then
here 0.1kbs means
+ * 1KB should be processed in less than 10s.
+ */
+ private static final String DATANODE_SLOW_PACKET_FLUSH_SPEED_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.flush.speed.kbs";
+ private static final double DEFAULT_DATANODE_SLOW_PACKET_FLUSH_SPEED = 0.1;
+
+ private final String name;
+ // this is a map of datanodeInfo->queued slow PacketAckData
+ private final LoadingCache>
datanodeSlowDataQueue;
+ private final ExcludeDatanodeManager excludeDatanodeManager;
+
+ private int minSlowDetectCount;
+ private long slowDataTtl;
+ private long slowPacketAckMs;
+ private double slowPacketAckSpeed;
+
+ public StreamSlowMonitor(Configuration conf, String name,
+ ExcludeDatanodeManager excludeDatanodeManager) {
+setConf(conf);
+this.name = name;
+this.excludeDatanodeManager = excludeDatanodeManager;
+this.datanodeSlowDataQueue = CacheBuilder.ne
[GitHub] [hbase] Apache-HBase commented on pull request #3944: HBASE-26576
Apache-HBase commented on pull request #3944: URL: https://github.com/apache/hbase/pull/3944#issuecomment-1000638183 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 31s | master passed | | +1 :green_heart: | compile | 1m 30s | master passed | | +1 :green_heart: | shadedjars | 9m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 32s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 9m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 225m 17s | hbase-server in the patch failed. | | | | 263m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3944 | | JIRA Issue | HBASE-26576 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux aa9d696fd09c 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/testReport/ | | Max. process+thread count | 2541 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26598) Fix excessive connections in MajorCompactor
[
https://issues.apache.org/jira/browse/HBASE-26598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464880#comment-17464880
]
Hudson commented on HBASE-26598:
Results for branch branch-2.4
[build #263 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix excessive connections in MajorCompactor
> ---
>
> Key: HBASE-26598
> URL: https://issues.apache.org/jira/browse/HBASE-26598
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, tooling
>Reporter: Muhammad Samir Khan
>Assignee: Muhammad Samir Khan
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> MajorCompactor creates excessive connections: once every time
> MajorCompactionRequest.getConnection is called. Can do better by using the
> Connection object created in MajorCompactor.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26609) Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad
[
https://issues.apache.org/jira/browse/HBASE-26609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464881#comment-17464881
]
Hudson commented on HBASE-26609:
Results for branch branch-2.4
[build #263 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Round the size to MB or KB at the end of calculation in
> HRegionServer.createRegionLoad
> --
>
> Key: HBASE-26609
> URL: https://issues.apache.org/jira/browse/HBASE-26609
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> This is a follow up issue of HBASE-26340.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26615) Snapshot referenced data files are deleted when delete a table with merge regions
[
https://issues.apache.org/jira/browse/HBASE-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464882#comment-17464882
]
Hudson commented on HBASE-26615:
Results for branch branch-2.4
[build #263 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/263/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Snapshot referenced data files are deleted when delete a table with merge
> regions
> -
>
> Key: HBASE-26615
> URL: https://issues.apache.org/jira/browse/HBASE-26615
> Project: HBase
> Issue Type: Bug
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> In our cluster, we have a feature: take a snapshot when delete a table.
> But when we restore the snapshot, we found that some data files are deleted.
> The problem is that, when delete a table with merge regions, HBase only
> archive regions in meta, and merged parent regions are deleted in file system
> which contain data files in snapshot .
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000632003 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 41s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 23s | master passed | | +1 :green_heart: | compile | 4m 52s | master passed | | +1 :green_heart: | checkstyle | 2m 10s | master passed | | +1 :green_heart: | spotbugs | 4m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 18s | the patch passed | | +1 :green_heart: | compile | 4m 50s | the patch passed | | +1 :green_heart: | javac | 4m 50s | the patch passed | | +1 :green_heart: | checkstyle | 2m 9s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 48s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 4m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 65m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ddccab216a7d 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-20788) Write up a doc about how to rolling upgrade from 1.x to 2.x
[ https://issues.apache.org/jira/browse/HBASE-20788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464879#comment-17464879 ] Gary commented on HBASE-20788: -- we want to upgrade hbase from 1.2.6 to 2.3.7,but in ref guide we just found summary steps, can anyone help to show detailed steps? > Write up a doc about how to rolling upgrade from 1.x to 2.x > --- > > Key: HBASE-20788 > URL: https://issues.apache.org/jira/browse/HBASE-20788 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0-alpha-1 > > Attachments: HBASE-20788.master.001.patch, > HBASE-20788.master.002.patch, HBASE-20788.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3944: HBASE-26576
Apache-HBase commented on pull request #3944: URL: https://github.com/apache/hbase/pull/3944#issuecomment-1000624734 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 36s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 12m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 19s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 12m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 165m 40s | hbase-server in the patch passed. | | | | 210m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3944 | | JIRA Issue | HBASE-26576 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2ba4089bf036 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/testReport/ | | Max. process+thread count | 3586 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26079) Use StoreFileTracker when splitting and merging
[ https://issues.apache.org/jira/browse/HBASE-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26079. --- Fix Version/s: HBASE-26067 Hadoop Flags: Reviewed Resolution: Fixed > Use StoreFileTracker when splitting and merging > --- > > Key: HBASE-26079 > URL: https://issues.apache.org/jira/browse/HBASE-26079 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Wellington Chevreuil >Priority: Major > Fix For: HBASE-26067 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (HBASE-26079) Use StoreFileTracker when splitting and merging
[ https://issues.apache.org/jira/browse/HBASE-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-26079: --- > Use StoreFileTracker when splitting and merging > --- > > Key: HBASE-26079 > URL: https://issues.apache.org/jira/browse/HBASE-26079 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Duo Zhang >Assignee: Wellington Chevreuil >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-25398) Cleanup unused/unneeded temporarily HFiles
[ https://issues.apache.org/jira/browse/HBASE-25398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25398. --- Resolution: Won't Fix > Cleanup unused/unneeded temporarily HFiles > --- > > Key: HBASE-25398 > URL: https://issues.apache.org/jira/browse/HBASE-25398 > Project: HBase > Issue Type: Sub-task >Reporter: Tak-Lon (Stephen) Wu >Priority: Major > > Cleanup records and write a new cleaner to move unneeded files from data > directory into archives (other than compaction) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-25399) Write HBCK tools to rebuild the tracking
[
https://issues.apache.org/jira/browse/HBASE-25399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-25399.
---
Resolution: Won't Fix
> Write HBCK tools to rebuild the tracking
>
>
> Key: HBASE-25399
> URL: https://issues.apache.org/jira/browse/HBASE-25399
> Project: HBase
> Issue Type: Sub-task
>Reporter: Tak-Lon (Stephen) Wu
>Priority: Major
>
> {color:#00}support online region repairing, and maybe offline repairing
> as well
> {color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-24749) Direct insert HFiles and Persist in-memory HFile tracking
[
https://issues.apache.org/jira/browse/HBASE-24749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464862#comment-17464862
]
Duo Zhang commented on HBASE-24749:
---
I also resolved all the sub tasks as Won't fix. Feel free to reopen and link
them to other issues if they are still needed.
Thanks.
> Direct insert HFiles and Persist in-memory HFile tracking
> -
>
> Key: HBASE-24749
> URL: https://issues.apache.org/jira/browse/HBASE-24749
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, HFile
>Affects Versions: 3.0.0-alpha-1
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
> Labels: design, discussion, objectstore, storeFile, storeengine
> Attachments: 1B100m-25m25m-performance.pdf, Apache HBase - Direct
> insert HFiles and Persist in-memory HFile tracking.pdf
>
>
> We propose a new feature (a new store engine) to remove the {{.tmp}}
> directory used in the commit stage for common HFile operations such as flush
> and compaction to improve the write throughput and latency on object stores.
> Specifically for S3 filesystems, this will also mitigate read-after-write
> inconsistencies caused by immediate HFiles validation after moving the
> HFile(s) to data directory.
> Please see attached for this proposal and the initial result captured with
> 25m (25m operations) and 1B (100m operations) YCSB workload A LOAD and RUN,
> and workload C RUN result.
> The goal of this JIRA is to discuss with the community if the proposed
> improvement on the object stores use case makes senses and if we miss
> anything should be included.
> Improvement Highlights
> 1. Lower write latency, especially the p99+
> 2. Higher write throughput on flush and compaction
> 3. Lower MTTR on region (re)open or assignment
> 4. Remove consistent check dependencies (e.g. DynamoDB) supported by file
> system implementation
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HBASE-25394) Support Snapshot related operation with direct insert HFiles into data/CF directory
[
https://issues.apache.org/jira/browse/HBASE-25394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-25394.
---
Resolution: Won't Fix
> Support Snapshot related operation with direct insert HFiles into data/CF
> directory
> ---
>
> Key: HBASE-25394
> URL: https://issues.apache.org/jira/browse/HBASE-25394
> Project: HBase
> Issue Type: Sub-task
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Szabolcs Bukros
>Priority: Major
>
> {color:#00}Support restore snapshot, clone snapshot with direct insert
> into data directory{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HBASE-25396) Integration with direct insert into data/cf directory
[ https://issues.apache.org/jira/browse/HBASE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25396. --- Resolution: Won't Fix > Integration with direct insert into data/cf directory > - > > Key: HBASE-25396 > URL: https://issues.apache.org/jira/browse/HBASE-25396 > Project: HBase > Issue Type: Sub-task >Reporter: Tak-Lon (Stephen) Wu >Priority: Major > > Utilizes direct insert compactor, file writer, committer and etc while > constructing the PersistedStoreEngine -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-25397) Migrate hbase:storefile system table into editable meta
[
https://issues.apache.org/jira/browse/HBASE-25397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-25397.
---
Resolution: Won't Fix
> Migrate hbase:storefile system table into editable meta
> ---
>
> Key: HBASE-25397
> URL: https://issues.apache.org/jira/browse/HBASE-25397
> Project: HBase
> Issue Type: Sub-task
>Reporter: Tak-Lon (Stephen) Wu
>Priority: Major
>
> {color:#00}editable meta supported by HBASE-23055 allow us to write new
> column family to hbase:meta, such that we will introduce a new column family
> cf:storefile to hbase:meta with different trackable columns to served as the
> in-memory tracking of e.g. `storefiles` and `compactedfile` in
> StoreFileManager.{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HBASE-24749) Direct insert HFiles and Persist in-memory HFile tracking
[
https://issues.apache.org/jira/browse/HBASE-24749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-24749.
---
Resolution: Implemented
The feature has been implemented in HBASE-26067.
Resolve.
> Direct insert HFiles and Persist in-memory HFile tracking
> -
>
> Key: HBASE-24749
> URL: https://issues.apache.org/jira/browse/HBASE-24749
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, HFile
>Affects Versions: 3.0.0-alpha-1
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
> Labels: design, discussion, objectstore, storeFile, storeengine
> Attachments: 1B100m-25m25m-performance.pdf, Apache HBase - Direct
> insert HFiles and Persist in-memory HFile tracking.pdf
>
>
> We propose a new feature (a new store engine) to remove the {{.tmp}}
> directory used in the commit stage for common HFile operations such as flush
> and compaction to improve the write throughput and latency on object stores.
> Specifically for S3 filesystems, this will also mitigate read-after-write
> inconsistencies caused by immediate HFiles validation after moving the
> HFile(s) to data directory.
> Please see attached for this proposal and the initial result captured with
> 25m (25m operations) and 1B (100m operations) YCSB workload A LOAD and RUN,
> and workload C RUN result.
> The goal of this JIRA is to discuss with the community if the proposed
> improvement on the object stores use case makes senses and if we miss
> anything should be included.
> Improvement Highlights
> 1. Lower write latency, especially the p99+
> 2. Higher write throughput on flush and compaction
> 3. Lower MTTR on region (re)open or assignment
> 4. Remove consistent check dependencies (e.g. DynamoDB) supported by file
> system implementation
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3979: HBASE-26622 Update error-prone to 2.10
Apache-HBase commented on pull request #3979: URL: https://github.com/apache/hbase/pull/3979#issuecomment-1000606354 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 39s | master passed | | +1 :green_heart: | compile | 3m 19s | master passed | | +1 :green_heart: | shadedjars | 8m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 11s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 43s | the patch passed | | +1 :green_heart: | compile | 3m 21s | the patch passed | | +1 :green_heart: | javac | 3m 21s | the patch passed | | +1 :green_heart: | shadedjars | 8m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 192m 53s | root in the patch passed. | | | | 243m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3979 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8b975504c594 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/testReport/ | | Max. process+thread count | 6922 (vs. ulimit of 3) | | modules | C: hbase-build-configuration hbase-common hbase-asyncfs hbase-server hbase-mapreduce hbase-thrift hbase-endpoint . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000592436 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 3s | master passed | | +1 :green_heart: | compile | 2m 17s | master passed | | +1 :green_heart: | shadedjars | 9m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 3s | the patch passed | | +1 :green_heart: | compile | 2m 21s | the patch passed | | +1 :green_heart: | javac | 2m 21s | the patch passed | | +1 :green_heart: | shadedjars | 9m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 205m 34s | hbase-server in the patch passed. | | -1 :x: | unit | 8m 42s | hbase-thrift in the patch failed. | | | | 255m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3254d2457b2a 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-thrift.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/testReport/ | | Max. process+thread count | 2578 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000582086 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | compile | 2m 56s | master passed | | +1 :green_heart: | shadedjars | 11m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 54s | the patch passed | | +1 :green_heart: | compile | 2m 45s | the patch passed | | +1 :green_heart: | javac | 2m 45s | the patch passed | | +1 :green_heart: | shadedjars | 11m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 50s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 172m 41s | hbase-server in the patch passed. | | -1 :x: | unit | 7m 16s | hbase-thrift in the patch failed. | | | | 228m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux eeebb55026b8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-thrift.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/testReport/ | | Max. process+thread count | 3782 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3944: HBASE-26576
Apache-HBase commented on pull request #3944: URL: https://github.com/apache/hbase/pull/3944#issuecomment-1000577205 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | compile | 3m 13s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 1s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 3m 14s | the patch passed | | +1 :green_heart: | javac | 3m 14s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 4 new + 18 unchanged - 1 fixed = 22 total (was 19) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 49m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3944 | | JIRA Issue | HBASE-26576 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux fbb2ce293a75 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3944: HBASE-26576
Apache-HBase commented on pull request #3944: URL: https://github.com/apache/hbase/pull/3944#issuecomment-1000563734 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 51s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 9m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 46s | the patch passed | | +1 :green_heart: | compile | 1m 29s | the patch passed | | +1 :green_heart: | javac | 1m 29s | the patch passed | | +1 :green_heart: | shadedjars | 9m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 225m 0s | hbase-server in the patch passed. | | | | 263m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3944 | | JIRA Issue | HBASE-26576 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f2d4bb940058 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/testReport/ | | Max. process+thread count | 2596 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3979: HBASE-26622 Update error-prone to 2.10
Apache-HBase commented on pull request #3979: URL: https://github.com/apache/hbase/pull/3979#issuecomment-1000561000 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 8m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 57s | master passed | | +1 :green_heart: | spotbugs | 15m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | -1 :x: | compile | 0m 22s | root in the patch failed. | | -0 :warning: | javac | 0m 22s | root in the patch failed. | | -0 :warning: | checkstyle | 1m 58s | root: The patch generated 1 new + 84 unchanged - 1 fixed = 85 total (was 85) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 19m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 17m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 47s | The patch does not generate ASF License warnings. | | | | 87m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3979 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux eeb01cb91ce4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/artifact/yetus-general-check/output/patch-compile-root.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/artifact/yetus-general-check/output/patch-compile-root.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | Max. process+thread count | 141 (vs. ulimit of 3) | | modules | C: hbase-build-configuration hbase-common hbase-asyncfs hbase-server hbase-mapreduce hbase-thrift hbase-endpoint . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3944: HBASE-26576
Apache-HBase commented on pull request #3944: URL: https://github.com/apache/hbase/pull/3944#issuecomment-1000549760 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | master passed | | +1 :green_heart: | compile | 1m 8s | master passed | | +1 :green_heart: | shadedjars | 8m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | shadedjars | 9m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 174m 29s | hbase-server in the patch failed. | | | | 207m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3944 | | JIRA Issue | HBASE-26576 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3d62ae96511d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/testReport/ | | Max. process+thread count | 3729 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26590) Hbase-client Meta lookup performance regression between hbase-1 and hbase-2
[ https://issues.apache.org/jira/browse/HBASE-26590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464814#comment-17464814 ] Huaxiang Sun commented on HBASE-26590: -- I am modifying my test code to exclude the connection setup/teardown from the reported time (should not be there at the first place). Will report back when I have more testing results. > Hbase-client Meta lookup performance regression between hbase-1 and hbase-2 > --- > > Key: HBASE-26590 > URL: https://issues.apache.org/jira/browse/HBASE-26590 > Project: HBase > Issue Type: Improvement > Components: meta >Affects Versions: 2.4.0, 2.5.0, 2.3.7, 2.6.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Major > > One of our users complained higher latency after application upgrades from > hbase-1.2 client (CDH-5.16.2) to hbase-2.4.5 client with meta replica Load > Balance mode during app restart. I reproduced the regression by a test for > meta lookup. > At my test cluster, there are 160k regions for the test table, so there are > 160k entries in meta region. Used one thread to do 1 million meta lookup > against the meta region server. > > ||Version ||Meta Replica Load Balance Enabled||Time || > ||2.4.5-with-fixed||Yes||336458ms|| > ||2.4.5-with-fixed||No||333253ms|| > ||2.4.5||Yes||469980ms|| > ||2.4.5||No||470515ms|| > | *cdh-5.16.2*| *No* | *323412ms*| > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000538420 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 4m 49s | master passed | | +1 :green_heart: | checkstyle | 2m 8s | master passed | | +1 :green_heart: | spotbugs | 4m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 16s | the patch passed | | +1 :green_heart: | compile | 4m 51s | the patch passed | | +1 :green_heart: | javac | 4m 51s | the patch passed | | +1 :green_heart: | checkstyle | 2m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 17s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 4m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 63m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 0773bbba61d6 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000512111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
Apache-HBase commented on pull request #3980: URL: https://github.com/apache/hbase/pull/3980#issuecomment-1000511281 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 0s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 11s | master passed | | +1 :green_heart: | compile | 2m 21s | master passed | | +1 :green_heart: | shadedjars | 9m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 5s | the patch passed | | +1 :green_heart: | compile | 2m 19s | the patch passed | | +1 :green_heart: | javac | 2m 19s | the patch passed | | +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 12m 19s | hbase-server in the patch failed. | | -1 :x: | unit | 8m 38s | hbase-thrift in the patch failed. | | | | 62m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3980 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2f010185cc58 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-thrift.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/1/testReport/ | | Max. process+thread count | 1504 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server hbase-thrift U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3980/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3979: HBASE-26622 Update error-prone to 2.10
Apache-HBase commented on pull request #3979: URL: https://github.com/apache/hbase/pull/3979#issuecomment-1000509787 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 52s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 8m 57s | master passed | | +1 :green_heart: | checkstyle | 2m 2s | master passed | | +1 :green_heart: | spotbugs | 13m 23s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | -1 :x: | compile | 0m 24s | root in the patch failed. | | -0 :warning: | javac | 0m 24s | root in the patch failed. | | -0 :warning: | checkstyle | 2m 0s | root: The patch generated 1 new + 81 unchanged - 1 fixed = 82 total (was 82) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 19m 34s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 14m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 10s | The patch does not generate ASF License warnings. | | | | 81m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3979 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux f390188fd770 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-general-check/output/patch-compile-root.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-general-check/output/patch-compile-root.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | Max. process+thread count | 141 (vs. ulimit of 3) | | modules | C: hbase-build-configuration hbase-common hbase-asyncfs hbase-server . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3979: HBASE-26622 Update error-prone to 2.10
Apache-HBase commented on pull request #3979: URL: https://github.com/apache/hbase/pull/3979#issuecomment-1000507376 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 54s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 10s | master passed | | +1 :green_heart: | compile | 3m 34s | master passed | | +1 :green_heart: | shadedjars | 9m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 3m 39s | the patch passed | | +1 :green_heart: | javac | 3m 38s | the patch passed | | +1 :green_heart: | shadedjars | 9m 26s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 53s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 23m 51s | root in the patch failed. | | | | 74m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3979 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0b86e1f59970 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/testReport/ | | Max. process+thread count | 838 (vs. ulimit of 3) | | modules | C: hbase-build-configuration hbase-common hbase-asyncfs hbase-server . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3979: HBASE-26622 Update error-prone to 2.10
Apache-HBase commented on pull request #3979: URL: https://github.com/apache/hbase/pull/3979#issuecomment-1000505585 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 1m 6s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | compile | 2m 49s | master passed | | +1 :green_heart: | shadedjars | 8m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 41s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 2m 50s | the patch passed | | +1 :green_heart: | javac | 2m 50s | the patch passed | | +1 :green_heart: | shadedjars | 8m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 36s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 28m 5s | root in the patch failed. | | | | 69m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3979 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c4bc3f5a461b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/testReport/ | | Max. process+thread count | 1122 (vs. ulimit of 3) | | modules | C: hbase-build-configuration hbase-common hbase-asyncfs hbase-server . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3979/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3944: HBASE-26576
Apache-HBase commented on pull request #3944: URL: https://github.com/apache/hbase/pull/3944#issuecomment-1000498944 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 3m 11s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 7s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 3m 12s | the patch passed | | +1 :green_heart: | javac | 3m 12s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 4 new + 18 unchanged - 1 fixed = 22 total (was 19) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 49m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3944 | | JIRA Issue | HBASE-26576 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1484465320f5 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 70703ca81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3944/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] rmarsch opened a new pull request #3980: HBASE-26623 Report CallDroppedException in exception metrics
rmarsch opened a new pull request #3980: URL: https://github.com/apache/hbase/pull/3980 `CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons. It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side. https://issues.apache.org/jira/browse/HBASE-26623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26623) Report CallDroppedException in exception metrics
Richard Marscher created HBASE-26623: Summary: Report CallDroppedException in exception metrics Key: HBASE-26623 URL: https://issues.apache.org/jira/browse/HBASE-26623 Project: HBase Issue Type: Improvement Components: IPC/RPC, metrics, rpc Reporter: Richard Marscher Assignee: Richard Marscher `CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons (my use case). It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side. I have a patch I will submit soon. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] d-c-manning commented on a change in pull request #3972: HBASE-26596 region_mover should gracefully ignore null response from RSGroupAdmin#getRSGroupOfServer
d-c-manning commented on a change in pull request #3972:
URL: https://github.com/apache/hbase/pull/3972#discussion_r774737883
##
File path: bin/region_mover.rb
##
@@ -493,11 +493,22 @@ def getFilename(options, targetServer, port)
return filename
end
+def getServersRSGroup(rsgroup_admin, hostname, port)
+ rsgroup = rsgroup_admin.getRSGroupOfServer(Address.fromParts(hostname,
+ java.lang.Integer.parseInt(port)))
+ # If the rsgroup is nil, that means this server belongs to no rsgroup.
+ # It should be already offline.
+ # Here we directly log and exit.
+ return rsgroup unless rsgroup.nil?
+ $LOG.info('The server ' + hostname + 'belongs to no rsgroup. Exit regions
moving.')
+ exit 0
Review comment:
The code and syntax looks good to me
>Here, I see one minor issue: if we go with this change, still the unload
region would not make a graceful exit:
Can we instead call rsgroupAdmin.getRSGroupOfServer outside of
getSameRSGroupServers method and if it returns nil, we can log a message and
then exit gracefully?
I do question this design decision, though. If a server has regions on it,
but fails to find any target servers to unload to, then that seems like a
reason to exit with an error code.
So then I would propose that we explicitly check for this case. It's okay to
exit with 0 if the server has no rsgroup AND has no regions. This would
obviously require additional refactoring of the code, since currently we don't
check for regions until after we obtain the rsgroup of the server. And in the
case mentioned in HBASE-26596, I don't know whether a call to
`getOnlineRegions` would succeed.
This is only my opinion, so feel free to ignore if you feel differently 🙂
##
File path: bin/region_mover.rb
##
@@ -493,11 +493,22 @@ def getFilename(options, targetServer, port)
return filename
end
+def getServersRSGroup(rsgroup_admin, hostname, port)
+ rsgroup = rsgroup_admin.getRSGroupOfServer(Address.fromParts(hostname,
+ java.lang.Integer.parseInt(port)))
+ # If the rsgroup is nil, that means this server belongs to no rsgroup.
+ # It should be already offline.
+ # Here we directly log and exit.
+ return rsgroup unless rsgroup.nil?
+ $LOG.info('The server ' + hostname + 'belongs to no rsgroup. Exit regions
moving.')
Review comment:
This feels like it justifies at least a `warn` level log
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (HBASE-26622) Update to error-prone 2.10
Mike Drob created HBASE-26622: - Summary: Update to error-prone 2.10 Key: HBASE-26622 URL: https://issues.apache.org/jira/browse/HBASE-26622 Project: HBase Issue Type: Task Reporter: Mike Drob -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26609) Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad
[
https://issues.apache.org/jira/browse/HBASE-26609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464748#comment-17464748
]
Hudson commented on HBASE-26609:
Results for branch branch-2
[build #424 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Round the size to MB or KB at the end of calculation in
> HRegionServer.createRegionLoad
> --
>
> Key: HBASE-26609
> URL: https://issues.apache.org/jira/browse/HBASE-26609
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> This is a follow up issue of HBASE-26340.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26613) The logic of the method incrementIV in Encryption class has problem
[
https://issues.apache.org/jira/browse/HBASE-26613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464746#comment-17464746
]
Hudson commented on HBASE-26613:
Results for branch branch-2
[build #424 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The logic of the method incrementIV in Encryption class has problem
> ---
>
> Key: HBASE-26613
> URL: https://issues.apache.org/jira/browse/HBASE-26613
> Project: HBase
> Issue Type: Bug
> Components: Performance, security
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> {code:java}
> public static void incrementIv(byte[] iv, int v) {
> int length = iv.length;
> boolean carry = true;
> // TODO: Optimize for v > 1, e.g. 16, 32
> do {
> for (int i = 0; i < length; i++) {
> if (carry) {
> iv[i] = (byte) ((iv[i] + 1) & 0xFF);
> carry = 0 == iv[i];
> } else {
> break;
> }
> }
> v--;
> } while (v > 0);
> }
> {code}
> The carry flag is outside the while loop and the inner loop check the carry
> value is true, so that when it was set to false, it will make the rest
> iterations busy spinning.
> And the logic becomes whatever the v is, the IV will be incremented by (1 or
> 2). As the description in todo, this function should support increment value
> > 1. So that this is not what we expect I think.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26598) Fix excessive connections in MajorCompactor
[
https://issues.apache.org/jira/browse/HBASE-26598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464747#comment-17464747
]
Hudson commented on HBASE-26598:
Results for branch branch-2
[build #424 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/424/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix excessive connections in MajorCompactor
> ---
>
> Key: HBASE-26598
> URL: https://issues.apache.org/jira/browse/HBASE-26598
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, tooling
>Reporter: Muhammad Samir Khan
>Assignee: Muhammad Samir Khan
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> MajorCompactor creates excessive connections: once every time
> MajorCompactionRequest.getConnection is called. Can do better by using the
> Connection object created in MajorCompactor.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3978: HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master
Apache-HBase commented on pull request #3978: URL: https://github.com/apache/hbase/pull/3978#issuecomment-1000459167 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 54s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 31s | master passed | | +1 :green_heart: | compile | 3m 1s | master passed | | +1 :green_heart: | shadedjars | 9m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 13m 33s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +0 :ok: | mvndep | 4m 34s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 15s | the patch passed | | +1 :green_heart: | compile | 2m 59s | the patch passed | | +1 :green_heart: | javac | 2m 59s | the patch passed | | +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 13m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 427m 5s | root in the patch passed. | | | | 506m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3978 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 07244e9da979 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/testReport/ | | Max. process+thread count | 4165 (vs. ulimit of 3) | | modules | C: hbase-checkstyle hbase-annotations hbase-build-configuration hbase-logging hbase-protocol-shaded hbase-common hbase-metrics-api hbase-metrics hbase-hadoop-compat hbase-client hbase-zookeeper hbase-replication hbase-balancer hbase-resource-bundle hbase-http hbase-asyncfs hbase-procedure hbase-server hbase-mapreduce hbase-testing-util hbase-thrift hbase-shell hbase-endpoint hbase-backup hbase-it hbase-rest hbase-examples hbase-shaded hbase-shaded/hbase-shaded-client hbase-shaded/hbase-shaded-client-byo-hadoop hbase-shaded/hbase-shaded-mapreduce hbase-external-blockcache hbase-hbtop hbase-compression hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd hbase-assembly hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase- shaded-with-hadoop-check-invariants hbase-archetypes hbase-archetypes/hbase-client-project hbase-archetypes/hbase-shaded-client-project hbase-archetypes/hbase-archetype-builder . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26598) Fix excessive connections in MajorCompactor
[
https://issues.apache.org/jira/browse/HBASE-26598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464629#comment-17464629
]
Hudson commented on HBASE-26598:
Results for branch branch-2.5
[build #9 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix excessive connections in MajorCompactor
> ---
>
> Key: HBASE-26598
> URL: https://issues.apache.org/jira/browse/HBASE-26598
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, tooling
>Reporter: Muhammad Samir Khan
>Assignee: Muhammad Samir Khan
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> MajorCompactor creates excessive connections: once every time
> MajorCompactionRequest.getConnection is called. Can do better by using the
> Connection object created in MajorCompactor.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26609) Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad
[
https://issues.apache.org/jira/browse/HBASE-26609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464630#comment-17464630
]
Hudson commented on HBASE-26609:
Results for branch branch-2.5
[build #9 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Round the size to MB or KB at the end of calculation in
> HRegionServer.createRegionLoad
> --
>
> Key: HBASE-26609
> URL: https://issues.apache.org/jira/browse/HBASE-26609
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> This is a follow up issue of HBASE-26340.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26613) The logic of the method incrementIV in Encryption class has problem
[
https://issues.apache.org/jira/browse/HBASE-26613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464628#comment-17464628
]
Hudson commented on HBASE-26613:
Results for branch branch-2.5
[build #9 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.5/9/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The logic of the method incrementIV in Encryption class has problem
> ---
>
> Key: HBASE-26613
> URL: https://issues.apache.org/jira/browse/HBASE-26613
> Project: HBase
> Issue Type: Bug
> Components: Performance, security
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> {code:java}
> public static void incrementIv(byte[] iv, int v) {
> int length = iv.length;
> boolean carry = true;
> // TODO: Optimize for v > 1, e.g. 16, 32
> do {
> for (int i = 0; i < length; i++) {
> if (carry) {
> iv[i] = (byte) ((iv[i] + 1) & 0xFF);
> carry = 0 == iv[i];
> } else {
> break;
> }
> }
> v--;
> } while (v > 0);
> }
> {code}
> The carry flag is outside the while loop and the inner loop check the carry
> value is true, so that when it was set to false, it will make the rest
> iterations busy spinning.
> And the logic becomes whatever the v is, the IV will be incremented by (1 or
> 2). As the description in todo, this function should support increment value
> > 1. So that this is not what we expect I think.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
Apache-HBase commented on pull request #3800: URL: https://github.com/apache/hbase/pull/3800#issuecomment-1000348074 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 38s | master passed | | +1 :green_heart: | compile | 2m 10s | master passed | | +1 :green_heart: | shadedjars | 10m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 6m 33s | the patch passed | | +1 :green_heart: | compile | 2m 19s | the patch passed | | +1 :green_heart: | javac | 2m 19s | the patch passed | | +1 :green_heart: | shadedjars | 11m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 0m 38s | hbase-asyncfs in the patch failed. | | -1 :x: | unit | 228m 54s | hbase-server in the patch failed. | | | | 283m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3800 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ff4d696415b4 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-asyncfs.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/testReport/ | | Max. process+thread count | 2363 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3978: HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master
Apache-HBase commented on pull request #3978: URL: https://github.com/apache/hbase/pull/3978#issuecomment-1000340602 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 40s | master passed | | +1 :green_heart: | compile | 3m 13s | master passed | | +1 :green_heart: | shadedjars | 8m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 17m 2s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 26s | master passed | | +0 :ok: | mvndep | 4m 50s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 33s | the patch passed | | +1 :green_heart: | compile | 3m 14s | the patch passed | | +1 :green_heart: | javac | 3m 14s | the patch passed | | +1 :green_heart: | shadedjars | 8m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 17m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 197m 20s | root in the patch passed. | | | | 280m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3978 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 219c2a2b5194 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/testReport/ | | Max. process+thread count | 6534 (vs. ulimit of 3) | | modules | C: hbase-checkstyle hbase-annotations hbase-build-configuration hbase-logging hbase-protocol-shaded hbase-common hbase-metrics-api hbase-metrics hbase-hadoop-compat hbase-client hbase-zookeeper hbase-replication hbase-balancer hbase-resource-bundle hbase-http hbase-asyncfs hbase-procedure hbase-server hbase-mapreduce hbase-testing-util hbase-thrift hbase-shell hbase-endpoint hbase-backup hbase-it hbase-rest hbase-examples hbase-shaded hbase-shaded/hbase-shaded-client hbase-shaded/hbase-shaded-client-byo-hadoop hbase-shaded/hbase-shaded-mapreduce hbase-external-blockcache hbase-hbtop hbase-compression hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd hbase-assembly hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase- shaded-with-hadoop-check-invariants hbase-archetypes hbase-archetypes/hbase-client-project hbase-archetypes/hbase-shaded-client-project hbase-archetypes/hbase-archetype-builder . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26620) Add 3.0.0-alpha-2 to download page
[ https://issues.apache.org/jira/browse/HBASE-26620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26620. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~psomogyi] for reviewing! > Add 3.0.0-alpha-2 to download page > -- > > Key: HBASE-26620 > URL: https://issues.apache.org/jira/browse/HBASE-26620 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache9 merged pull request #3977: HBASE-26620 Add 3.0.0-alpha-2 to download page
Apache9 merged pull request #3977: URL: https://github.com/apache/hbase/pull/3977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26328) Clone snapshot doesn't load reference files into FILE SFT impl
[
https://issues.apache.org/jira/browse/HBASE-26328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464601#comment-17464601
]
Hudson commented on HBASE-26328:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Clone snapshot doesn't load reference files into FILE SFT impl
> --
>
> Key: HBASE-26328
> URL: https://issues.apache.org/jira/browse/HBASE-26328
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, snapshots
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> After cloning a snapshot from a FILE SFT enabled table, noticed that none of
> the cloned table files were added into the FILE SFT meta files, in fact, FILE
> SFT meta dir didn't even got created. Scanning this cloned table gives no
> results, as none of the files are tracked. I believe we need to call
> StoreFileTracker.add during the snapshot cloning.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26265) Update ref guide to mention the new store file tracker implementations
[
https://issues.apache.org/jira/browse/HBASE-26265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464606#comment-17464606
]
Hudson commented on HBASE-26265:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Update ref guide to mention the new store file tracker implementations
> --
>
> Key: HBASE-26265
> URL: https://issues.apache.org/jira/browse/HBASE-26265
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Duo Zhang
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: HBASE-26067
>
>
> For example, when to use these store file trackers.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26280) Use store file tracker when snapshoting
[
https://issues.apache.org/jira/browse/HBASE-26280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464598#comment-17464598
]
Hudson commented on HBASE-26280:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use store file tracker when snapshoting
> ---
>
> Key: HBASE-26280
> URL: https://issues.apache.org/jira/browse/HBASE-26280
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, snapshots
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26598) Fix excessive connections in MajorCompactor
[
https://issues.apache.org/jira/browse/HBASE-26598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464607#comment-17464607
]
Hudson commented on HBASE-26598:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix excessive connections in MajorCompactor
> ---
>
> Key: HBASE-26598
> URL: https://issues.apache.org/jira/browse/HBASE-26598
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, tooling
>Reporter: Muhammad Samir Khan
>Assignee: Muhammad Samir Khan
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> MajorCompactor creates excessive connections: once every time
> MajorCompactionRequest.getConnection is called. Can do better by using the
> Connection object created in MajorCompactor.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26609) Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad
[
https://issues.apache.org/jira/browse/HBASE-26609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464608#comment-17464608
]
Hudson commented on HBASE-26609:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Round the size to MB or KB at the end of calculation in
> HRegionServer.createRegionLoad
> --
>
> Key: HBASE-26609
> URL: https://issues.apache.org/jira/browse/HBASE-26609
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> This is a follow up issue of HBASE-26340.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26454) CreateTableProcedure still relies on temp dir and renames when creating table FS layout
[
https://issues.apache.org/jira/browse/HBASE-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464604#comment-17464604
]
Hudson commented on HBASE-26454:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CreateTableProcedure still relies on temp dir and renames when creating table
> FS layout
> ---
>
> Key: HBASE-26454
> URL: https://issues.apache.org/jira/browse/HBASE-26454
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: HBASE-26067
>
>
> As pointed by [~elserj] when reviewing some other issue, internally,
> CreateTableProcedure still creates the FS layout on temp folder first, then
> rename to actual dir. I think this is unnecessary, we could create it
> straight in the final dir, just like we did in [CloneSnapshotProcedure
> here|https://github.com/apache/hbase/pull/3749/files#r728424375].
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26386) Refactor StoreFileTracker implementations to expose the set method
[
https://issues.apache.org/jira/browse/HBASE-26386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464600#comment-17464600
]
Hudson commented on HBASE-26386:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor StoreFileTracker implementations to expose the set method
> --
>
> Key: HBASE-26386
> URL: https://issues.apache.org/jira/browse/HBASE-26386
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
> As when clone/restore a snapshot, we need to set the store file list directly
> to StoreFileTracker, which means we need this method in upper layer.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[
https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464602#comment-17464602
]
Hudson commented on HBASE-26263:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Rolling Upgrading] Persist the StoreFileTracker configurations to
> TableDescriptor for existing tables
> --
>
> Key: HBASE-26263
> URL: https://issues.apache.org/jira/browse/HBASE-26263
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, master
>Reporter: Duo Zhang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: HBASE-26067
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26224) Introduce a MigrationStoreFileTracker to support migrating from different store file tracker implementations
[
https://issues.apache.org/jira/browse/HBASE-26224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464594#comment-17464594
]
Hudson commented on HBASE-26224:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce a MigrationStoreFileTracker to support migrating from different
> store file tracker implementations
>
>
> Key: HBASE-26224
> URL: https://issues.apache.org/jira/browse/HBASE-26224
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
> As mentioned in this section in the design doc:
> https://docs.google.com/document/d/16Nr1Fn3VaXuz1g1FTiME-bnGR3qVK5B-raXshOkDLcY/edit#heading=h.vv41957yf4i
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26248) Should find a suitable way to let users specify the store file tracker implementation
[
https://issues.apache.org/jira/browse/HBASE-26248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464596#comment-17464596
]
Hudson commented on HBASE-26248:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should find a suitable way to let users specify the store file tracker
> implementation
> -
>
> Key: HBASE-26248
> URL: https://issues.apache.org/jira/browse/HBASE-26248
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
> Now all the implementations are marked as IA.Private, so there is no safe way
> for users to specify them.
> But for simplify, maybe we should not expose the full class name of the
> implementation classes to users. Just follow the way in WALFactory, to
> introduce an alias name for the implemention, for example
> DEFAULT, FILE, REGION, etc.
> Thoughts?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25988) Store the store file list by a file
[
https://issues.apache.org/jira/browse/HBASE-25988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464592#comment-17464592
]
Hudson commented on HBASE-25988:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Store the store file list by a file
> ---
>
> Key: HBASE-25988
> URL: https://issues.apache.org/jira/browse/HBASE-25988
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26264) Add more checks to prevent misconfiguration on store file tracker
[
https://issues.apache.org/jira/browse/HBASE-26264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464597#comment-17464597
]
Hudson commented on HBASE-26264:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add more checks to prevent misconfiguration on store file tracker
> -
>
> Key: HBASE-26264
> URL: https://issues.apache.org/jira/browse/HBASE-26264
> Project: HBase
> Issue Type: Sub-task
> Components: conf, HFile
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
> For example, when creating new tables, if the store file tracker is
> configured to MigrationStoreFileTracker, we should fail the creation and tell
> users you should not use this store file tracker for a new table.
> And when switching the implementation for store file tracker, we should let
> end users first switch to MigrationStoreFileTracker, and then to the final
> target store file tracker. If end users configure wrong store file tracker,
> we should fail the alter table operation.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26271) Cleanup the broken store files under data directory
[
https://issues.apache.org/jira/browse/HBASE-26271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464603#comment-17464603
]
Hudson commented on HBASE-26271:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Cleanup the broken store files under data directory
> ---
>
> Key: HBASE-26271
> URL: https://issues.apache.org/jira/browse/HBASE-26271
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: Duo Zhang
>Assignee: Szabolcs Bukros
>Priority: Major
> Fix For: HBASE-26067
>
>
> As for some new store file tracker implementation, we allow flush/compaction
> to write directly to data directory, so if we crash in the middle, there will
> be broken store files left in the data directory.
> We should find a proper way to delete these broken files.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26326) CreateTableProcedure fails when FileBasedStoreFileTracker is set in global config
[
https://issues.apache.org/jira/browse/HBASE-26326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464599#comment-17464599
]
Hudson commented on HBASE-26326:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CreateTableProcedure fails when FileBasedStoreFileTracker is set in global
> config
> -
>
> Key: HBASE-26326
> URL: https://issues.apache.org/jira/browse/HBASE-26326
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> CreateTableProcedure tries to instantiate SFT impl in its preCreate method so
> that the given SFT impl can update the tbl descriptor with all its required
> configs. When global config is set to FILE, FileBasedStoreFileTracker
> currently needs store dir path at its construction time (via the passed
> StoreContext param), but we don't have this info at this stage, so
> FileBasedStoreFileTracker get's an NPE:
> {noformat}
> 2021-10-01 14:28:21,310 ERROR
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=54, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION,
> locked=true; CreateTableProcedure table=test-3
> java.lang.UnsupportedOperationException: Constructor threw an exception for
> org.apache.hadoop.hbase.regionserver.storefiletracker.MigrationStoreFileTracker
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:66)
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.newInstance(ReflectionUtils.java:72)
> at
> org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerFactory.create(StoreFileTrackerFactory.java:107)
> at
> org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerFactory.persistTrackerConfig(StoreFileTrackerFactory.java:164)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.preCreate(CreateTableProcedure.java:274)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
> at
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:58)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:58)
> ... 12 more
> Caused by: java.lang.UnsupportedOperationException: Constructor threw an
> exception for
> org.apache.hadoop.hbase.regionserver.storefiletracker.FileBasedStoreFileTracker
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:66)
> at
> org.apache.hadoop.hbase.util.ReflectionUtils.newInstance(ReflectionUtils.java:72)
> at
> org.apache.hadoop.hbas
[jira] [Commented] (HBASE-26064) Introduce a StoreFileTracker to abstract the store file tracking logic
[
https://issues.apache.org/jira/browse/HBASE-26064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464591#comment-17464591
]
Hudson commented on HBASE-26064:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce a StoreFileTracker to abstract the store file tracking logic
> --
>
> Key: HBASE-26064
> URL: https://issues.apache.org/jira/browse/HBASE-26064
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: HBASE-26067
>
>
> As mentioned in the parent issue, we could abstract a storefile metadata
> system, then we could use different backend to store the storefile list, and
> also it is possible to generalize the migration between different tracking
> system.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26286) Add support for specifying store file tracker when restoring or cloning snapshot
[
https://issues.apache.org/jira/browse/HBASE-26286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464605#comment-17464605
]
Hudson commented on HBASE-26286:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add support for specifying store file tracker when restoring or cloning
> snapshot
>
>
> Key: HBASE-26286
> URL: https://issues.apache.org/jira/browse/HBASE-26286
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, snapshots
>Reporter: Duo Zhang
>Assignee: Szabolcs Bukros
>Priority: Major
> Fix For: HBASE-26067
>
>
> As discussed in HBASE-26280.
> https://issues.apache.org/jira/browse/HBASE-26280?focusedCommentId=17414894&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17414894
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26246) Persist the StoreFileTracker configurations to TableDescriptor when creating table
[
https://issues.apache.org/jira/browse/HBASE-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464595#comment-17464595
]
Hudson commented on HBASE-26246:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Persist the StoreFileTracker configurations to TableDescriptor when creating
> table
> --
>
> Key: HBASE-26246
> URL: https://issues.apache.org/jira/browse/HBASE-26246
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, master
>Reporter: Duo Zhang
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: HBASE-26067
>
>
> As discussed in this section in the design doc:
> https://docs.google.com/document/d/16Nr1Fn3VaXuz1g1FTiME-bnGR3qVK5B-raXshOkDLcY/edit#heading=h.78r2mdeyquug
> If we use different SFT implementation at master side and region server side,
> it is likely to cause data loss, which is a very serious misconfiguration
> problem.
> A possible solution is to make sure that master and region server always load
> the configurations about StoreEngine from the same place. To archive this, a
> possible way is to always set the StoreEngine configurations to the
> TableDescriptor, even if user does not explicitly set it when creating a
> table.
> And also, when upgrading, we should check whether the existing tables have
> StoreEngine configurations, if not, we need to set them.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26079) Use StoreFileTracker when splitting and merging
[
https://issues.apache.org/jira/browse/HBASE-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464593#comment-17464593
]
Hudson commented on HBASE-26079:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use StoreFileTracker when splitting and merging
> ---
>
> Key: HBASE-26079
> URL: https://issues.apache.org/jira/browse/HBASE-26079
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Wellington Chevreuil
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26613) The logic of the method incrementIV in Encryption class has problem
[
https://issues.apache.org/jira/browse/HBASE-26613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464590#comment-17464590
]
Hudson commented on HBASE-26613:
Results for branch master
[build #475 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/475/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The logic of the method incrementIV in Encryption class has problem
> ---
>
> Key: HBASE-26613
> URL: https://issues.apache.org/jira/browse/HBASE-26613
> Project: HBase
> Issue Type: Bug
> Components: Performance, security
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> {code:java}
> public static void incrementIv(byte[] iv, int v) {
> int length = iv.length;
> boolean carry = true;
> // TODO: Optimize for v > 1, e.g. 16, 32
> do {
> for (int i = 0; i < length; i++) {
> if (carry) {
> iv[i] = (byte) ((iv[i] + 1) & 0xFF);
> carry = 0 == iv[i];
> } else {
> break;
> }
> }
> v--;
> } while (v > 0);
> }
> {code}
> The carry flag is outside the while loop and the inner loop check the carry
> value is true, so that when it was set to false, it will make the rest
> iterations busy spinning.
> And the logic becomes whatever the v is, the IV will be incremented by (1 or
> 2). As the description in todo, this function should support increment value
> > 1. So that this is not what we expect I think.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
Apache-HBase commented on pull request #3800: URL: https://github.com/apache/hbase/pull/3800#issuecomment-1000286880 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 3s | master passed | | +1 :green_heart: | compile | 1m 26s | master passed | | +1 :green_heart: | shadedjars | 8m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 8m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 54s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 0m 27s | hbase-asyncfs in the patch failed. | | -1 :x: | unit | 150m 3s | hbase-server in the patch failed. | | | | 183m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3800 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fddd99dd18a9 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-asyncfs.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/testReport/ | | Max. process+thread count | 3639 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3976: HBASE-26615 Snapshot referenced data files are deleted when delete a …
Apache-HBase commented on pull request #3976: URL: https://github.com/apache/hbase/pull/3976#issuecomment-1000281040 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 11s | branch-2 passed | | +1 :green_heart: | compile | 1m 4s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 1m 16s | the patch passed | | +1 :green_heart: | javac | 1m 16s | the patch passed | | +1 :green_heart: | shadedjars | 7m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 160m 50s | hbase-server in the patch passed. | | | | 189m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3976 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3d7a3601f1cf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / c6ac99684c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/testReport/ | | Max. process+thread count | 4286 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3976: HBASE-26615 Snapshot referenced data files are deleted when delete a …
Apache-HBase commented on pull request #3976: URL: https://github.com/apache/hbase/pull/3976#issuecomment-1000269526 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | branch-2 passed | | +1 :green_heart: | compile | 1m 11s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 7m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 134m 16s | hbase-server in the patch passed. | | | | 165m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3976 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 44bfcb532ef8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / c6ac99684c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/testReport/ | | Max. process+thread count | 4304 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3971: HBASE-26615 Snapshot referenced data files are deleted when delete a …
Apache-HBase commented on pull request #3971: URL: https://github.com/apache/hbase/pull/3971#issuecomment-1000249274 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 41s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 214m 57s | hbase-server in the patch passed. | | | | 248m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3971/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3971 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f0bb0d2b8fb5 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / dbdef5982a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3971/3/testReport/ | | Max. process+thread count | 2381 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3971/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3978: HBASE-26621 Set version as 3.0.0-alpha-3-SNAPSHOT in master
Apache-HBase commented on pull request #3978: URL: https://github.com/apache/hbase/pull/3978#issuecomment-1000230144 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 10m 15s | master passed | | +1 :green_heart: | checkstyle | 2m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | master passed | | +0 :ok: | mvndep | 4m 44s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | +1 :green_heart: | compile | 9m 47s | the patch passed | | +1 :green_heart: | javac | 9m 47s | the patch passed | | +1 :green_heart: | checkstyle | 2m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 58s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 21m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 8m 33s | The patch does not generate ASF License warnings. | | | | 85m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3978 | | Optional Tests | dupname asflicense javac hadoopcheck xml compile checkstyle | | uname | Linux 4e73133d115d 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 126 (vs. ulimit of 3) | | modules | C: hbase-checkstyle hbase-annotations hbase-build-configuration hbase-logging hbase-protocol-shaded hbase-common hbase-metrics-api hbase-metrics hbase-hadoop-compat hbase-client hbase-zookeeper hbase-replication hbase-balancer hbase-resource-bundle hbase-http hbase-asyncfs hbase-procedure hbase-server hbase-mapreduce hbase-testing-util hbase-thrift hbase-shell hbase-endpoint hbase-backup hbase-it hbase-rest hbase-examples hbase-shaded hbase-shaded/hbase-shaded-client hbase-shaded/hbase-shaded-client-byo-hadoop hbase-shaded/hbase-shaded-mapreduce hbase-external-blockcache hbase-hbtop hbase-compression hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd hbase-assembly hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase- shaded-with-hadoop-check-invariants hbase-archetypes hbase-archetypes/hbase-client-project hbase-archetypes/hbase-shaded-client-project hbase-archetypes/hbase-archetype-builder . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3978/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26615) Snapshot referenced data files are deleted when delete a table with merge regions
[ https://issues.apache.org/jira/browse/HBASE-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464470#comment-17464470 ] Yi Mei commented on HBASE-26615: Pushed to branch-2.4+. Thanks [~zhangduo] and [~huangzhuoyue] for reviewing. > Snapshot referenced data files are deleted when delete a table with merge > regions > - > > Key: HBASE-26615 > URL: https://issues.apache.org/jira/browse/HBASE-26615 > Project: HBase > Issue Type: Bug >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10 > > > In our cluster, we have a feature: take a snapshot when delete a table. > But when we restore the snapshot, we found that some data files are deleted. > The problem is that, when delete a table with merge regions, HBase only > archive regions in meta, and merged parent regions are deleted in file system > which contain data files in snapshot . > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26615) Snapshot referenced data files are deleted when delete a table with merge regions
[ https://issues.apache.org/jira/browse/HBASE-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei resolved HBASE-26615. Fix Version/s: 2.5.0 3.0.0-alpha-3 2.4.10 Resolution: Fixed > Snapshot referenced data files are deleted when delete a table with merge > regions > - > > Key: HBASE-26615 > URL: https://issues.apache.org/jira/browse/HBASE-26615 > Project: HBase > Issue Type: Bug >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10 > > > In our cluster, we have a feature: take a snapshot when delete a table. > But when we restore the snapshot, we found that some data files are deleted. > The problem is that, when delete a table with merge regions, HBase only > archive regions in meta, and merged parent regions are deleted in file system > which contain data files in snapshot . > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
Apache-HBase commented on pull request #3800: URL: https://github.com/apache/hbase/pull/3800#issuecomment-1000227853 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 6s | master passed | | +1 :green_heart: | compile | 4m 36s | master passed | | +1 :green_heart: | checkstyle | 1m 50s | master passed | | +1 :green_heart: | spotbugs | 3m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 21s | the patch passed | | +1 :green_heart: | compile | 4m 50s | the patch passed | | -0 :warning: | javac | 0m 41s | hbase-asyncfs generated 2 new + 22 unchanged - 7 fixed = 24 total (was 29) | | +1 :green_heart: | checkstyle | 0m 14s | hbase-asyncfs: The patch generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) | | +1 :green_heart: | checkstyle | 1m 18s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 27m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 5m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 72m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3800 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux aa4899f0a19e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/artifact/yetus-general-check/output/diff-compile-javac-hbase-asyncfs.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3800/16/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mymeiyi merged pull request #3976: HBASE-26615 Snapshot referenced data files are deleted when delete a …
mymeiyi merged pull request #3976: URL: https://github.com/apache/hbase/pull/3976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3971: HBASE-26615 Snapshot referenced data files are deleted when delete a …
Apache-HBase commented on pull request #3971: URL: https://github.com/apache/hbase/pull/3971#issuecomment-1000205871 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 9m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 48s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 8m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 145m 59s | hbase-server in the patch passed. | | | | 180m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3971/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3971 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1094a1897b7c 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / dbdef5982a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3971/3/testReport/ | | Max. process+thread count | 3745 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3971/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3976: HBASE-26615 Snapshot referenced data files are deleted when delete a …
Apache-HBase commented on pull request #3976: URL: https://github.com/apache/hbase/pull/3976#issuecomment-1000200743 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | branch-2 passed | | +1 :green_heart: | compile | 3m 35s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 17s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 25s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 3m 47s | the patch passed | | +1 :green_heart: | javac | 3m 47s | the patch passed | | -0 :warning: | checkstyle | 1m 28s | hbase-server: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 49m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3976 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a3d0cef74d72 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / c6ac99684c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 86 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3976/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3977: HBASE-26620 Add 3.0.0-alpha-2 to download page
Apache-HBase commented on pull request #3977: URL: https://github.com/apache/hbase/pull/3977#issuecomment-1000191869 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +1 :green_heart: | mvnsite | 9m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | mvnsite | 9m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 29m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3977/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3977 | | Optional Tests | dupname asflicense mvnsite xml | | uname | Linux 6bee2064c24a 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Max. process+thread count | 89 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3977/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
Apache9 commented on a change in pull request #3800:
URL: https://github.com/apache/hbase/pull/3800#discussion_r774459962
##
File path:
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/monitor/StreamSlowMonitor.java
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.asyncfs.monitor;
+
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_EXCLUDE_DATANODE_TTL;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_EXCLUDE_DATANODE_TTL_KEY;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT_KEY;
+
+import java.util.Deque;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.TimeUnit;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.conf.ConfigurationObserver;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheBuilder;
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader;
+import org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache;
+
+/**
+ * Class for monitor the wal file flush performance.
+ * Each active wal file has a StreamSlowMonitor.
+ */
+@InterfaceAudience.Private
+public class StreamSlowMonitor implements ConfigurationObserver {
+ private static final Logger LOG =
LoggerFactory.getLogger(StreamSlowMonitor.class);
+
+ /**
+ * Configure for the min count for a datanode detected slow.
+ * If a datanode is detected slow times up to this count, then it will be
added to the exclude
+ * datanode cache by {@link
ExcludeDatanodeManager#tryAddExcludeDN(DatanodeInfo, String)}
+ * of this regionsever.
+ */
+ private static final String WAL_SLOW_DETECT_MIN_COUNT_KEY =
+"hbase.regionserver.async.wal.min.slow.detect.count";
+ private static final int DEFAULT_WAL_SLOW_DETECT_MIN_COUNT = 3;
+
+ /**
+ * Configure for the TTL of the data that a datanode detected slow.
+ */
+ private static final String WAL_SLOW_DETECT_DATA_TTL_KEY =
+"hbase.regionserver.async.wal.slow.detect.data.ttl.ms";
+ private static final long DEFAULT_WAL_SLOW_DETECT_DATA_TTL = 10 * 60 * 1000;
// 10min in ms
+
+ /**
+ * Configure for the slow packet process time, a duration from send to ACK.
+ * It is preferred that this value should not be less than 1s.
+ */
+ private static final String DATANODE_SLOW_PACKET_PROCESS_TIME_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.process.time.millis";
+ private static final long DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME = 6000;
//6s in ms
+
+ /**
+ * Configure for the packet flush speed.
+ * e.g. If the configured slow slow packet process time is larger than 1s,
then here 0.1kbs means
+ * 100B should be processed in less than 1s.
+ * If the configured slow slow packet process time is larger than 10s, then
here 0.1kbs means
+ * 1KB should be processed in less than 10s.
+ */
+ private static final String DATANODE_SLOW_PACKET_FLUSH_SPEED_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.flush.speed.kbs";
+ private static final double DEFAULT_DATANODE_SLOW_PACKET_FLUSH_SPEED = 0.1;
+
+ private final String name;
+ // this is a map of datanodeInfo->queued slow PacketAckData
+ private final LoadingCache>
datanodeSlowDataQueue;
+ private final ExcludeDatanodeManager excludeDatanodeManager;
+
+ private int minSlowDetectCount;
+ private long slowDataTtl;
+ private long slowPacketAckMs;
+ private double slowPacketAckSpeed;
+
+ public StreamSlowMonitor(Configuration conf, String name,
+ ExcludeDatanodeManager excludeDatanodeManager) {
+setConf(conf);
+this.name = name;
+this.excludeDatanodeManager = excludeDatanodeManager;
+this.datanodeSlowDataQueue = CacheBuilder.new
[GitHub] [hbase] sunhelly commented on a change in pull request #3800: HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL
sunhelly commented on a change in pull request #3800:
URL: https://github.com/apache/hbase/pull/3800#discussion_r774450839
##
File path:
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/monitor/StreamSlowMonitor.java
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.asyncfs.monitor;
+
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_EXCLUDE_DATANODE_TTL;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.DEFAULT_WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_EXCLUDE_DATANODE_TTL_KEY;
+import static
org.apache.hadoop.hbase.io.asyncfs.monitor.ExcludeDatanodeManager.WAL_MAX_EXCLUDE_SLOW_DATANODE_COUNT_KEY;
+
+import java.util.Deque;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.TimeUnit;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.conf.ConfigurationObserver;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheBuilder;
+import org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader;
+import org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache;
+
+/**
+ * Class for monitor the wal file flush performance.
+ * Each active wal file has a StreamSlowMonitor.
+ */
+@InterfaceAudience.Private
+public class StreamSlowMonitor implements ConfigurationObserver {
+ private static final Logger LOG =
LoggerFactory.getLogger(StreamSlowMonitor.class);
+
+ /**
+ * Configure for the min count for a datanode detected slow.
+ * If a datanode is detected slow times up to this count, then it will be
added to the exclude
+ * datanode cache by {@link
ExcludeDatanodeManager#tryAddExcludeDN(DatanodeInfo, String)}
+ * of this regionsever.
+ */
+ private static final String WAL_SLOW_DETECT_MIN_COUNT_KEY =
+"hbase.regionserver.async.wal.min.slow.detect.count";
+ private static final int DEFAULT_WAL_SLOW_DETECT_MIN_COUNT = 3;
+
+ /**
+ * Configure for the TTL of the data that a datanode detected slow.
+ */
+ private static final String WAL_SLOW_DETECT_DATA_TTL_KEY =
+"hbase.regionserver.async.wal.slow.detect.data.ttl.ms";
+ private static final long DEFAULT_WAL_SLOW_DETECT_DATA_TTL = 10 * 60 * 1000;
// 10min in ms
+
+ /**
+ * Configure for the slow packet process time, a duration from send to ACK.
+ * It is preferred that this value should not be less than 1s.
+ */
+ private static final String DATANODE_SLOW_PACKET_PROCESS_TIME_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.process.time.millis";
+ private static final long DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME = 6000;
//6s in ms
+
+ /**
+ * Configure for the packet flush speed.
+ * e.g. If the configured slow slow packet process time is larger than 1s,
then here 0.1kbs means
+ * 100B should be processed in less than 1s.
+ * If the configured slow slow packet process time is larger than 10s, then
here 0.1kbs means
+ * 1KB should be processed in less than 10s.
+ */
+ private static final String DATANODE_SLOW_PACKET_FLUSH_SPEED_KEY =
+"hbase.regionserver.async.wal.datanode.slow.packet.flush.speed.kbs";
+ private static final double DEFAULT_DATANODE_SLOW_PACKET_FLUSH_SPEED = 0.1;
+
+ private final String name;
+ // this is a map of datanodeInfo->queued slow PacketAckData
+ private final LoadingCache>
datanodeSlowDataQueue;
+ private final ExcludeDatanodeManager excludeDatanodeManager;
+
+ private int minSlowDetectCount;
+ private long slowDataTtl;
+ private long slowPacketAckMs;
+ private double slowPacketAckSpeed;
+
+ public StreamSlowMonitor(Configuration conf, String name,
+ ExcludeDatanodeManager excludeDatanodeManager) {
+setConf(conf);
+this.name = name;
+this.excludeDatanodeManager = excludeDatanodeManager;
+this.datanodeSlowDataQueue = CacheBuilder.ne
[GitHub] [hbase] Apache-HBase commented on pull request #3977: HBASE-26620 Add 3.0.0-alpha-2 to download page
Apache-HBase commented on pull request #3977: URL: https://github.com/apache/hbase/pull/3977#issuecomment-1000175853 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 3m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3977/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3977 | | Optional Tests | | | uname | Linux f14d883bdd25 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Max. process+thread count | 54 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3977/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3977: HBASE-26620 Add 3.0.0-alpha-2 to download page
Apache-HBase commented on pull request #3977: URL: https://github.com/apache/hbase/pull/3977#issuecomment-1000174584 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3977/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3977 | | Optional Tests | | | uname | Linux b325a494949f 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a67490abd3 | | Max. process+thread count | 46 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3977/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] mymeiyi opened a new pull request #3976: HBASE-26615 Snapshot referenced data files are deleted when delete a …
mymeiyi opened a new pull request #3976: URL: https://github.com/apache/hbase/pull/3976 …table with merge regions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26620) Add 3.0.0-alpha-2 to download page
[ https://issues.apache.org/jira/browse/HBASE-26620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26620: -- Fix Version/s: 3.0.0-alpha-3 > Add 3.0.0-alpha-2 to download page > -- > > Key: HBASE-26620 > URL: https://issues.apache.org/jira/browse/HBASE-26620 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26621) Set version as 3.0.0-alpha-3-SNAPSHOT in master
Duo Zhang created HBASE-26621: - Summary: Set version as 3.0.0-alpha-3-SNAPSHOT in master Key: HBASE-26621 URL: https://issues.apache.org/jira/browse/HBASE-26621 Project: HBase Issue Type: Sub-task Components: website Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 3.0.0-alpha-3 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26607) Put up 3.0.0-alpha-2RC2
[ https://issues.apache.org/jira/browse/HBASE-26607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26607. --- Assignee: Duo Zhang Resolution: Fixed 3.0.0-alpha-2RC2 is released as 3.0.0-alpha-2. > Put up 3.0.0-alpha-2RC2 > --- > > Key: HBASE-26607 > URL: https://issues.apache.org/jira/browse/HBASE-26607 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26607) Put up 3.0.0-alpha-2RC2
[ https://issues.apache.org/jira/browse/HBASE-26607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26607: -- Component/s: community > Put up 3.0.0-alpha-2RC2 > --- > > Key: HBASE-26607 > URL: https://issues.apache.org/jira/browse/HBASE-26607 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26620) Add 3.0.0-alpha-2 to download page
Duo Zhang created HBASE-26620: - Summary: Add 3.0.0-alpha-2 to download page Key: HBASE-26620 URL: https://issues.apache.org/jira/browse/HBASE-26620 Project: HBase Issue Type: Sub-task Components: website Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26615) Snapshot referenced data files are deleted when delete a table with merge regions
[ https://issues.apache.org/jira/browse/HBASE-26615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei reassigned HBASE-26615: -- Assignee: Yi Mei > Snapshot referenced data files are deleted when delete a table with merge > regions > - > > Key: HBASE-26615 > URL: https://issues.apache.org/jira/browse/HBASE-26615 > Project: HBase > Issue Type: Bug >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > > In our cluster, we have a feature: take a snapshot when delete a table. > But when we restore the snapshot, we found that some data files are deleted. > The problem is that, when delete a table with merge regions, HBase only > archive regions in meta, and merged parent regions are deleted in file system > which contain data files in snapshot . > -- This message was sent by Atlassian Jira (v8.20.1#820001)
