[GitHub] [hbase] Apache-HBase commented on pull request #3434: HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable
Apache-HBase commented on pull request #3434: URL: https://github.com/apache/hbase/pull/3434#issuecomment-870247452 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 52s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 28s | master passed | | +1 :green_heart: | compile | 1m 37s | master passed | | +1 :green_heart: | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 18s | the patch passed | | +1 :green_heart: | compile | 1m 38s | the patch passed | | +1 :green_heart: | javac | 1m 38s | the patch passed | | +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 2s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 128m 37s | hbase-server in the patch passed. | | | | 171m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3434 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7598c6f678d9 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/testReport/ | | Max. process+thread count | 3407 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3434: HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable
Apache-HBase commented on pull request #3434: URL: https://github.com/apache/hbase/pull/3434#issuecomment-870245104 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 55s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 27s | the patch passed | | +1 :green_heart: | shadedjars | 8m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 48s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 131m 11s | hbase-server in the patch passed. | | | | 165m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3434 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a08622e1855f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/testReport/ | | Max. process+thread count | 3640 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3434: HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable
Apache9 commented on a change in pull request #3434:
URL: https://github.com/apache/hbase/pull/3434#discussion_r660287517
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java
##
@@ -207,6 +234,17 @@ public static boolean readWithExtra(ByteBuff buf,
FSDataInputStream dis, int nec
*/
public static boolean preadWithExtra(ByteBuff buff, FSDataInputStream dis,
long position,
int necessaryLen, int extraLen) throws IOException {
+boolean preadbytebuffer = dis.hasCapability("in:preadbytebuffer");
Review comment:
We need to test it every time? Is it possible to test once when the
FSDataInputStream is created?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3430: HBASE-26029 It is not reliable to use nodeDeleted event to track regi…
Apache9 commented on a change in pull request #3430:
URL: https://github.com/apache/hbase/pull/3430#discussion_r660286508
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/replication/ClaimReplicationQueuesProcedure.java
##
@@ -0,0 +1,140 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.replication;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.ServerProcedureInterface;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
+import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.procedure2.ProcedureYieldException;
+import org.apache.hadoop.hbase.replication.ReplicationException;
+import org.apache.hadoop.hbase.replication.ReplicationQueueStorage;
+import org.apache.hadoop.hbase.util.RetryCounter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.ClaimReplicationQueuesStateData;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+/**
+ * Used to assign the replication queues of a dead server to other region
servers.
+ */
+@InterfaceAudience.Private
+public class ClaimReplicationQueuesProcedure extends
Procedure
+ implements ServerProcedureInterface {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(ClaimReplicationQueuesProcedure.class);
+
+ private ServerName crashedServer;
+
+ private RetryCounter retryCounter;
+
+ public ClaimReplicationQueuesProcedure() {
+ }
+
+ public ClaimReplicationQueuesProcedure(ServerName crashedServer) {
+this.crashedServer = crashedServer;
+ }
+
+ @Override
+ public ServerName getServerName() {
+return crashedServer;
+ }
+
+ @Override
+ public boolean hasMetaTableRegion() {
+return false;
+ }
+
+ @Override
+ public ServerOperationType getServerOperationType() {
+return ServerOperationType.CLAIM_REPLICATION_QUEUES;
+ }
+
+ @Override
+ protected Procedure[] execute(MasterProcedureEnv env)
+throws ProcedureYieldException, ProcedureSuspendedException,
InterruptedException {
+ReplicationQueueStorage storage =
env.getReplicationPeerManager().getQueueStorage();
+try {
+ List queues = storage.getAllQueues(crashedServer);
+ if (queues.isEmpty()) {
+LOG.debug("Finish claiming replication queues for {}", crashedServer);
+storage.removeReplicatorIfQueueIsEmpty(crashedServer);
+// we are done
+return null;
+ }
+ LOG.debug("There are {} replication queues need to be claimed for {}",
queues.size(),
+crashedServer);
+ List targetServers =
+env.getMasterServices().getServerManager().getOnlineServersList();
+ if (targetServers.isEmpty()) {
+throw new ReplicationException("no region server available");
+ }
+ Collections.shuffle(targetServers);
+ ClaimReplicationQueueRemoteProcedure[] procs =
+new ClaimReplicationQueueRemoteProcedure[Math.min(queues.size(),
targetServers.size())];
+ for (int i = 0; i < procs.length; i++) {
+procs[i] = new ClaimReplicationQueueRemoteProcedure(crashedServer,
queues.get(i),
Review comment:
It will be scheduled next time. The exit condition is there is no
replication queue for the given dead region server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26029) It is not reliable to use nodeDeleted event to track region server's death
[ https://issues.apache.org/jira/browse/HBASE-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371044#comment-17371044 ] Bharath Vissapragada commented on HBASE-26029: -- Hooking this up with SCP is neat, should clean up all the dead server management code from ReplicationSourceManager, +1. > It is not reliable to use nodeDeleted event to track region server's death > -- > > Key: HBASE-26029 > URL: https://issues.apache.org/jira/browse/HBASE-26029 > Project: HBase > Issue Type: Bug > Components: Replication, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > > When implementing HBASE-26011, [~sunxin] pointed out an interesting scenario, > where a region server up and down between two sync requests, then we can not > know the death of the region server. > https://github.com/apache/hbase/pull/3405#discussion_r656720923 > This is a valid point, and when thinking of a solution, I noticed that, the > current zk iplementation has the same problem. Notice that, a watcher on zk > can only be triggered once, so after zk triggers the watcher, and before you > set a new watcher, it is possible that a region server is up and down, and > you will miss the nodeDeleted event for this region server. > I think, the general approach here, which could works for both master based > and zk based replication tracker is that, we should not rely on the tracker > to tell you which region server is dead. Instead, we just provide the list of > live regionservers, and the upper layer should compare this list with the > expected list(for replication, the list should be gotten by listing > replicators), to detect the dead region servers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371034#comment-17371034
]
Bharath Vissapragada commented on HBASE-26021:
--
https://github.com/apache/hbase/pull/3435 I did a quick revert in the interest
of time ([~vjasani] hope its ok). It was a bit involved due to the other piled
up changes on top of it. I put it up for review to see how jenkins reacts to
it. I think it could use some additional test coverage for the new interface
methods.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R^Dmeta
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
>
[GitHub] [hbase] bharathv opened a new pull request #3435: HBASE-26021: Undo the incompatible serialization change in HBASE-7767
bharathv opened a new pull request #3435: URL: https://github.com/apache/hbase/pull/3435 Revert "HBASE-7767 Get rid of ZKTable, and table enable/disable state in ZK (Andrey Stepachev)" Fixes the ConnectionRegistry and dependencies due to the revert. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ddupg commented on a change in pull request #3430: HBASE-26029 It is not reliable to use nodeDeleted event to track regi…
ddupg commented on a change in pull request #3430:
URL: https://github.com/apache/hbase/pull/3430#discussion_r660247427
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
##
@@ -838,185 +795,116 @@ public void postLogRoll(Path newLog) throws IOException
{
}
}
- @Override
- public void regionServerRemoved(ServerName regionserver) {
-transferQueues(regionserver);
- }
-
- /**
- * Transfer all the queues of the specified to this region server. First it
tries to grab a lock
- * and if it works it will move the old queues and finally will delete the
old queues.
- *
- * It creates one old source for any type of source of the old rs.
- */
- private void transferQueues(ServerName deadRS) {
-if (server.getServerName().equals(deadRS)) {
- // it's just us, give up
+ void claimQueue(ServerName deadRS, String queue) {
+// Wait a bit before transferring the queues, we may be shutting down.
+// This sleep may not be enough in some cases.
+try {
+ Thread.sleep(sleepBeforeFailover +
+(long) (ThreadLocalRandom.current().nextFloat() *
sleepBeforeFailover));
+} catch (InterruptedException e) {
+ LOG.warn("Interrupted while waiting before transferring a queue.");
+ Thread.currentThread().interrupt();
+}
+// We try to lock that rs' queue directory
+if (server.isStopped()) {
+ LOG.info("Not transferring queue since we are shutting down");
+ return;
+}
+// After claim the queues from dead region server, wewill skip to start the
Review comment:
NIT: `wewill` miss a space.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/replication/ClaimReplicationQueuesProcedure.java
##
@@ -0,0 +1,140 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.replication;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.ServerProcedureInterface;
+import org.apache.hadoop.hbase.procedure2.Procedure;
+import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
+import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException;
+import org.apache.hadoop.hbase.procedure2.ProcedureUtil;
+import org.apache.hadoop.hbase.procedure2.ProcedureYieldException;
+import org.apache.hadoop.hbase.replication.ReplicationException;
+import org.apache.hadoop.hbase.replication.ReplicationQueueStorage;
+import org.apache.hadoop.hbase.util.RetryCounter;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.ClaimReplicationQueuesStateData;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ProcedureProtos;
+
+/**
+ * Used to assign the replication queues of a dead server to other region
servers.
+ */
+@InterfaceAudience.Private
+public class ClaimReplicationQueuesProcedure extends
Procedure
+ implements ServerProcedureInterface {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(ClaimReplicationQueuesProcedure.class);
+
+ private ServerName crashedServer;
+
+ private RetryCounter retryCounter;
+
+ public ClaimReplicationQueuesProcedure() {
+ }
+
+ public ClaimReplicationQueuesProcedure(ServerName crashedServer) {
+this.crashedServer = crashedServer;
+ }
+
+ @Override
+ public ServerName getServerName() {
+return crashedServer;
+ }
+
+ @Override
+ public boolean hasMetaTableRegion() {
+return false;
+ }
+
+ @Override
+ public ServerOperationType getServerOperationType() {
+return ServerOperationType.CLAIM_REPLICATION_QUEUES;
+ }
+
+ @Override
+ protected Procedure[] execute(MasterProcedureEnv env)
+throws ProcedureYieldException, ProcedureSuspendedException,
InterruptedException {
+ReplicationQueueStorage storage =
[GitHub] [hbase] Apache-HBase commented on pull request #3434: HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable
Apache-HBase commented on pull request #3434: URL: https://github.com/apache/hbase/pull/3434#issuecomment-870201695 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 3m 57s | master passed | | +1 :green_heart: | checkstyle | 1m 27s | master passed | | +1 :green_heart: | spotbugs | 2m 45s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | compile | 4m 3s | the patch passed | | +1 :green_heart: | javac | 4m 3s | the patch passed | | +1 :green_heart: | checkstyle | 1m 23s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 13s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 52m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3434 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 992c1cda610a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3434/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25130) [branch-1] Masters in-memory serverHoldings map is not cleared during hbck repair
[
https://issues.apache.org/jira/browse/HBASE-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25130:
--
Fix Version/s: 1.7.1
> [branch-1] Masters in-memory serverHoldings map is not cleared during hbck
> repair
> -
>
> Key: HBASE-25130
> URL: https://issues.apache.org/jira/browse/HBASE-25130
> Project: HBase
> Issue Type: Bug
>Reporter: Sandeep Guggilam
>Assignee: Victor Li
>Priority: Major
> Fix For: 1.7.1
>
>
> {color:#1d1c1d}Incase of repairing overlaps, hbck essentially calls the
> closeRegion RPC on RS followed by offline RPC on Master to offline all the
> overlap regions that would be merged into a new region. {color}
> {color:#1d1c1d}However the offline RPC doesn’t remove it from the
> serverHoldings map unless the new state is MERGED/SPLIT
> ([https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java#L719])
> b{color}{color:#1d1c1d}ut the new state in this case is OFFLINE. {color}
> {color:#1d1c1d}This is actually intended to match with the META entries and
> would be removed later when the region is online on a different server.
> However, in our case , the region would never be online on a new server,
> hence the region info is never cleared from the map that is used by balancer
> and SCP for incorrect reeassignment.{color}
> {color:#1d1c1d}We might need to tackle this by removing the entries from the
> map when hbck actually deletes{color}{color:#1d1c1d} the meta entries for
> this region which kind of matches the in-memory map’s expectation with the
> META state.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] jojochuang opened a new pull request #3434: HBASE-21946. Use hasCapabilities() to detect and switch pread API accordingly.
jojochuang opened a new pull request #3434: URL: https://github.com/apache/hbase/pull/3434 Use StreamCapabilities.hasCapabilities() to determine if the pread is supported by the input stream. Use reflection to invoke the bytebuffer pread method when it is available (because the ByteBufferPositionedReadable API is not available until Hadoop 3.3). This way, we can maintain backward compatibility for any Hadoop version above 2.9. Passed a few unit tests building with Hadoop 3.3.1 and 3.1.2. Open this PR to run through the whole precommit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-21946) Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-21946: Summary: Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable (was: Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared) > Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Wei-Chiu Chuang >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3433: HBASE-26035 Redundant null check in the compareTo function
Apache-HBase commented on pull request #3433: URL: https://github.com/apache/hbase/pull/3433#issuecomment-870124268 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 0m 32s | master passed | | +1 :green_heart: | checkstyle | 0m 15s | master passed | | +1 :green_heart: | spotbugs | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 13s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 37m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3433 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 70d9e0a59ae0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat U: hbase-hadoop-compat | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3433: HBASE-26035 Redundant null check in the compareTo function
Apache-HBase commented on pull request #3433: URL: https://github.com/apache/hbase/pull/3433#issuecomment-870122757 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | shadedjars | 9m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 50s | the patch passed | | +1 :green_heart: | compile | 0m 20s | the patch passed | | +1 :green_heart: | javac | 0m 20s | the patch passed | | +1 :green_heart: | shadedjars | 9m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. | | | | 32m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3433 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 90383c6e7170 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/testReport/ | | Max. process+thread count | 301 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat U: hbase-hadoop-compat | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3433: HBASE-26035 Redundant null check in the compareTo function
Apache-HBase commented on pull request #3433: URL: https://github.com/apache/hbase/pull/3433#issuecomment-870121257 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | shadedjars | 8m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 0m 20s | the patch passed | | +1 :green_heart: | javac | 0m 20s | the patch passed | | +1 :green_heart: | shadedjars | 8m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | hbase-hadoop-compat in the patch passed. | | | | 27m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3433 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a92058d144ae 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/testReport/ | | Max. process+thread count | 442 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat U: hbase-hadoop-compat | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3433/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26035) Redundant null check in the compareTo function
Almog Tavor created HBASE-26035:
---
Summary: Redundant null check in the compareTo function
Key: HBASE-26035
URL: https://issues.apache.org/jira/browse/HBASE-26035
Project: HBase
Issue Type: Bug
Reporter: Almog Tavor
Remove a redundant null check in the compareTo function
{code:java}
if (!(source instanceof MetricsRegionSourceImpl)) {
return -1;
}
MetricsRegionSourceImpl impl = (MetricsRegionSourceImpl) source;
if (impl == null) {
return -1;
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25396) Integration with direct insert into data/cf directory
[ https://issues.apache.org/jira/browse/HBASE-25396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-25396: -- Summary: Integration with direct insert into data/cf directory (was: Integtration with Direct insert into data/cf directory) > Integration with direct insert into data/cf directory > - > > Key: HBASE-25396 > URL: https://issues.apache.org/jira/browse/HBASE-25396 > Project: HBase > Issue Type: Sub-task >Reporter: Tak-Lon (Stephen) Wu >Priority: Major > > Utilizes direct insert compactor, file writer, committer and etc while > constructing the PersistedStoreEngine -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] almogtavor opened a new pull request #3433: Remove redundant comparison
almogtavor opened a new pull request #3433: URL: https://github.com/apache/hbase/pull/3433 Useless check, impl cannot be null (guarded by instanceof) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25392) Direct insert compacted HFiles into data directory.
[
https://issues.apache.org/jira/browse/HBASE-25392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-25392:
--
Description: {color:#00}when performing compaction, write directly to
the data directory {color} (was: {color:#00}when performing compaction,
write directory to the data directory {color})
> Direct insert compacted HFiles into data directory.
> ---
>
> Key: HBASE-25392
> URL: https://issues.apache.org/jira/browse/HBASE-25392
> Project: HBase
> Issue Type: Sub-task
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Wellington Chevreuil
>Priority: Major
>
> {color:#00}when performing compaction, write directly to the data
> directory {color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #3389: HBASE-25392 Direct insert compacted HFiles into data directory.
saintstack commented on a change in pull request #3389:
URL: https://github.com/apache/hbase/pull/3389#discussion_r660167381
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
##
@@ -291,12 +291,36 @@ public void setCacheDataOnWrite(boolean cacheDataOnWrite)
{
* cacheIndexesOnWrite
* cacheBloomsOnWrite
*/
- public void enableCacheOnWrite() {
+ public void enableCacheOnWriteForCompactions() {
this.cacheDataOnWrite = true;
this.cacheIndexesOnWrite = true;
this.cacheBloomsOnWrite = true;
}
+ /**
+ * If hbase.rs.cachecompactedblocksonwrite configuration is set to true and
+ * 'totalCompactedFilesSize' is lower than
'cacheCompactedDataOnWriteThreshold',
+ * enables cache on write for below properties:
+ * - cacheDataOnWrite
+ * - cacheIndexesOnWrite
+ * - cacheBloomsOnWrite
+ *
+ * Otherwise, sets 'cacheDataOnWrite' only to false.
+ *
+ * @param totalCompactedFilesSize the total size of compacted files.
+ * @return true if the checks mentioned above pass and the cache is enabled,
false otherwise.
+ */
+ public boolean enableCacheOnWriteForCompactions(long
totalCompactedFilesSize) {
Review comment:
Why the threshold check?
In description it talks of cacheCompactedDataOnWriteThreshold but the method
we call is getCacheCompactedBlocksOnWriteThreshold (and config is
cachecompactedblocksonwrite) and we check the return against
totalCompactedFilesSize. Its a little confusing on what is being compared here.
Any chance of some method renames or method name alignment w/ configs?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
##
@@ -533,4 +550,46 @@ protected InternalScanner createScanner(HStore store,
ScanInfo scanInfo,
return new StoreScanner(store, scanInfo, scanners, smallestReadPoint,
earliestPutTs,
dropDeletesFromRow, dropDeletesToRow);
}
+
+ /**
+ * Default implementation for committing store files created after a
compaction. Assumes new files
+ * had been created on a temp directory, so it renames those files into the
actual store dir,
+ * then create a reader and cache it into the store.
+ * @param cr the compaction request.
+ * @param newFiles the new files created by this compaction under a temp dir.
+ * @param user the running user.
+ * @return A list of the resulting store files already placed in the store
dir and loaded into the
+ * store cache.
+ * @throws IOException if the commit fails.
+ */
+ public List commitCompaction(CompactionRequestImpl cr,
List newFiles, User user)
+ throws IOException {
+List sfs = new ArrayList<>(newFiles.size());
+for (Path newFile : newFiles) {
+ assert newFile != null;
+ this.store.validateStoreFile(newFile);
+ // Move the file into the right spot
+ HStoreFile sf = createFileInStoreDir(newFile);
+ if (this.store.getCoprocessorHost() != null) {
+this.store.getCoprocessorHost().postCompact(this.store, sf,
cr.getTracker(), cr, user);
+ }
+ assert sf != null;
+ sfs.add(sf);
+}
+return sfs;
+ }
+
+ /**
+ * Assumes new file was created initially on a temp folder.
Review comment:
Don't we want the Compactor asking the StoreEngine to do the writing for
us? Rather than doing it here?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DirectStoreCompactor.java
##
@@ -0,0 +1,85 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.compactions;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.io.compress.Compression;
+import org.apache.hadoop.hbase.io.hfile.CacheConfig;
+import org.apache.hadoop.hbase.io.hfile.HFileContext;
+import org.apache.hadoop.hbase.regionserver.HStore;
+import org.apache.hadoop.hbase.regionserver.HStoreFile;
+import org.apache.hadoop.hbase.regionserver.StoreFileWriter;
+import org.apache.yetus.audience.InterfaceAudience;
+
[jira] [Commented] (HBASE-25902) 1.x to 2.3.x upgrade does not work; you must install an hbase2 that is earlier than hbase-2.3.0 first
[ https://issues.apache.org/jira/browse/HBASE-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370927#comment-17370927 ] Michael Stack commented on HBASE-25902: --- Thanks for working on this [~vjasani] I looked at the PR. Left a few comments. Only alternative I can think of is checking for the presence of the new CFs and if not present on HMaster startup, throw an exception which asks the operator to run a migration script to add the CFs. > 1.x to 2.3.x upgrade does not work; you must install an hbase2 that is > earlier than hbase-2.3.0 first > - > > Key: HBASE-25902 > URL: https://issues.apache.org/jira/browse/HBASE-25902 > Project: HBase > Issue Type: Bug > Components: meta, Operability >Affects Versions: 2.3.0, 2.4.0 >Reporter: Michael Stack >Assignee: Viraj Jasani >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.5 > > Attachments: NoSuchColumnFamilyException.png > > > Making note of this issue in case others run into it. At my place of employ, > we tried to upgrade a cluster that was an hbase-1.2.x version to an > hbase-2.3.5 but it failed because meta didn't have the 'table' column family. > Up to 2.3.0, hbase:meta was hardcoded. HBASE-12035 added the 'table' CF for > hbase-2.0.0. HBASE-23782 (2.3.0) undid hardcoding of the hbase:meta schema; > i.e. reading hbase:meta schema from the filesystem. The hbase:meta schema is > only created on initial install. If an upgrade over existing data, the > hbase-1 hbase:meta will not be suitable for hbase-2.3.x context as it will be > missing columnfamilies needed to run (HBASE-23055 made it so hbase:meta could > be altered (2.3.0) but probably of no use since Master won't come up). > It would be a nice-to-have if a user could go from hbase1 to hbase.2.3.0 w/o > having to first install an hbase2 that is earlier than 2.3.0 but needs to be > demand before we would work on it; meantime, install an intermediate hbase2 > version before going to hbase-2.3.0+ if coming from hbase-1.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #3417: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
saintstack commented on a change in pull request #3417:
URL: https://github.com/apache/hbase/pull/3417#discussion_r660162631
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1047,6 +1074,29 @@ private void
finishActiveMasterInitialization(MonitoredTask status)
// Set master as 'initialized'.
setInitialized(true);
+if (tableFamilyDesc == null && replBarrierFamilyDesc == null) {
+ // create missing CFs in meta table after master is set to 'initialized'.
+ createMissingCFsInMetaDuringUpgrade(metaDescriptor);
+
+ // Throwing this Exception to abort active master is painful but this
+ // seems the only way to add missing CFs in meta while upgrading from
+ // HBase 1 to 2 (where HBase 2 has HBASE-23055 & HBASE-23782 checked-in).
+ // So, why do we abort active master after adding missing CFs in meta?
+ // When we reach here, we would have already bypassed
NoSuchColumnFamilyException
+ // in initClusterSchemaService(), meaning ClusterSchemaService is not
+ // correctly initialized but we bypassed it. Similarly, we bypassed
+ // tableStateManager.start() as well. Hence, we should better abort
+ // current active master because our main task - adding missing CFs
+ // in meta table is done (possible only after master state is set as
+ // initialized) at the expense of bypassing few important tasks as part
+ // of active master init routine. So now we abort active master so that
+ // next active master init will not face any issues and all mandatory
+ // services will be started during master init phase.
Review comment:
Nice note.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1047,6 +1074,29 @@ private void
finishActiveMasterInitialization(MonitoredTask status)
// Set master as 'initialized'.
setInitialized(true);
+if (tableFamilyDesc == null && replBarrierFamilyDesc == null) {
+ // create missing CFs in meta table after master is set to 'initialized'.
+ createMissingCFsInMetaDuringUpgrade(metaDescriptor);
+
+ // Throwing this Exception to abort active master is painful but this
+ // seems the only way to add missing CFs in meta while upgrading from
+ // HBase 1 to 2 (where HBase 2 has HBASE-23055 & HBASE-23782 checked-in).
+ // So, why do we abort active master after adding missing CFs in meta?
+ // When we reach here, we would have already bypassed
NoSuchColumnFamilyException
+ // in initClusterSchemaService(), meaning ClusterSchemaService is not
+ // correctly initialized but we bypassed it. Similarly, we bypassed
+ // tableStateManager.start() as well. Hence, we should better abort
+ // current active master because our main task - adding missing CFs
+ // in meta table is done (possible only after master state is set as
+ // initialized) at the expense of bypassing few important tasks as part
+ // of active master init routine. So now we abort active master so that
+ // next active master init will not face any issues and all mandatory
+ // services will be started during master init phase.
+ throw new IOException("Stopping active master after missing CFs are "
Review comment:
We are not stopping, we are aborting?
Would it be better to throw a PleaseRestartMeException here? (Would have to
create it...).
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -953,9 +955,25 @@ private void
finishActiveMasterInitialization(MonitoredTask status)
if (!waitForMetaOnline()) {
return;
}
+TableDescriptor metaDescriptor = tableDescriptors.get(
+TableName.META_TABLE_NAME);
+final ColumnFamilyDescriptor tableFamilyDesc = metaDescriptor
+.getColumnFamily(HConstants.TABLE_FAMILY);
+final ColumnFamilyDescriptor replBarrierFamilyDesc =
+metaDescriptor.getColumnFamily(HConstants.REPLICATION_BARRIER_FAMILY);
+
this.assignmentManager.joinCluster();
// The below depends on hbase:meta being online.
-this.tableStateManager.start();
+try {
+ this.tableStateManager.start();
+} catch (NoSuchColumnFamilyException e) {
+ if (tableFamilyDesc == null && replBarrierFamilyDesc == null) {
+LOG.info("For missing CFs in meta, this Exception is expected", e);
Review comment:
You think operator will know what 'missing CFs in meta' is about? Would
it be better to talk about migration from hbase1 to hbase2?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1106,6 +1156,38 @@ private void
finishActiveMasterInitialization(MonitoredTask status)
}
}
+ private void createMissingCFsInMetaDuringUpgrade(
+ TableDescriptor metaDescriptor) throws IOException {
+TableDescriptor
[jira] [Resolved] (HBASE-25130) [branch-1] Masters in-memory serverHoldings map is not cleared during hbck repair
[
https://issues.apache.org/jira/browse/HBASE-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victor Li resolved HBASE-25130.
---
Release Note: Fixed on branch-1.
Resolution: Fixed
https://github.com/apache/hbase/pull/3402
> [branch-1] Masters in-memory serverHoldings map is not cleared during hbck
> repair
> -
>
> Key: HBASE-25130
> URL: https://issues.apache.org/jira/browse/HBASE-25130
> Project: HBase
> Issue Type: Bug
>Reporter: Sandeep Guggilam
>Assignee: Victor Li
>Priority: Major
>
> {color:#1d1c1d}Incase of repairing overlaps, hbck essentially calls the
> closeRegion RPC on RS followed by offline RPC on Master to offline all the
> overlap regions that would be merged into a new region. {color}
> {color:#1d1c1d}However the offline RPC doesn’t remove it from the
> serverHoldings map unless the new state is MERGED/SPLIT
> ([https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java#L719])
> b{color}{color:#1d1c1d}ut the new state in this case is OFFLINE. {color}
> {color:#1d1c1d}This is actually intended to match with the META entries and
> would be removed later when the region is online on a different server.
> However, in our case , the region would never be online on a new server,
> hence the region info is never cleared from the map that is used by balancer
> and SCP for incorrect reeassignment.{color}
> {color:#1d1c1d}We might need to tackle this by removing the entries from the
> map when hbck actually deletes{color}{color:#1d1c1d} the meta entries for
> this region which kind of matches the in-memory map’s expectation with the
> META state.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22923) hbase:meta is assigned to localhost when we downgrade the hbase version
[
https://issues.apache.org/jira/browse/HBASE-22923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370843#comment-17370843
]
Andrew Kyle Purtell commented on HBASE-22923:
-
[~vjasani] Way back when RSGroups was first proposed, there were concerns about
it, and to resolve those concerns the community imposed this requirement: _All
RSgroups changes must be optional, and confined to the hbase-rsgroups module_,
with only case by case exceptions for enabling plug points in hbase-server and
other core modules. This requirement demanded the admin APIs for rsgroups
should also be in the hbase-rsgroups module, and this is the reason for
RSGroupInfoManager.
So that's the historical reason for the separation, and I assume for
compatibility guideline reasons hbase-rsgroups was only merged into the core
modules in master branch for HBase 3.
Anyway, in your scenario, this seems like the key problem:
{quote}Bring up new RegionServer on higher version. [...] One dedicated thread
in AM will try to assign meta (and other system tables) to newly brought RS
with higher version but will fail to bring it online because newly brought up
RS is not under jurisdiction of system RSGroup.
{quote}
This behavior is introduced specifically for rolling upgrade, because META
might need to be migrated, and earlier thinking was a regionserver could do it,
but latest thinking is it is better for the master to migrate META and other
tables, thus changing how we do rolling upgrades, thus changing the
requirements for this behavior. At the very least we can make it optional.
> hbase:meta is assigned to localhost when we downgrade the hbase version
> ---
>
> Key: HBASE-22923
> URL: https://issues.apache.org/jira/browse/HBASE-22923
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.8
>Reporter: wenbang
>Priority: Major
>
> When we downgrade the hbase version（rsgroup enable）, we found that the
> hbase:meta table could not be assigned.
> {code:java}
> master.AssignmentManager: Failed assignment of hbase:meta,,1.1588230740 to
> localhost,1,1, trying to assign elsewhere instead; try=1 of 10
> java.io.IOException: Call to localhost/127.0.0.1:1 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: localhost/127.0.0.1:1
> {code}
> hbase group list:
> HBASE_META group（hbase:meta and other system tables）
> default group
> 1.Down grade all servers in HBASE_META first
> 2.higher version servers is in default
> 3.hbase:meta assigned to localhost
> For system table, we assign them to a server with highest version.
> AssignmentManager#getExcludedServersForSystemTable
> But did not consider the rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15
[ https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370838#comment-17370838 ] Michael Stack commented on HBASE-26031: --- Thanks [~busbey] > Validate nightly builds run on new ci workers hbase11-hbase15 > - > > Key: HBASE-26031 > URL: https://issues.apache.org/jira/browse/HBASE-26031 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: image-2021-06-24-16-14-03-721.png > > > Per slack, asf infra has finished adding in nodes hbase10-hbase15 to > ci-hadoop. > make sure they can run nightly. > # Set labels for all these node to "hbase-staging" > # Push a feature branch off of current HEAD that updates the agent labels to > use "hbase-staging" > # trigger a bunch of runs. make sure *something* runs on each of the nodes > # Set labels for the nodes to "hbase" > # delete feature branch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25761) POC: hbase:meta,,1 as ROOT
[ https://issues.apache.org/jira/browse/HBASE-25761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370837#comment-17370837 ] Michael Stack commented on HBASE-25761: --- I took a look at the one-pager [~toffer] . Suggest you integrate it into the [https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit#heading=h.hdf0rnyevxz2] design doc replacing the content of the current 4.1.1. section which is just an outline of what your one-pager fills out (You can have your last paragraph as 'Downside'). It is appealing that old clients will just continue to work if no split (Requirement 2.0.4 and 2.0.5 rolling upgrade should be easier too). The 'first meta region' conditional clauses are a little awkward but seems easier than back-filling ROOT (again). The PoC seems fine... a bunch of if/thens w/ the ugly comparator story but I think this not a particular of this approach – any impl will have some version of this (I think). [~zhangduo] Do you want to write up your re-intro of root table showing how it would remain compatible – '...just need to copy location to zookeeper...'. How would the old clients work and would it violate requirement 2.0.3? Agree that if split meta is not on by default, it means it will not get exercised and there is the danger that it could then stall as a feature and become dead code but the path outlined above where it is off until the cluster is migrated to splittable-meta and then later, splitting is enabled as default makes sense to me. Shall we do an online meeting to discuss? This week? Usual time? Can talk about this approach and others to consider. Would be good to call an approach so could get basics into 3.0.0. > POC: hbase:meta,,1 as ROOT > -- > > Key: HBASE-25761 > URL: https://issues.apache.org/jira/browse/HBASE-25761 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Francis Christopher Liu >Priority: Major > > One of the proposals up in the split-meta design doc suggests a > sleight-of-hand where the current hard-coded hbase:meta,,1 Region is > leveraged to serve as first Region of a split hbase:meta but also does > double-duty as 'ROOT'. This suggestion was put aside as a complicating > recursion in chat but then Francis noticed on a re-read of the BigTable > paper, that this is how they describe they do 'ROOT': "The root tablet is > just the first tablet in the METADATA table, but is treated specially -- it > is never split..." > This issue is for playing around with this notion to see what the problems > are so can do a better description of this approach here, in the design: > https://docs.google.com/document/d/11ChsSb2LGrSzrSJz8pDCAw5IewmaMV0ZDN1LrMkAj4s/edit?ts=606c120f#heading=h.ikbhxlcthjle -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15
[ https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370830#comment-17370830 ] Sean Busbey edited comment on HBASE-26031 at 6/28/21, 8:20 PM: --- looking through the logs on the builds that ran on hbase11-hbase15 I don't see any red flags that indicate a problem with host configuration. There are some environment looking issues that are probably due to them being new and will probably be transient, e.g. maven central not liking how fast we are pulling all of the dependencies down across executors on a single host. was (Author: busbey): looking through the logs on the builds that ran on hbase11-hbase15 I don't see any red flags that indicate a problem with host configuration. There are some environment looking issues that are probably due to them being new and will probably be transient, e.g. maven central not linking how fast we are pulling all of the dependencies down across executors on a single host. > Validate nightly builds run on new ci workers hbase11-hbase15 > - > > Key: HBASE-26031 > URL: https://issues.apache.org/jira/browse/HBASE-26031 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: image-2021-06-24-16-14-03-721.png > > > Per slack, asf infra has finished adding in nodes hbase10-hbase15 to > ci-hadoop. > make sure they can run nightly. > # Set labels for all these node to "hbase-staging" > # Push a feature branch off of current HEAD that updates the agent labels to > use "hbase-staging" > # trigger a bunch of runs. make sure *something* runs on each of the nodes > # Set labels for the nodes to "hbase" > # delete feature branch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15
[ https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370834#comment-17370834 ] Sean Busbey commented on HBASE-26031: - I just moved hbase11 into the normal pool. presuming nothing untoward shows up from it over the rest of today then tomorrow I'll move hbase12 and hbase13. Then the last two to follow on Wednesday. > Validate nightly builds run on new ci workers hbase11-hbase15 > - > > Key: HBASE-26031 > URL: https://issues.apache.org/jira/browse/HBASE-26031 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: image-2021-06-24-16-14-03-721.png > > > Per slack, asf infra has finished adding in nodes hbase10-hbase15 to > ci-hadoop. > make sure they can run nightly. > # Set labels for all these node to "hbase-staging" > # Push a feature branch off of current HEAD that updates the agent labels to > use "hbase-staging" > # trigger a bunch of runs. make sure *something* runs on each of the nodes > # Set labels for the nodes to "hbase" > # delete feature branch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15
[ https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370832#comment-17370832 ] Sean Busbey commented on HBASE-26031: - if no-one objects I'm going to phase these into the normal pool while continuing to run branch builds on the hosts that haven't moved over yet. I figure that will help with keeping the cache of dependency artifacts warm to lesson the chance they have an issue when doing normal builds. > Validate nightly builds run on new ci workers hbase11-hbase15 > - > > Key: HBASE-26031 > URL: https://issues.apache.org/jira/browse/HBASE-26031 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: image-2021-06-24-16-14-03-721.png > > > Per slack, asf infra has finished adding in nodes hbase10-hbase15 to > ci-hadoop. > make sure they can run nightly. > # Set labels for all these node to "hbase-staging" > # Push a feature branch off of current HEAD that updates the agent labels to > use "hbase-staging" > # trigger a bunch of runs. make sure *something* runs on each of the nodes > # Set labels for the nodes to "hbase" > # delete feature branch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15
[ https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370830#comment-17370830 ] Sean Busbey commented on HBASE-26031: - looking through the logs on the builds that ran on hbase11-hbase15 I don't see any red flags that indicate a problem with host configuration. There are some environment looking issues that are probably due to them being new and will probably be transient, e.g. maven central not linking how fast we are pulling all of the dependencies down across executors on a single host. > Validate nightly builds run on new ci workers hbase11-hbase15 > - > > Key: HBASE-26031 > URL: https://issues.apache.org/jira/browse/HBASE-26031 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: image-2021-06-24-16-14-03-721.png > > > Per slack, asf infra has finished adding in nodes hbase10-hbase15 to > ci-hadoop. > make sure they can run nightly. > # Set labels for all these node to "hbase-staging" > # Push a feature branch off of current HEAD that updates the agent labels to > use "hbase-staging" > # trigger a bunch of runs. make sure *something* runs on each of the nodes > # Set labels for the nodes to "hbase" > # delete feature branch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15
[ https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370824#comment-17370824 ] Sean Busbey commented on HBASE-26031: - I put hbase10 back into the usual rotation over the weekend once I saw these notes, fyi. > Validate nightly builds run on new ci workers hbase11-hbase15 > - > > Key: HBASE-26031 > URL: https://issues.apache.org/jira/browse/HBASE-26031 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: image-2021-06-24-16-14-03-721.png > > > Per slack, asf infra has finished adding in nodes hbase10-hbase15 to > ci-hadoop. > make sure they can run nightly. > # Set labels for all these node to "hbase-staging" > # Push a feature branch off of current HEAD that updates the agent labels to > use "hbase-staging" > # trigger a bunch of runs. make sure *something* runs on each of the nodes > # Set labels for the nodes to "hbase" > # delete feature branch -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22923) hbase:meta is assigned to localhost when we downgrade the hbase version
[
https://issues.apache.org/jira/browse/HBASE-22923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370820#comment-17370820
]
Viraj Jasani commented on HBASE-22923:
--
Here is what can reproduce this easily (and it can happen unknowingly in
production):
# Create 'system' RSGroup. Move all system tables to this RSGroup.
# Now we have system tables in system RSGroup and all other tables in default
RSGroup.
# Bring up new RegionServer on higher version. Since it's IP address is not
yet known to master, it will be added to 'default' RSGroup by default (or let's
say unknowingly one of default RSGroup's RegionServer is restarted and brought
to higher version during rolling upgrade).
# One dedicated thread in AM will try to assign meta (and other system tables)
to newly brought RS with higher version but will fail to bring it online
because newly brought up RS is not under jurisdiction of system RSGroup. And we
will get the same error as reported by [~wenbang] as per Jira description.
Just had a quick glance, and it seems that we have purposefully kept
RSGroupInfoManager (together with it's coproc endpoint) away from hbase-server
(hbase-rsgroup not reachable) in branch-1 and branch-2, and it is again moved
back to hbase-server module in trunk.
[~zhangduo] [~zghao] does this mean branch-1 and 2 have no way for AM to
interact with RSGroup APIs directly? If we have a way, it would be better for
AM to identify that any RS brought up with higher version than rest of RS
should belong to 'system' RSGroup (for meta to move) and if it doesn't belong
to system RSGroup, then do not move meta region to it regardless of the version
difference.
[~anoop.hbase]
> hbase:meta is assigned to localhost when we downgrade the hbase version
> ---
>
> Key: HBASE-22923
> URL: https://issues.apache.org/jira/browse/HBASE-22923
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.8
>Reporter: wenbang
>Priority: Major
>
> When we downgrade the hbase version（rsgroup enable）, we found that the
> hbase:meta table could not be assigned.
> {code:java}
> master.AssignmentManager: Failed assignment of hbase:meta,,1.1588230740 to
> localhost,1,1, trying to assign elsewhere instead; try=1 of 10
> java.io.IOException: Call to localhost/127.0.0.1:1 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: localhost/127.0.0.1:1
> {code}
> hbase group list:
> HBASE_META group（hbase:meta and other system tables）
> default group
> 1.Down grade all servers in HBASE_META first
> 2.higher version servers is in default
> 3.hbase:meta assigned to localhost
> For system table, we assign them to a server with highest version.
> AssignmentManager#getExcludedServersForSystemTable
> But did not consider the rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370818#comment-17370818
]
Hudson commented on HBASE-25980:
Results for branch branch-2
[build #288 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370817#comment-17370817
]
Hudson commented on HBASE-25914:
Results for branch branch-2
[build #288 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: callDetails.png, largeLog.png, opeartionDetails1.png,
> operationDetails2.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26015) Should implement getRegionServers(boolean) method in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370816#comment-17370816
]
Hudson commented on HBASE-26015:
Results for branch branch-2
[build #288 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/288/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should implement getRegionServers(boolean) method in AsyncAdmin
> ---
>
> Key: HBASE-26015
> URL: https://issues.apache.org/jira/browse/HBASE-26015
> Project: HBase
> Issue Type: Task
> Components: Admin, Client
>Reporter: Duo Zhang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> We have this method in Admin but not in AsyncAdmin, we should align these two
> interfaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370785#comment-17370785
]
Andrew Kyle Purtell commented on HBASE-26021:
-
The revert is not difficult to identify. There is exactly one backport commit.
Revert it. Then fix master registry, where it depended on this. Commit the
revert of the back port and the fix for master registry in one PR. Then release
1.7.0.1.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R^Dmeta
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
>
[GitHub] [hbase] Apache-HBase commented on pull request #3425: HBASE-25991 Do compaction on compaction server
Apache-HBase commented on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869918972 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-25714 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 43s | HBASE-25714 passed | | +1 :green_heart: | compile | 1m 27s | HBASE-25714 passed | | +1 :green_heart: | shadedjars | 9m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 26s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 8m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 209m 1s | hbase-server in the patch passed. | | | | 245m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3425 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fb9eb22414d5 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / da0fa3000e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/testReport/ | | Max. process+thread count | 3642 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3425: HBASE-25991 Do compaction on compaction server
Apache-HBase commented on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869891540 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 14m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-25714 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 26s | HBASE-25714 passed | | +1 :green_heart: | compile | 1m 5s | HBASE-25714 passed | | +1 :green_heart: | shadedjars | 8m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 57s | the patch passed | | +1 :green_heart: | compile | 1m 7s | the patch passed | | +1 :green_heart: | javac | 1m 7s | the patch passed | | +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 161m 44s | hbase-server in the patch passed. | | | | 206m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3425 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 51f31beef188 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / da0fa3000e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/testReport/ | | Max. process+thread count | 3940 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370757#comment-17370757
]
Hudson commented on HBASE-25980:
Results for branch branch-2.3
[build #246 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26015) Should implement getRegionServers(boolean) method in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370756#comment-17370756
]
Hudson commented on HBASE-26015:
Results for branch branch-2.3
[build #246 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/246/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should implement getRegionServers(boolean) method in AsyncAdmin
> ---
>
> Key: HBASE-26015
> URL: https://issues.apache.org/jira/browse/HBASE-26015
> Project: HBase
> Issue Type: Task
> Components: Admin, Client
>Reporter: Duo Zhang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> We have this method in Admin but not in AsyncAdmin, we should align these two
> interfaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #3425: HBASE-25991 Do compaction on compaction server
saintstack commented on a change in pull request #3425:
URL: https://github.com/apache/hbase/pull/3425#discussion_r659933917
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/compactionserver/CSRpcServices.java
##
@@ -88,8 +93,18 @@ void start() {
public CompactResponse requestCompaction(RpcController controller,
CompactionProtos.CompactRequest request) {
requestCount.increment();
+ServerName rsServerName = ProtobufUtil.toServerName(request.getServer());
+RegionInfo regionInfo = ProtobufUtil.toRegionInfo(request.getRegionInfo());
+ColumnFamilyDescriptor cfd =
ProtobufUtil.toColumnFamilyDescriptor(request.getFamily());
+boolean major = request.getMajor();
+int priority = request.getPriority();
+List favoredNodes =
Collections.singletonList(request.getServer());
LOG.info("Receive compaction request from {}",
ProtobufUtil.toString(request));
-compactionServer.compactionThreadManager.requestCompaction();
+CompactionTask compactionTask =
+
CompactionTask.newBuilder().setRsServerName(rsServerName).setRegionInfo(regionInfo)
+
.setColumnFamilyDescriptor(cfd).setRequestMajor(major).setPriority(priority)
+
.setFavoredNodes(favoredNodes).setSubmitTime(System.currentTimeMillis()).build();
+compactionServer.compactionThreadManager.requestCompaction(compactionTask);
Review comment:
So 'throttling' is done via the current compaction throttling mechanism
it seems. Good. Maybe later Master would have an overall view on compactions
and do its own macro-level throttling...
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactThreadControl.java
##
@@ -0,0 +1,151 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import
org.apache.hadoop.hbase.regionserver.throttle.CompactionThroughputControllerFactory;
+import org.apache.hadoop.hbase.regionserver.throttle.ThroughputController;
+import
org.apache.hadoop.hbase.regionserver.throttle.ThroughputControllerService;
+import org.apache.hadoop.hbase.util.StealJobQueue;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import
org.apache.hbase.thirdparty.com.google.common.util.concurrent.ThreadFactoryBuilder;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Comparator;
+import java.util.Iterator;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.RejectedExecutionHandler;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import java.util.function.BiConsumer;
+
+@InterfaceAudience.Private
+public class CompactThreadControl {
Review comment:
What does a CompactThreadControl do?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/compactionserver/CompactionServerStorage.java
##
@@ -0,0 +1,139 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.compactionserver;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import
[GitHub] [hbase] apurtell merged pull request #3402: HBASE-25130 - Fix master in-memory server holding map after:
apurtell merged pull request #3402: URL: https://github.com/apache/hbase/pull/3402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25714) Offload the compaction job to independent Compaction Server
[ https://issues.apache.org/jira/browse/HBASE-25714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370688#comment-17370688 ] Michael Stack commented on HBASE-25714: --- Looks great. Left some comments on the design doc (sorry for being late). Linked an old issue that this would seem to supercede > Offload the compaction job to independent Compaction Server > --- > > Key: HBASE-25714 > URL: https://issues.apache.org/jira/browse/HBASE-25714 > Project: HBase > Issue Type: Umbrella >Reporter: Yulin Niu >Assignee: Yulin Niu >Priority: Major > > The basic idea is add a role "CompactionServer" to take the Compaction job. > HMaster is responsible for scheduling the compaction job to different > CompactionServer. > [design > doc|https://docs.google.com/document/d/1exmhQpQArAgnryLaV78K3260rKm64BHBNzZE4VdTz0c/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26025) Add a flag to mark if the IOError can be solved by retry in thrift IOError
[
https://issues.apache.org/jira/browse/HBASE-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370664#comment-17370664
]
Hudson commented on HBASE-26025:
Results for branch branch-1
[build #139 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/139/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/139//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/139//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/139//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Add a flag to mark if the IOError can be solved by retry in thrift IOError
> --
>
> Key: HBASE-26025
> URL: https://issues.apache.org/jira/browse/HBASE-26025
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 1.7.1, 2.4.5
>
>
> Currently, if an HBaseIOException occurs, the thrift client can only get the
> error message. This is inconvenient for the client constructing a retry
> mechanism to handle the exception. So I added a canRetry mark in IOError to
> make the client side exception handling smarter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
Apache-HBase commented on pull request #3372: URL: https://github.com/apache/hbase/pull/3372#issuecomment-869785498 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 43s | master passed | | +1 :green_heart: | compile | 2m 13s | master passed | | +1 :green_heart: | shadedjars | 9m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 29s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 48s | the patch passed | | +1 :green_heart: | compile | 2m 13s | the patch passed | | +1 :green_heart: | javac | 2m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 13s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 39s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 183m 54s | hbase-server in the patch passed. | | | | 226m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3372 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6545d7a31364 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/testReport/ | | Max. process+thread count | 2772 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3425: HBASE-25991 Do compaction on compaction server
Apache-HBase commented on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869768347 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-25714 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | HBASE-25714 passed | | +1 :green_heart: | compile | 3m 12s | HBASE-25714 passed | | +1 :green_heart: | checkstyle | 1m 6s | HBASE-25714 passed | | +1 :green_heart: | spotbugs | 2m 5s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 3m 11s | the patch passed | | +1 :green_heart: | javac | 3m 11s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 6 new + 92 unchanged - 9 fixed = 98 total (was 101) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 53s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | -1 :x: | spotbugs | 2m 19s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 47m 50s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Write to static field org.apache.hadoop.hbase.regionserver.CompactThreadControl.rejection from instance method new org.apache.hadoop.hbase.regionserver.CompactThreadControl(ThroughputControllerService, int, int, Comparator, BiConsumer) At CompactThreadControl.java:from instance method new org.apache.hadoop.hbase.regionserver.CompactThreadControl(ThroughputControllerService, int, int, Comparator, BiConsumer) At CompactThreadControl.java:[line 55] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3425 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2d6734e067f3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / da0fa3000e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html | | Max. process+thread count | 95 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
Apache-HBase commented on pull request #3372: URL: https://github.com/apache/hbase/pull/3372#issuecomment-869737381 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 36s | master passed | | +1 :green_heart: | compile | 1m 54s | master passed | | +1 :green_heart: | shadedjars | 8m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 22s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 41s | the patch passed | | +1 :green_heart: | compile | 1m 49s | the patch passed | | +1 :green_heart: | javac | 1m 49s | the patch passed | | +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 46s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 17s | hbase-client in the patch passed. | | -1 :x: | unit | 129m 52s | hbase-server in the patch failed. | | | | 166m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3372 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 318ab4c2091d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/testReport/ | | Max. process+thread count | 4113 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26015) Should implement getRegionServers(boolean) method in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370621#comment-17370621
]
Hudson commented on HBASE-26015:
Results for branch branch-2.4
[build #153 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should implement getRegionServers(boolean) method in AsyncAdmin
> ---
>
> Key: HBASE-26015
> URL: https://issues.apache.org/jira/browse/HBASE-26015
> Project: HBase
> Issue Type: Task
> Components: Admin, Client
>Reporter: Duo Zhang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> We have this method in Admin but not in AsyncAdmin, we should align these two
> interfaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370622#comment-17370622
]
Hudson commented on HBASE-25980:
Results for branch branch-2.4
[build #153 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/153/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] catalin-luca commented on pull request #3369: [BACKPORT] HBASE-25859 Reference class incorrectly parses the protobuf magic marker
catalin-luca commented on pull request #3369: URL: https://github.com/apache/hbase/pull/3369#issuecomment-869724807 @busbey Any idea how I can re-trigger the build? Current failure is un-related to the changes in the patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370601#comment-17370601
]
Hudson commented on HBASE-25914:
Results for branch master
[build #334 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: callDetails.png, largeLog.png, opeartionDetails1.png,
> operationDetails2.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26015) Should implement getRegionServers(boolean) method in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370600#comment-17370600
]
Hudson commented on HBASE-26015:
Results for branch master
[build #334 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Should implement getRegionServers(boolean) method in AsyncAdmin
> ---
>
> Key: HBASE-26015
> URL: https://issues.apache.org/jira/browse/HBASE-26015
> Project: HBase
> Issue Type: Task
> Components: Admin, Client
>Reporter: Duo Zhang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> We have this method in Admin but not in AsyncAdmin, we should align these two
> interfaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370602#comment-17370602
]
Hudson commented on HBASE-25980:
Results for branch master
[build #334 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/334/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] nyl3532016 edited a comment on pull request #3425: HBASE-25991 Do compaction on compaction server
nyl3532016 edited a comment on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869608162 > Please add more javadoc for CompactionServer related class on how to share the same logic with HRegionServer? I do not think we want to duplicate the code twice... `CompactionThreadManager` reuse `HStore.selectCompaction`,`HStore.throttleCompaction`,`CompactionContext.compact` of `HRegionserver`, which are core logic of compaction. Yes,we have a few duplicate code, just like compaction thread pool and CompactionRunner. I will try to eliminate thread pool duplicate code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3426: HBASE-26032 Make HRegion.getStores() an O(1) operation
Apache-HBase commented on pull request #3426: URL: https://github.com/apache/hbase/pull/3426#issuecomment-869678634 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 21s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 9m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 9m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 137m 42s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 59s | hbase-examples in the patch passed. | | | | 176m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3426 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b65a89dcbea7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/testReport/ | | Max. process+thread count | 3339 (vs. ulimit of 3) | | modules | C: hbase-server hbase-examples U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3426: HBASE-26032 Make HRegion.getStores() an O(1) operation
Apache-HBase commented on pull request #3426: URL: https://github.com/apache/hbase/pull/3426#issuecomment-869664047 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 7s | master passed | | +1 :green_heart: | compile | 1m 38s | master passed | | +1 :green_heart: | shadedjars | 8m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 16s | the patch passed | | +1 :green_heart: | compile | 1m 43s | the patch passed | | +1 :green_heart: | javac | 1m 43s | the patch passed | | +1 :green_heart: | shadedjars | 8m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 122m 48s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 45s | hbase-examples in the patch passed. | | | | 158m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3426 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 80c06df244ed 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/testReport/ | | Max. process+thread count | 3444 (vs. ulimit of 3) | | modules | C: hbase-server hbase-examples U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
Apache-HBase commented on pull request #3372: URL: https://github.com/apache/hbase/pull/3372#issuecomment-869659741 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | compile | 5m 4s | master passed | | +1 :green_heart: | checkstyle | 2m 2s | master passed | | +0 :ok: | refguide | 3m 42s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 3m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 58s | the patch passed | | +1 :green_heart: | compile | 5m 1s | the patch passed | | +1 :green_heart: | javac | 5m 1s | the patch passed | | +1 :green_heart: | checkstyle | 0m 24s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 28s | hbase-client: The patch generated 0 new + 29 unchanged - 1 fixed = 29 total (was 30) | | +1 :green_heart: | checkstyle | 1m 8s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 3m 45s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 20m 3s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 69m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3372 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide xml | | uname | Linux 510c5b923e14 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/artifact/yetus-general-check/output/branch-site/book.html | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu reassigned HBASE-21946: Assignee: Wei-Chiu Chuang (was: Zheng Hu) > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Wei-Chiu Chuang >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370564#comment-17370564 ] Zheng Hu commented on HBASE-21946: -- [~weichiu] Please just go ahead, i would like to review your patch for this. Thanks. > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
Apache-HBase commented on pull request #3372: URL: https://github.com/apache/hbase/pull/3372#issuecomment-869614204 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 3s | master passed | | +1 :green_heart: | compile | 2m 36s | master passed | | +1 :green_heart: | shadedjars | 10m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 18s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 52s | the patch passed | | +1 :green_heart: | compile | 2m 18s | the patch passed | | +1 :green_heart: | javac | 2m 18s | the patch passed | | +1 :green_heart: | shadedjars | 9m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 15s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 43s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 181m 30s | hbase-server in the patch passed. | | | | 230m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3372 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7d198451a57a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/testReport/ | | Max. process+thread count | 2763 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on pull request #3425: HBASE-25991 Do compaction on compaction server
nyl3532016 commented on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869608162 > Please add more javadoc for CompactionServer related class on how to share the same logic with HRegionServer? I do not think we want to duplicate the code twice... `CompactionThreadManager` reuse `HStore.selectCompaction`,`HStore.throttleCompaction`,`CompactionContext.compact` of `HRegionserver`, which are core logic of compaction. Yes,we have a few duplicate code, just like compaction thread pool and CompactionRunner. They are simple, and a little strange to reuse these code from `CompactSplit` class -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3426: HBASE-26032 Make HRegion.getStores() an O(1) operation
Apache-HBase commented on pull request #3426: URL: https://github.com/apache/hbase/pull/3426#issuecomment-869597221 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | compile | 3m 43s | master passed | | +1 :green_heart: | checkstyle | 1m 25s | master passed | | +1 :green_heart: | spotbugs | 2m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 3m 48s | the patch passed | | +1 :green_heart: | javac | 3m 48s | the patch passed | | -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 6 new + 249 unchanged - 0 fixed = 255 total (was 249) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 51m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3426 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5d14063c651a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server hbase-examples U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3426/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
Apache-HBase commented on pull request #3372: URL: https://github.com/apache/hbase/pull/3372#issuecomment-869575765 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | compile | 1m 49s | master passed | | +1 :green_heart: | shadedjars | 8m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 23s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 1m 52s | the patch passed | | +1 :green_heart: | javac | 1m 52s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 46s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 15s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 130m 7s | hbase-server in the patch passed. | | | | 167m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3372 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 65e16b7d73b5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/testReport/ | | Max. process+thread count | 4095 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] jojochuang commented on pull request #3426: HBASE-26032 Make HRegion.getStores() an O(1) operation
jojochuang commented on pull request #3426: URL: https://github.com/apache/hbase/pull/3426#issuecomment-869568788 Ok I came up with a hack to maintain backward compat: We can retain the return type of getStores. For HRegion, it returns a special List subclass that behaves just like a Collection: pass the Collection object, delegate operations to it, and reject any List-only operation. There is no operation in the HBase code that must require performing List operations on the stores. If a coprocessor need List semantics, it can easily convert to a real list. See if you guys like this approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25985) ReplicationSourceWALReader#run - Reset sleepMultiplier in loop once out of any IOE
[ https://issues.apache.org/jira/browse/HBASE-25985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John resolved HBASE-25985. Resolution: Invalid > ReplicationSourceWALReader#run - Reset sleepMultiplier in loop once out of > any IOE > -- > > Key: HBASE-25985 > URL: https://issues.apache.org/jira/browse/HBASE-25985 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
Apache-HBase commented on pull request #3372: URL: https://github.com/apache/hbase/pull/3372#issuecomment-869507843 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | compile | 5m 1s | master passed | | +1 :green_heart: | checkstyle | 2m 4s | master passed | | +0 :ok: | refguide | 3m 40s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 3m 56s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 5m 6s | the patch passed | | +1 :green_heart: | javac | 5m 6s | the patch passed | | +1 :green_heart: | checkstyle | 0m 23s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 29s | hbase-client: The patch generated 0 new + 29 unchanged - 1 fixed = 29 total (was 30) | | +1 :green_heart: | checkstyle | 1m 9s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 3m 40s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 20m 1s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | -1 :x: | spotbugs | 2m 25s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 69m 17s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Nullcheck of tblDesc at line 191 of value previously dereferenced in org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorker.calculatePlans(TableName) At RegionNormalizerWorker.java:191 of value previously dereferenced in org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorker.calculatePlans(TableName) At RegionNormalizerWorker.java:[line 191] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3372 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide xml | | uname | Linux 774707785bf0 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/artifact/yetus-general-check/output/branch-site/book.html | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/artifact/yetus-general-check/output/patch-site/book.html | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3372/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3430: HBASE-26029 It is not reliable to use nodeDeleted event to track regi…
Apache-HBase commented on pull request #3430: URL: https://github.com/apache/hbase/pull/3430#issuecomment-869480710 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 48s | master passed | | +1 :green_heart: | compile | 2m 45s | master passed | | +1 :green_heart: | shadedjars | 9m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 51s | the patch passed | | +1 :green_heart: | compile | 2m 42s | the patch passed | | +1 :green_heart: | javac | 2m 42s | the patch passed | | +1 :green_heart: | shadedjars | 9m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 5s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 41s | hbase-replication in the patch passed. | | +1 :green_heart: | unit | 183m 4s | hbase-server in the patch passed. | | | | 225m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3430/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3430 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 1f47d477b1fc 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3430/3/testReport/ | | Max. process+thread count | 2683 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3430/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370485#comment-17370485 ] Wei-Chiu Chuang commented on HBASE-21946: - I have a patch that uses DFSInputStream.hasCapability() API to detect which read() API to use. Using this approach, I can come up with a solution that is compatible with any Hadoop version >= 2.9.1. [~openinx] can i take over this jira? > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3431: HBASE-26021 Fix TableDescriptor serialization to maintain compatibility with HBase 2
Apache-HBase commented on pull request #3431: URL: https://github.com/apache/hbase/pull/3431#issuecomment-869456579 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 51s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 57s | branch-1 passed | | +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 44s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 41s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 8s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 5s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 52s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | checkstyle | 1m 39s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 33s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 2m 54s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 142m 51s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 191m 3s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase | | | hadoop.hbase.master.TestMasterFailover | | | hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole | | | hadoop.hbase.procedure.TestFailedProcCleanup | | | hadoop.hbase.master.handler.TestEnableTableHandler | | | hadoop.hbase.master.TestMasterRestartAfterDisablingTable | | | hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3431/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3431 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 144ce4cc3759 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3431/out/precommit/personality/provided.sh | | git revision | branch-1 / b2f8ec9 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3431/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3431/2/testReport/ | | Max. process+thread count | 4587 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3431/2/console | | versions | git=1.9.1
[jira] [Updated] (HBASE-26018) Perf improvement in L1 cache
[ https://issues.apache.org/jira/browse/HBASE-26018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-26018: - Status: Patch Available (was: In Progress) > Perf improvement in L1 cache > > > Key: HBASE-26018 > URL: https://issues.apache.org/jira/browse/HBASE-26018 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.4, 2.3.5, 3.0.0-alpha-1 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5 > > Attachments: computeIfPresent.png > > > After HBASE-25698 is in, all L1 caching strategies perform buffer.retain() in > order to maintain refCount atomically while retrieving cached blocks > (CHM#computeIfPresent). Retaining refCount is coming up bit expensive in > terms of performance. Using computeIfPresent API, CHM uses coarse grained > segment locking and even if our computation is not so complex (we just call > block retain API), it will block other update APIs for the keys within bucket > that is locked. computeIfPresent keeps showing up on flame graphs as well > (attached one of them). Specifically when we see aggressive cache hits for > meta blocks (with major blocks in cache), we would want to get away from > coarse grained locking. > One of the suggestions came up while reviewing PR#3215 is to treat cache read > API as optimistic read and deal with block retain based refCount issues by > catching the respective Exception and let it treat as cache miss. This should > allow us to go ahead with lockless get API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3425: HBASE-25991 Do compaction on compaction server
Apache-HBase commented on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869451882 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 21s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-25714 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 40s | HBASE-25714 passed | | +1 :green_heart: | compile | 1m 4s | HBASE-25714 passed | | +1 :green_heart: | shadedjars | 8m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 7m 53s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 59s | hbase-server in the patch passed. | | | | 183m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3425 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ecd34a94fc3f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / da0fa3000e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/4/testReport/ | | Max. process+thread count | 4329 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3430: HBASE-26029 It is not reliable to use nodeDeleted event to track regi…
Apache-HBase commented on pull request #3430: URL: https://github.com/apache/hbase/pull/3430#issuecomment-869441656 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 48s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 3s | master passed | | +1 :green_heart: | compile | 2m 13s | master passed | | +1 :green_heart: | shadedjars | 8m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 2m 1s | the patch passed | | +1 :green_heart: | javac | 2m 1s | the patch passed | | +1 :green_heart: | shadedjars | 8m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 48s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 25s | hbase-replication in the patch passed. | | +1 :green_heart: | unit | 135m 0s | hbase-server in the patch passed. | | | | 174m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3430/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3430 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 3f2bacde6252 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 22ec681ad9 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3430/3/testReport/ | | Max. process+thread count | 3491 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3430/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3425: HBASE-25991 Do compaction on compaction server
Apache-HBase commented on pull request #3425: URL: https://github.com/apache/hbase/pull/3425#issuecomment-869441238 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-25714 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | HBASE-25714 passed | | +1 :green_heart: | compile | 1m 15s | HBASE-25714 passed | | +1 :green_heart: | shadedjars | 7m 58s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | HBASE-25714 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | the patch passed | | +1 :green_heart: | compile | 1m 14s | the patch passed | | +1 :green_heart: | javac | 1m 14s | the patch passed | | +1 :green_heart: | shadedjars | 9m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 141m 54s | hbase-server in the patch passed. | | | | 174m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3425 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d5707c157aab 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-25714 / da0fa3000e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/4/testReport/ | | Max. process+thread count | 4271 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3425/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26027) The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
[
https://issues.apache.org/jira/browse/HBASE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Wang updated HBASE-26027:
---
Description:
The batch api of HTable contains a param named results to store result or
exception, its type is Object[].
If user pass an array with other type, eg:
org.apache.hadoop.hbase.client.Result, and if we need to put an exception into
it by some reason, then the ArrayStoreException will occur in
AsyncRequestFutureImpl.updateResult, then the
AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
actionsInProgress again and again, forever.
It is better to add an cutoff calculated by operationTimeout, instead of only
depend on the value of actionsInProgress.
BTW, this issue only for 2.x, since 3.x the implement has refactored.
How to reproduce:
1: add sleep in RSRpcServices.multi to mock slow response
{code:java}
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
{code}
2: set time out in config
{code:java}
conf.set("hbase.rpc.timeout","2000");
conf.set("hbase.client.operation.timeout","6000");
{code}
3: call batch api
{code:java}
Table table = HbaseUtil.getTable("test");
byte[] cf = Bytes.toBytes("f");
byte[] c = Bytes.toBytes("c1");
List gets = new ArrayList<>();
for (int i = 0; i < 10; i++) {
byte[] rk = Bytes.toBytes("rk-" + i);
Get get = new Get(rk);
get.addColumn(cf, c);
gets.add(get);
}
Result[] results = new Result[gets.size()];
table.batch(gets, results);
{code}
The log will looks like below:
{code:java}
[ERROR] [2021/06/22 23:23:00,676] hconnection-0x6b927fb-shared-pool3-t1 - id=1
error for test processing localhost,16020,1624343786295
java.lang.ArrayStoreException: org.apache.hadoop.hbase.DoNotRetryIOException
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.updateResult(AsyncRequestFutureImpl.java:1242)
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.trySetResultSimple(AsyncRequestFutureImpl.java:1087)
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.setError(AsyncRequestFutureImpl.java:1021)
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.manageError(AsyncRequestFutureImpl.java:683)
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.receiveGlobalFailure(AsyncRequestFutureImpl.java:716)
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.access$1500(AsyncRequestFutureImpl.java:69)
at
org.apache.hadoop.hbase.client.AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncRequestFutureImpl.java:219)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO ] [2021/06/22 23:23:10,375] main - #1, waiting for 10 actions to finish
on table: test
[INFO ] [2021/06/22 23:23:20,378] main - #1, waiting for 10 actions to finish
on table: test
[INFO ] [2021/06/22 23:23:30,384] main - #1, waiting for 10 actions to finish
on table:
[INFO ] [2021/06/22 23:23:40,387] main - #1, waiting for 10 actions to finish
on table: test
[INFO ] [2021/06/22 23:23:50,397] main - #1, waiting for 10 actions to finish
on table: test
[INFO ] [2021/06/22 23:24:00,400] main - #1, waiting for 10 actions to finish
on table: test
[INFO ] [2021/06/22 23:24:10,408] main - #1, waiting for 10 actions to finish
on table: test
[INFO ] [2021/06/22 23:24:20,413] main - #1, waiting for 10 actions to finish
on table: test
{code}
was:
The batch api of HTable contains a param named results to store result or
exception, its type is Object[].
If user pass an array with other type, eg:
org.apache.hadoop.hbase.client.Result, and we need to put an exception into it
by some reason, then the ArrayStoreException will occur in
AsyncRequestFutureImpl.updateResult, then the
AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
actionsInProgress again and again, forever.
It is better to add an cutoff calculated by operationTimeout, instead of only
depend on the value of actionsInProgress.
BTW, this issue only for 2.x, since 3.x the implement has refactored.
How to reproduce:
1: add sleep in RSRpcServices.multi to mock slow response
{code:java}
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
{code}
2: set time out in config
{code:java}
conf.set("hbase.rpc.timeout","2000");
conf.set("hbase.client.operation.timeout","6000");
{code}
3: call
[GitHub] [hbase] virajjasani closed pull request #3431: HBASE-26021 Fix TableDescriptor serialization to maintain compatibility with HBase 2
virajjasani closed pull request #3431: URL: https://github.com/apache/hbase/pull/3431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370439#comment-17370439
]
Viraj Jasani commented on HBASE-26021:
--
{quote}Coming to the fix, isn't the patch incomplete ? If we just serialize HTD
instead of TD (=HTD + table state), there is loss of information, right?
{quote}
Yes that is correct. Within HBase 1, this is possible. Since I was focusing on
HBase 1 to 2 testing, table state was not a concern for my testing. But yes,
this is not a proper fix. We will need proper revert backports (after
identifying which all reverts are applicable in branch-1) because table state
is lost in HBase 1 without fixing this.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R^Dmeta
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
>
