[
https://issues.apache.org/jira/browse/HDFS-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631849#comment-17631849
]
ASF GitHub Bot commented on HDFS-16837:
---------------------------------------
goiri commented on code in PR #5123:
URL: https://github.com/apache/hadoop/pull/5123#discussion_r1019478547
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ClientGSIContext.java:
##########
@@ -77,12 +82,35 @@ public void
updateResponseState(RpcResponseHeaderProto.Builder header) {
@Override
public synchronized void receiveResponseState(RpcResponseHeaderProto header)
{
if (header.hasRouterFederatedState()) {
- routerFederatedState = header.getRouterFederatedState();
+ routerFederatedState =
mergeRouterFederatedState(header.getRouterFederatedState());
+ routerFederatedState =
mergeRouterFederatedState(header.getRouterFederatedState());
} else {
lastSeenStateId.accumulate(header.getStateId());
}
}
+ /**
+ * Merge the local FederatedState and RemoteFederateState to get the max
value for each namespace.
+ * @param remoteFederateState the remote RouterFederatedState.
+ * @return one ByteString object which contains the max value of each
namespace.
+ */
+ private ByteString mergeRouterFederatedState(ByteString remoteFederateState)
{
+ Map<String, Long> localMapping = new HashMap<>(
+ ByteStringUtil.getRouterFederatedStateMap(this.routerFederatedState));
+ Map<String, Long> remoteMapping =
+ ByteStringUtil.getRouterFederatedStateMap(remoteFederateState);
+
+ for (String ns : remoteMapping.keySet()) {
+ long remoteValue = remoteMapping.get(ns);
+ long localValue = localMapping.getOrDefault(ns, 0L);
+ localMapping.put(ns, Math.max(remoteValue, localValue));
+ }
+ HdfsServerFederationProtos.RouterFederatedStateProto.Builder
federatedStateBuilder =
Review Comment:
To make it shorter to read, can you import
org.apache.hadoop.hdfs.federation.protocol.proto.HdfsServerFederationProtos.RouterFederatedStateProto?
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ByteStringUtil.java:
##########
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.util;
+
+import
org.apache.hadoop.hdfs.federation.protocol.proto.HdfsServerFederationProtos;
+import org.apache.hadoop.thirdparty.protobuf.ByteString;
+import org.apache.hadoop.thirdparty.protobuf.InvalidProtocolBufferException;
+
+import java.util.Collections;
+import java.util.Map;
+
+public class ByteStringUtil {
+
+ /**
+ * Utility function to parse routerFederatedState field in RPC headers.
+ */
+ public static Map<String, Long> getRouterFederatedStateMap(
+ ByteString byteString) {
+ if (byteString != null) {
Review Comment:
To make it easier to read:
```
if (byteString == null) {
return Collections.emptyMap();
}
```
> [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max
> state id for each namespace
> --------------------------------------------------------------------------------------------------------
>
> Key: HDFS-16837
> URL: https://issues.apache.org/jira/browse/HDFS-16837
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Labels: pull-request-available
>
> ClientGSIContext should merge local and remote RouterFederatedState to get
> the max state id for each namespace.
> And the related code as bellows:
> {code:java}
> @Override
> public synchronized void receiveResponseState(RpcResponseHeaderProto header) {
> if (header.hasRouterFederatedState()) {
> // BUG here
> routerFederatedState = header.getRouterFederatedState();
> } else {
> lastSeenStateId.accumulate(header.getStateId());
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
