[ https://issues.apache.org/jira/browse/HDFS-16283?focusedWorklogId=788968&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-788968 ]
ASF GitHub Bot logged work on HDFS-16283: ----------------------------------------- Author: ASF GitHub Bot Created on: 08/Jul/22 12:54 Start Date: 08/Jul/22 12:54 Worklog Time Spent: 10m Work Description: ZanderXu commented on code in PR #4524: URL: https://github.com/apache/hadoop/pull/4524#discussion_r916788569 ########## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java: ########## @@ -759,14 +764,49 @@ public boolean mkdirs(String src, FsPermission masked, boolean createParent) } } + private Map<String, FederationNamespaceInfo> getAvailableNamespaces() + throws IOException { + Map<String, FederationNamespaceInfo> allAvailableNamespaces = + new HashMap<>(); + namenodeResolver.getNamespaces().forEach( + k -> allAvailableNamespaces.put(k.getNameserviceId(), k)); + return allAvailableNamespaces; + } + + /** + * Try to get a list of FederationNamespaceInfo for renewLease RPC. + */ + private List<FederationNamespaceInfo> getRewLeaseNSs(List<String> namespaces) + throws IOException { + if (namespaces == null || namespaces.isEmpty()) { + return new ArrayList<>(namenodeResolver.getNamespaces()); + } + List<FederationNamespaceInfo> result = new ArrayList<>(); + Map<String, FederationNamespaceInfo> allAvailableNamespaces = + getAvailableNamespaces(); + for (String namespace : namespaces) { + if (!allAvailableNamespaces.containsKey(namespace)) { + return new ArrayList<>(namenodeResolver.getNamespaces()); Review Comment: `namenodeResolver.getNamespaces()` is a hashSet, I want to a List so that we can use `invokeSingle` method to forward this rpc when there is only one namespace. ``` List<FederationNamespaceInfo> nss = getRenewLeaseNSs(namespaces); if (nss.size() == 1) { rpcClient.invokeSingle(nss.get(0).getNameserviceId(), method); } else { rpcClient.invokeConcurrent(nss, method, false, false); } ``` Of course, `Set` can also achieve this goal. Issue Time Tracking ------------------- Worklog Id: (was: 788968) Time Spent: 6h (was: 5h 50m) > RBF: improve renewLease() to call only a specific NameNode rather than make > fan-out calls > ----------------------------------------------------------------------------------------- > > Key: HDFS-16283 > URL: https://issues.apache.org/jira/browse/HDFS-16283 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf > Reporter: Aihua Xu > Assignee: Aihua Xu > Priority: Major > Labels: pull-request-available > Attachments: RBF_ improve renewLease() to call only a specific > NameNode rather than make fan-out calls.pdf > > Time Spent: 6h > Remaining Estimate: 0h > > Currently renewLease() against a router will make fan-out to all the > NameNodes. Since renewLease() call is so frequent and if one of the NameNodes > are slow, then eventually the router queues are blocked by all renewLease() > and cause router degradation. > We will make a change in the client side to keep track of NameNode Id in > additional to current fileId so routers understand which NameNodes the client > is renewing lease against. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org