[
https://issues.apache.org/jira/browse/HDFS-15900?focusedWorklogId=570164&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-570164
]
ASF GitHub Bot logged work on HDFS-15900:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Mar/21 00:42
Start Date: 23/Mar/21 00:42
Worklog Time Spent: 10m
Work Description: hdaikoku commented on a change in pull request #2787:
URL: https://github.com/apache/hadoop/pull/2787#discussion_r599167124
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/TestStateStoreMembershipState.java
##########
@@ -473,6 +478,53 @@ public void testRegistrationExpiredAndDeletion()
}, 100, 3000);
}
+ @Test
+ public void testNamespaceInfoWithUnavailableNameNodeRegistration() throws
IOException {
+ // Populate the state store with one ACTIVE NameNode entry and one
UNAVAILABLE NameNode entry
+ // 1) ns0:nn0 - ACTIVE
+ // 2) ns0:nn1 - UNAVAILABLE
+ List<MembershipState> registrationList = new ArrayList<>();
+ String router = ROUTERS[0];
+ String ns = NAMESERVICES[0];
+ String rpcAddress = "testrpcaddress";
+ String serviceAddress = "testserviceaddress";
+ String lifelineAddress = "testlifelineaddress";
+ String blockPoolId = "testblockpool";
+ String clusterId = "testcluster";
+ String webScheme = "http";
+ String webAddress = "testwebaddress";
+ boolean safemode = false;
+
+ MembershipState record = MembershipState.newInstance(
+ router, ns, NAMENODES[0], clusterId, blockPoolId,
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.ACTIVE, safemode);
+ registrationList.add(record);
+
+ // Set empty clusterId and blockPoolId for UNAVAILABLE NameNode
+ record = MembershipState.newInstance(
+ router, ns, NAMENODES[1], "", "",
+ rpcAddress, serviceAddress, lifelineAddress, webScheme,
+ webAddress, FederationNamenodeServiceState.UNAVAILABLE, safemode);
+ registrationList.add(record);
+
+ registerAndLoadRegistrations(registrationList);
+
+ GetNamespaceInfoRequest request = GetNamespaceInfoRequest.newInstance();
+ GetNamespaceInfoResponse response =
membershipStore.getNamespaceInfo(request);
+ Set<FederationNamespaceInfo> namespaces = response.getNamespaceInfo();
+
+ // Verify only one namespace is registered
+ assertThat(namespaces).hasSize(1);
+
+ // Verify the registered namespace has a valid pair of clusterId and
blockPoolId derived from ACTIVE NameNode
+ for (FederationNamespaceInfo namespace : namespaces) {
Review comment:
Agreed. Modified to verify only the first element from the iterator:
04ab61c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 570164)
Time Spent: 1h 50m (was: 1h 40m)
> RBF: empty blockpool id on dfsrouter caused by UNAVAILABLE NameNode
> -------------------------------------------------------------------
>
> Key: HDFS-15900
> URL: https://issues.apache.org/jira/browse/HDFS-15900
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
> Affects Versions: 3.3.0
> Reporter: Harunobu Daikoku
> Assignee: Harunobu Daikoku
> Priority: Major
> Labels: pull-request-available
> Attachments: image.png
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> We observed that when a NameNode becomes UNAVAILABLE, the corresponding
> blockpool id in MembershipStoreImpl#activeNamespaces on dfsrouter
> unintentionally sets to empty, its initial value.
> !image.png|height=250!
> As a result of this, concat operations through dfsrouter fail with the
> following error as it cannot resolve the block id in the recognized active
> namespaces.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RemoteException):
> Cannot locate a nameservice for block pool BP-...
> {noformat}
> A possible fix is to ignore UNAVAILABLE NameNode registrations, and set
> proper namespace information obtained from available NameNode registrations
> when constructing the cache of active namespaces.
>
> [https://github.com/apache/hadoop/blob/rel/release-3.3.0/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java#L207-L221]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
