[ https://issues.apache.org/jira/browse/HDFS-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787430#comment-13787430 ]
Hadoop QA commented on HDFS-5299: --------------------------------- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607029/HDFS-5299.000.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5110//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5110//console This message is automatically generated. > DFS client hangs in updatePipeline RPC when failover happened > ------------------------------------------------------------- > > Key: HDFS-5299 > URL: https://issues.apache.org/jira/browse/HDFS-5299 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 3.0.0, 2.1.0-beta > Reporter: Vinay > Assignee: Vinay > Priority: Blocker > Attachments: HDFS-5299.000.patch, HDFS-5299.patch > > > DFSClient got hanged in updatedPipeline call to namenode when the failover > happened at exactly sametime. > When we digged down, issue found to be with handling the RetryCache in > updatePipeline. > Here are the steps : > 1. Client was writing slowly. > 2. One of the datanode was down and updatePipeline was called to ANN. > 3. Call reached the ANN, while processing updatePipeline call it got shutdown. > 3. Now Client retried (Since the api marked as AtMostOnce) to another > NameNode. at that time still NN was in STANDBY and got StandbyException. > 4. Now one more time client failover happened. > 5. Now SNN became Active. > 6. Client called to current ANN again for updatePipeline, > Now client call got hanged in NN, waiting for the cached call with same > callid to be over. But this cached call is already got over last time with > StandbyException. > Conclusion : > Always whenever the new entry is added to cache we need to update the result > of the call before returning the call or throwing exception. > I can see similar issue multiple RPCs in FSNameSystem. -- This message was sent by Atlassian JIRA (v6.1#6144)