[ https://issues.apache.org/jira/browse/HDFS-15113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013664#comment-17013664 ]
Xiaoqiao He commented on HDFS-15113: ------------------------------------ submit v002 with unit test. Please help to take another review if have time. Thanks [~elgoiri]. > Missing IBR when NameNode restart if open processCommand async feature > ---------------------------------------------------------------------- > > Key: HDFS-15113 > URL: https://issues.apache.org/jira/browse/HDFS-15113 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode > Reporter: Xiaoqiao He > Assignee: Xiaoqiao He > Priority: Major > Attachments: HDFS-15113.001.patch, HDFS-15113.002.patch > > > Recently, I meet one case that NameNode missing block after restart which is > related with HDFS-14997. > a. during NameNode restart, it will return command `DNA_REGISTER` to DataNode > when receive some RPC request from DataNode. > b. when DataNode receive `DNA_REGISTER` command, it will run #reRegister > async. > {code:java} > void reRegister() throws IOException { > if (shouldRun()) { > // re-retrieve namespace info to make sure that, if the NN > // was restarted, we still match its version (HDFS-2120) > NamespaceInfo nsInfo = retrieveNamespaceInfo(); > // and re-register > register(nsInfo); > scheduler.scheduleHeartbeat(); > // HDFS-9917,Standby NN IBR can be very huge if standby namenode is down > // for sometime. > if (state == HAServiceState.STANDBY || state == > HAServiceState.OBSERVER) { > ibrManager.clearIBRs(); > } > } > } > {code} > c. As we know, #register will trigger BR immediately. > d. because #reRegister run async, so we could not make sure which one run > first between send FBR and clear IBR. If clean IBR run first, it will be OK. > But if send FBR first then clear IBR, it will missing some blocks received > between these two time point until next FBR. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org