[ https://issues.apache.org/jira/browse/HDFS-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272795#comment-17272795 ]
Wei-Chiu Chuang commented on HDFS-15794: ---------------------------------------- I think we made quite some improvements recently in here. One of which is HDFS-14997. > IBR and FBR use different queues to load data. > ---------------------------------------------- > > Key: HDFS-15794 > URL: https://issues.apache.org/jira/browse/HDFS-15794 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode > Reporter: JiangHua Zhu > Assignee: JiangHua Zhu > Priority: Major > > When DataNode reports data to NameNode, IBR and FBR are included here. > After the NameNode receives the DataNode request, it temporarily stores the > data in a queue, here it refers to > BlockManager#BlockReportProcessingThread#queue. > NameNodeRpcServer#blockReport() > for (int r = 0; r <reports.length; r++) { > final BlockListAsLongs blocks = reports[r].getBlocks(); > final int index = r; > noStaleStorages = bm.runBlockOp(() -> > bm.processReport(nodeReg, reports[index].getStorage(), > blocks, context)); > } > NameNodeRpcServer#blockReport() > for (final StorageReceivedDeletedBlocks r: receivedAndDeletedBlocks) { > bm.enqueueBlockOp(new Runnable() { > @Override > public void run() { > try { > namesystem.processIncrementalBlockReport(nodeReg, r); > } catch (Exception ex) { > // usually because the node is unregistered/dead. next heartbeat > // will correct the problem > blockStateChangeLog.error( > "*BLOCK* NameNode.blockReceivedAndDeleted: " > + "failed from "+ nodeReg + ":" + ex.getMessage()); > } > } > }); > } > The problem here is that when the NameNode is blocked in processing the IBR, > the FBR requested by the DN from the NameNode will be affected. Similarly, > when the NameNode processing FBR is blocked. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org