[ https://issues.apache.org/jira/browse/HDFS-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-9011: ---------------------------- Attachment: HDFS-9011.002.patch Thanks for the review, Nicholas! Update the patch to fix the failed unit tests. About the block report order, current the DN sends block report RPCs in a loop, thus looks like it is not possible that the NN gets out of order reports. {code} for (int r = 0; r < reports.size(); r++) { StorageBlockReport singleReport[] = { reports.get(r) }; DatanodeCommand cmd = bpNamenode.blockReport( bpRegistration, bpos.getBlockPoolId(), singleReport, new BlockReportContext(reports.size(), r, reportId, fullBrLeaseId)); numReportsSent++; numRPCs++; if (cmd != null) { cmds.add(cmd); } } {code} > Support splitting BlockReport of a storage into multiple RPC > ------------------------------------------------------------ > > Key: HDFS-9011 > URL: https://issues.apache.org/jira/browse/HDFS-9011 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Jing Zhao > Assignee: Jing Zhao > Attachments: HDFS-9011.000.patch, HDFS-9011.001.patch, > HDFS-9011.002.patch > > > Currently if a DataNode has too many blocks (more than 1m by default), it > sends multiple RPC to the NameNode for the block report, each RPC contains > report for a single storage. However, in practice we've seen sometimes even a > single storage can contains large amount of blocks and the report even > exceeds the max RPC data length. It may be helpful to support sending > multiple RPC for the block report of a storage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)