[jira] [Created] (HDFS-8574) When block count exceeds dfs.blockreport.split.threshold, the block report are sent in one per message

[Ajith S (JIRA)]

Ajith S created HDFS-8574:
-----------------------------

             Summary: When block count exceeds dfs.blockreport.split.threshold, 
the block report are sent in one per message
                 Key: HDFS-8574
                 URL: https://issues.apache.org/jira/browse/HDFS-8574
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.7.0
            Reporter: Ajith S
            Assignee: Ajith S


This piece of code in 
{{org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport()}}

{code}
// Send one block report per message.
        for (int r = 0; r < reports.length; r++) {
          StorageBlockReport singleReport[] = { reports[r] };
          DatanodeCommand cmd = bpNamenode.blockReport(
              bpRegistration, bpos.getBlockPoolId(), singleReport,
              new BlockReportContext(reports.length, r, reportId));
          numReportsSent++;
          numRPCs++;
          if (cmd != null) {
            cmds.add(cmd);
          }
{code}

is creating many cmds in case the block count exceeds the 
{{dfs.blockreport.split.threshold}} limit. A better way for this will be 
spliting the block reports in equal number of buckets of size 
{{dfs.blockreport.split.threshold}} therefore reducing the number of RPCs in 
block reporting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)