[
https://issues.apache.org/jira/browse/HADOOP-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12479369
]
dhruba borthakur commented on HADOOP-1079:
------------------------------------------
You are right in that block-reports exists only to catch exception scenarios.
Suppose a datanode fails to successfully deliver a blockreceived mesage to the
namenode. The namenode won't know about it till the next block report. Also, if
a rougue application deletes blk-* files on the datanode or a portion of the
data directories go bad, this will be detected by the next block report.
My proposal is the make the datanode remember that report that it sent to the
namenode the last time. Only those blocks that got added/deleted since the last
block report will be sent in the next incremental block report. This is a
mechanism to ensure that the namenode and datanode are in sych as far as block
information is concerned.
In my test case, there were no two succeeding hourly block reports that were
identical. I had 1800 data nodes and was running randomWriter. In this
scenario, using a hash to identify identical block reports might not buy us
anything.
> DFS Scalability: optimize processing time of block reports
> ----------------------------------------------------------
>
> Key: HADOOP-1079
> URL: https://issues.apache.org/jira/browse/HADOOP-1079
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
>
> I have a cluster that has 1800 datanodes. Each datanode has around 50000
> blocks and sends a block report to the namenode once every hour. This means
> that the namenode processes a block report once every 2 seconds. Each block
> report contains all blocks that the datanode currently hosts. This makes the
> namenode compare a huge number of blocks that practically remains the same
> between two consecutive reports. This wastes CPU on the namenode.
> The problem becomes worse when the number of datanodes increases.
> One proposal is to make succeeding block reports (after a successful send of
> a full block report) be incremental. This will make the namenode process only
> those blocks that were added/deleted in the last period.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
