[
https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386845#comment-15386845
]
Konstantin Shvachko commented on HDFS-10301:
--------------------------------------------
As I commented earlier I am not in favor of adding redundant fields. The
readability argument is also quite questionable, because you end up either
filling storage information in two fields, or sending it in different fields
for different types of block report messages.
In more details:
- Suppose we introduced {{repeated String allStorageIds}}.
- In full report (which is not split into multiple RPCs) we already have all
storage ids listed in StorageBlockReports. And we don't need {{allStorageIds}}.
If we nevertheless fill {{allStorageIds}} it will be confusing.
- In a report that is split into multiple RPCs we fill {{allStorageIds}},
because only one storage is reported. So in this case we will use a different
field to pass storageIDs.
- I think code is more _readable_ when the same information is passed via the
same fields, and is not duplicated.
> BlockReport retransmissions may lead to storages falsely being declared
> zombie if storage report processing happens out of order
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-10301
> URL: https://issues.apache.org/jira/browse/HDFS-10301
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.6.1
> Reporter: Konstantin Shvachko
> Assignee: Vinitha Reddy Gankidi
> Priority: Critical
> Attachments: HDFS-10301.002.patch, HDFS-10301.003.patch,
> HDFS-10301.004.patch, HDFS-10301.005.patch, HDFS-10301.006.patch,
> HDFS-10301.007.patch, HDFS-10301.008.patch, HDFS-10301.009.patch,
> HDFS-10301.01.patch, HDFS-10301.010.patch, HDFS-10301.011.patch,
> HDFS-10301.sample.patch, zombieStorageLogs.rtf
>
>
> When NameNode is busy a DataNode can timeout sending a block report. Then it
> sends the block report again. Then NameNode while process these two reports
> at the same time can interleave processing storages from different reports.
> This screws up the blockReportId field, which makes NameNode think that some
> storages are zombie. Replicas from zombie storages are immediately removed,
> causing missing blocks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
