[
https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371697#comment-15371697
]
Colin P. McCabe commented on HDFS-10301:
----------------------------------------
I apologize for the delays in reviewing. I am looking at HDFS-10301.007.patch.
Is this the latest patch?
I don't understand the purpose behind {{BlockListAsLongs#isStorageReportOnly}}.
This function is never called. This state doesn't seem to be stored anywhere
in what is sent over the wire, either. Is this an idea that was
half-implemented, or did I miss something?
{code}
if (blocks != BlockListAsLongs.STORAGE_REPORT_ONLY) {
{code}
Again, this is comparing by object reference equality not deep equality. This
is comment I also made in the last review that wasn't addressed.
My comment earlier is that I didn't want to overload block reports to be
storage reports. A storage report is not a kind of block report. They
shouldn't be using the same protobuf objects or Java data structures. This
isn't addressed in the current patch, which continues the confusing practice of
using the same data structure for both.
bq. In the upgrade case, there is no way to detect the zombie storages since
the old DNs do not send the information about the storages in the BR in the
last RPC. In practice, hot-swapping of DN drives and upgrading the DN may not
happen at the same time.
The set of storages that the DN reports can change for a variety of reasons,
most of which are not hotswap related. One reason is because a drive has
become bad and got kicked out of the set of currently active volumes. Another
reason is because the DN got taken down by the administrator, a volume got
removed, and the DN was brought back up. It's rather frustrating that your
patch doesn't support zombie storage removal during upgrade, and mine does, and
yet [~shv] is blocking my patch.
> BlockReport retransmissions may lead to storages falsely being declared
> zombie if storage report processing happens out of order
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-10301
> URL: https://issues.apache.org/jira/browse/HDFS-10301
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.6.1
> Reporter: Konstantin Shvachko
> Assignee: Vinitha Reddy Gankidi
> Priority: Critical
> Attachments: HDFS-10301.002.patch, HDFS-10301.003.patch,
> HDFS-10301.004.patch, HDFS-10301.005.patch, HDFS-10301.006.patch,
> HDFS-10301.007.patch, HDFS-10301.01.patch, HDFS-10301.sample.patch,
> zombieStorageLogs.rtf
>
>
> When NameNode is busy a DataNode can timeout sending a block report. Then it
> sends the block report again. Then NameNode while process these two reports
> at the same time can interleave processing storages from different reports.
> This screws up the blockReportId field, which makes NameNode think that some
> storages are zombie. Replicas from zombie storages are immediately removed,
> causing missing blocks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
