[
https://issues.apache.org/jira/browse/HDFS-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167188#comment-15167188
]
Rakesh R commented on HDFS-8786:
--------------------------------
Thanks [~jingzhao] for the useful comments.
bq. We can create replication task for an EC block only when we can make sure
we have all the internal blocks (the source nodes can be in
decommissioned/decommissioning state). Otherwise even if we have
decommissioned/decommissioning datanodes, we should still trigger an EC
reconstruction task.
I will add an extra check against the {{BlockManager#corruptReplicas}} map. If
true will trigger EC reconstruction task otherwise replication task. OK?
bq. For changes in BlockInfoStriped, instead of directly mixing decommission
related logic inside, it will be better to add a sorting logic in BlockManager
just as what we do for contiguous blocks.
I think you are suggesting to provide a comparator similar to
{{DFSUtil.DECOM_COMPARATOR}}. Just adding a thought to understand more on this
change -> I could see for a (m+k) block group, (m+k) storage units are sorted.
say (0,1,2,3,4). With the current behavior it will add new storage unit at the
end of the list (0,1,2,3,4,0) and later after the removal of decommission node,
the indices list will become (1,2,3,4,5,0). Will this break the sorted order
behavior? Please correct me if I'm missing anything. Thanks!
> Erasure coding: DataNode should transfer striped blocks before being
> decommissioned
> -----------------------------------------------------------------------------------
>
> Key: HDFS-8786
> URL: https://issues.apache.org/jira/browse/HDFS-8786
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Rakesh R
> Attachments: HDFS-8786-001.patch, HDFS-8786-draft.patch
>
>
> Per [discussion |
> https://issues.apache.org/jira/browse/HDFS-8697?focusedCommentId=14609004&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14609004]
> under HDFS-8697, it's too expensive to reconstruct block groups for decomm
> purpose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
