[ https://issues.apache.org/jira/browse/HDFS-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379277#comment-14379277 ]
Tsz Wo Nicholas Sze commented on HDFS-7344: ------------------------------------------- > In most recovery cases, each ECWorker only generates 1 block. ... For 1 missing block, we may not need to recover it at all since (6,3)-Reed-Solomon can tolerate 3 missing blocks. Also recovery is more efficient for 2- or 3- missing blocks. > Erasure Coding worker and support in DataNode > --------------------------------------------- > > Key: HDFS-7344 > URL: https://issues.apache.org/jira/browse/HDFS-7344 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode > Reporter: Kai Zheng > Assignee: Li Bo > Attachments: HDFS ECWorker Design.pdf, hdfs-ec-datanode.0108.zip, > hdfs-ec-datanode.0108.zip > > > According to HDFS-7285 and the design, this handles DataNode side extension > and related support for Erasure Coding, and implements ECWorker. It mainly > covers the following aspects, and separate tasks may be opened to handle each > of them. > * Process encoding work, calculating parity blocks as specified in block > groups and codec schema; > * Process decoding work, recovering data blocks according to block groups and > codec schema; > * Handle client requests for passive recovery blocks data and serving data on > demand while reconstructing; > * Write parity blocks according to storage policy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)