[
https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382723#comment-14382723
]
Rashmi Vinayak commented on HDFS-7715:
--------------------------------------
Hi Kai,
>So that means for all the three modes/versions of the algorithm, we can have
>flexible parameters rather than hard-coded values, right.
Yes, all three modes of the algorithm can have flexible parameters without
hard-coded values.
Re. using raw coders: By using raw coders, do you mean first encoding using RS
completely and then modify it for Hitchhiker? There will be some performance
impact if done this way since the encoder will have to process all the data
twice: first to RS encode and then to do modifications required for Hitchhiker.
But as you pointed, for the sake of simplicity of the implementation, it might
be fine to try this approach and see how much is the performance impact.
> Implement the Hitchhiker erasure coding algorithm
> -------------------------------------------------
>
> Key: HDFS-7715
> URL: https://issues.apache.org/jira/browse/HDFS-7715
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: jack liuquan
> Attachments: HDFS-7715-hhxor-decoder.patch,
> HDFS-7715-hhxor-encoder.patch
>
>
> [Hitchhiker |
> http://www.eecs.berkeley.edu/~nihar/publications/Hitchhiker_SIGCOMM14.pdf] is
> a new erasure coding algorithm developed as a research project at UC
> Berkeley. It has been shown to reduce network traffic and disk I/O by 25%-45%
> during data reconstruction. This JIRA aims to introduce Hitchhiker to the
> HDFS-EC framework, as one of the pluggable codec algorithms.
> The existing implementation is based on HDFS-RAID.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
