[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198415#comment-15198415
]
Andrew Wang commented on HDFS-3702:
-----------------------------------
Thanks for clarifying Arpit!
Unless block placement policy and storage policies are made totally pluggable,
I think it's unlikely we'll ever be able to add new kinds of BPP hints without
changes in HDFS. BPP is somewhat pluggable today, but out-of-tree
implementations are pretty discouraged for maintenance reasons, and last I
checked storage policies are still hardcoded.
Regarding co-location, my point was that it's unlikely we can express the
colocation constraints through any of our existing APIs, and it will require
integration work by downstreams anyway. The scope of colocation is much larger
than this JIRA though, so seems like something we can discuss further somewhere
else.
Perhaps most compelling, given that this is just a hint, we have the
flexibility of turning it into a no-op later on if we get downstream feedback
about the API. Sound reasonable?
> Add an option for NOT writing the blocks locally if there is a datanode on
> the same box as the client
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-3702
> URL: https://issues.apache.org/jira/browse/HDFS-3702
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.5.1
> Reporter: Nicolas Liochon
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: HDFS-3702.000.patch, HDFS-3702.001.patch,
> HDFS-3702.002.patch, HDFS-3702.003.patch, HDFS-3702.004.patch,
> HDFS-3702.005.patch, HDFS-3702.006.patch, HDFS-3702.007.patch,
> HDFS-3702.008.patch, HDFS-3702_Design.pdf
>
>
> This is useful for Write-Ahead-Logs: these files are writen for recovery
> only, and are not read when there are no failures.
> Taking HBase as an example, these files will be read only if the process that
> wrote them (the 'HBase regionserver') dies. This will likely come from a
> hardware failure, hence the corresponding datanode will be dead as well. So
> we're writing 3 replicas, but in reality only 2 of them are really useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
