[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195976#comment-15195976
]
Andrew Wang commented on HDFS-3702:
-----------------------------------
Since we're just adding a flag to the existing {{create}} flags enumset, it
doesn't affect our API signature. Note there are no changes in FileSystem or
DistributedFileSystem. It also doesn't involve any NN memory overhead, which is
a nice bonus compared to a storage policy with xattrs.
I also like this scheme also since it gives us a lot of flexibility at the
application level. For example, applications like distcp or the httpfs and nfs
gateway might always want this flag on (no matter the destination folder), to
avoid data load imbalance. For HBase's WAL, it would give them the flexibility
to redo their filesystem layout, for instance if all WALs no longer go in a
single "/logs" directory.
Overall, it feels a lot like Linux-y filesystem hints like fadvise / madvise,
and a good use of flags.
> Add an option for NOT writing the blocks locally if there is a datanode on
> the same box as the client
> -----------------------------------------------------------------------------------------------------
>
> Key: HDFS-3702
> URL: https://issues.apache.org/jira/browse/HDFS-3702
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.5.1
> Reporter: Nicolas Liochon
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: HDFS-3702.000.patch, HDFS-3702.001.patch,
> HDFS-3702.002.patch, HDFS-3702.003.patch, HDFS-3702.004.patch,
> HDFS-3702.005.patch, HDFS-3702.006.patch, HDFS-3702.007.patch,
> HDFS-3702.008.patch, HDFS-3702_Design.pdf
>
>
> This is useful for Write-Ahead-Logs: these files are writen for recovery
> only, and are not read when there are no failures.
> Taking HBase as an example, these files will be read only if the process that
> wrote them (the 'HBase regionserver') dies. This will likely come from a
> hardware failure, hence the corresponding datanode will be dead as well. So
> we're writing 3 replicas, but in reality only 2 of them are really useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
