[
https://issues.apache.org/jira/browse/HDFS-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589542#comment-13589542
]
Jonathan Hsieh commented on HDFS-2576:
--------------------------------------
[~devaraj], [~saint....@gmail.com] I basically agree with short-term assessment
of incremental work for incremental benefit, I just didn't see any discussion
about alternatives and what potential ideals are.
If we eventually went with the "ideal" approach, HDFS would still need this
hint API internally to HDFS so it is likely an incremental step. I don't think
that hbase would be the only major user of this feature -- [~atm] mentions
"file formats which will be used in joins" -- and other applications would want
low-latency access in the face of failures will want this kind of control as
well.
I don't know all the internals of the hfds balancer, but it would seem that if
it a block contained the dirent, the affinity group info should be available.
We'd probably need to define an interface and semantics where files created
under a subdir inherit the affinity of its parent and/or a mechanism to modify
affinity groups.
> Namenode should have a favored nodes hint to enable clients to have control
> over block placement.
> -------------------------------------------------------------------------------------------------
>
> Key: HDFS-2576
> URL: https://issues.apache.org/jira/browse/HDFS-2576
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Pritam Damania
> Attachments: hdfs-2576-1.txt
>
>
> Sometimes Clients like HBase are required to dynamically compute the
> datanodes it wishes to place the blocks for a file for higher level of
> locality. For this purpose there is a need of a way to give the Namenode a
> hint in terms of a favoredNodes parameter about the locations where the
> client wants to put each block. The proposed solution is a favored nodes
> parameter in the addBlock() method and in the create() file method to enable
> the clients to give the hints to the NameNode about the locations of each
> replica of the block. Note that this would be just a hint and finally the
> NameNode would look at disk usage, datanode load etc. and decide whether it
> can respect the hints or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
