[
https://issues.apache.org/jira/browse/HDFS-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13687114#comment-13687114
]
Vinod Kumar Vavilapalli commented on HDFS-4849:
-----------------------------------------------
bq. Applications cannot set the client name. Even if they do, multiple threads
can reuse the same client name. I do not think one can assume that applications
are doing the right thing and setting a unique name. Also in case of YARN,
where multiple application masters could be running, not sure if there is
anyway to guarantee this. Yarn or MR folks, can you comment?
There is no service currently in Yarn/MR that lets one create unique IDs.
You'll need an external service like Zookeeper. Didn't completely read through
everything, but is idempotent create/append dependent on the client generating
unique names? It is an assumption that should be very explicitly communicated
to clients, but even then, will something break for someone else in the file
system if some rogue user doesn't generate unique IDs?
Also, even if YARN/MR gives a service like that, what about all other
applications which directly only deal with the DFS?
> Idempotent create and append operations.
> ----------------------------------------
>
> Key: HDFS-4849
> URL: https://issues.apache.org/jira/browse/HDFS-4849
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.0.4-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: idempotentCreate.patch, idempotentCreate.patch,
> idempotentCreate.patch
>
>
> create, append and delete operations can be made idempotent. This will reduce
> chances for a job or other app failures when NN fails over.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
