[
https://issues.apache.org/jira/browse/HDFS-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667163#comment-13667163
]
Todd Lipcon commented on HDFS-4849:
-----------------------------------
The idempotent create is also potentially problematic with multi-threaded
clients. A given client may have multiple threads race to create the same file,
and those threads would share the same client name (and hence lease). Without
per-file leases, a second thread trying to create a file already being created
would end up getting back the same block ID and causing some havoc.
One solution to this would be to include a nonce in the create() call, and
store that in the INodeFileUnderConstruction, so that if you retry with the
same nonce, it would identify it correctly as a retry.
Delete is much harder to make truly idempotent without changing semantics --
I've thought about it a bit and the only "solution" I could come up with is
something like NFS's "duplicate request cache" -
http://www.freesoft.org/CIE/RFC/1813/47.htm -- but we'd have to wire it through
persistently, etc, and it's still somewhat "best effort".
> Idempotent create, append and delete operations.
> ------------------------------------------------
>
> Key: HDFS-4849
> URL: https://issues.apache.org/jira/browse/HDFS-4849
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.0.4-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
>
> create, append and delete operations can be made idempotent. This will reduce
> chances for a job or other app failures when NN fails over.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
