[
https://issues.apache.org/jira/browse/HDFS-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729751#comment-14729751
]
Chris Douglas commented on HDFS-9020:
-------------------------------------
Talking with [~jghoman] and [~daryn], the following seems like the least
objectionable modification to WebHDFS.
# Change the server response for {{PUT}} to return a session cookie for the
stream, addressing a client cache at the DN
# Subsequent {{POST}} operations including this cookie address a client
instance, maintained per user (the session cookie is _not_ used for auth)
# If the instance is not found (possibly because the client is retrying and
directed to another server), a new client is created and the lease recovered
for the file
# {{POST}} (append) operations include an optional, expected offset for the
stream
# {{POST}} (append) operations include an optional header/trailer specifying
whether the stream should be flushed, synced, or closed
Backwards compatibility is maintained by using only the chunked {{PUT}}
operation if the server does not return a session cookie (i.e., the existing
code path).
Thoughts? I'll also attach a brief list of alternative designs for discussion.
> Support hflush/hsync in WebHDFS
> -------------------------------
>
> Key: HDFS-9020
> URL: https://issues.apache.org/jira/browse/HDFS-9020
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
> Reporter: Chris Douglas
> Attachments: HDFS-9020-alt.txt
>
>
> In the current implementation, hflush/hsync have no effect on WebHDFS
> streams, particularly w.r.t. visibility to other clients. This proposes to
> extend the protocol and implementation to enable this functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
