[ https://issues.apache.org/jira/browse/HDFS-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16643613#comment-16643613 ]
Steve Loughran commented on HDFS-13713: --------------------------------------- Now that the 0-byte HDFS write is in, the oustanding issue here is handling concurrent writes to the same destination. We could maybe be vague about what happens, i.e. * support for concurrent writes is one of : reject or commit, * if it is commit, then *eventually* the last committed object becomes what you get for a read. That is, right from the outset make it clear there's a risk Key point: the output will be isolated and atomically written Then we need the file uploader to support concurrent uploads. [~ehiggs]: do you have a chance to look @ this? > Add specification of Multipart Upload API to FS specification, with contract > tests > ---------------------------------------------------------------------------------- > > Key: HDFS-13713 > URL: https://issues.apache.org/jira/browse/HDFS-13713 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, test > Affects Versions: 3.2.0 > Reporter: Steve Loughran > Assignee: Ewan Higgs > Priority: Blocker > Attachments: HADOOP-13713-004.patch, HADOOP-13713-004.patch, > HADOOP-13713-005.patch, HADOOP-13713-006.patch, HADOOP-13713-007.patch, > HDFS-13713.001.patch, HDFS-13713.002.patch, HDFS-13713.003.patch, > multipartuploader.md > > > There's nothing in the FS spec covering the new API. Add it in a new .md file > * add FS model with the notion of a function mapping (uploadID -> Upload), > the operations (list, commit, abort). The [TLA+ > mode|https://issues.apache.org/jira/secure/attachment/12865161/objectstore.pdf]l > of HADOOP-13786 shows how to do this. > * Contract tests of not just the successful path, but all the invalid ones. > * implementations of the contract tests of all FSs which support the new API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org