[ https://issues.apache.org/jira/browse/HDFS-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624225#comment-16624225 ]
Steve Loughran commented on HDFS-13713: --------------------------------------- patch 004. Review, with a fair amount of revision of the tests; little bit on the API FileSystemMultipartUploader * switch IOE on complete to PathIsDirectoryException if dest is now a dir * move to map<> over list. This simplifies testing as it removes the need for a specific test (duplicates) * add some reusable helper methods to validate preconditions S3AMultipartUploader * Support 0-byte parts. Needed to allow a 0-byte file upload. AbstractContractMultipartUploaderTest * MPU create done in test setup * Created upload handle is *always* * factored out core put/abort operations * track the handle and dest file of the upload used in the test, always abort in test teardown. Some up us get billed by AWS! * test that 0-byte files can be uploaded ITestS3AContractMultipartUploader * switch to scale test timeout, as one of tests (testMultipartUploadAbort before I changed the block count) was timing out filesystem.md * clarify that the ordering of parts is the "natural" ordering; part 1 comes first. testing: local FS & HDFS, remote : s3 ireland w/ s3guard enabled. made sure there weren't any outstanding uploads afterwards. I'm happy with this: the API is good and with the 0-byte uploads test working, rounded out functionality. > Add specification of Multipart Upload API to FS specification, with contract > tests > ---------------------------------------------------------------------------------- > > Key: HDFS-13713 > URL: https://issues.apache.org/jira/browse/HDFS-13713 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs, test > Affects Versions: 3.2.0 > Reporter: Steve Loughran > Assignee: Ewan Higgs > Priority: Blocker > Attachments: HADOOP-13713-004.patch, HDFS-13713.001.patch, > HDFS-13713.002.patch, HDFS-13713.003.patch, multipartuploader.md > > > There's nothing in the FS spec covering the new API. Add it in a new .md file > * add FS model with the notion of a function mapping (uploadID -> Upload), > the operations (list, commit, abort). The [TLA+ > mode|https://issues.apache.org/jira/secure/attachment/12865161/objectstore.pdf]l > of HADOOP-13786 shows how to do this. > * Contract tests of not just the successful path, but all the invalid ones. > * implementations of the contract tests of all FSs which support the new API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org