ehiggs commented on a change in pull request #609: HADOOP-16193. add extra S3A
MPU test to see what happens if a file is created during the MPU
URL: https://github.com/apache/hadoop/pull/609#discussion_r274229987
##########
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractMultipartUploader.java
##########
@@ -159,4 +170,47 @@ public void testDirectoryInTheWay() throws Exception {
public void testMultipartUploadReverseOrder() throws Exception {
ContractTestUtils.skip("skipped for speed");
}
+
+ /**
+ * This creates and then deletes a zero-byte file while an upload
+ * is in progress, and verifies that the uploaded file is ultimately
+ * visible.
+ */
+ @Test
+ public void testMultipartOverlapWithTransientFile() throws Throwable {
+ // until there's a way to explicitly ask for a multipart uploader from a
+ // specific FS, explicitly create one bonded to the raw FS.
+ describe("testMultipartOverlapWithTransientFile");
+ S3AFileSystem fs = getFileSystem();
+ Path path = path("testMultipartOverlapWithTransientFile");
+ fs.delete(path, true);
+ MultipartUploader mpu = mpu(1);
+ UploadHandle upload1 = mpu.initialize(path);
+ byte[] dataset = dataset(1024, '0', 10);
+ final Map<Integer, PartHandle> handles = new HashMap<>();
+ LOG.info("Uploading multipart entry");
+ PartHandle value = mpu.putPart(path, new ByteArrayInputStream(dataset), 1,
+ upload1,
+ dataset.length);
+ // upload 1K
+ handles.put(1, value);
+ // confirm the path is absent
+ ContractTestUtils.assertPathDoesNotExist(fs,
+ "path being uploaded", path);
+ // now create an empty file
+ ContractTestUtils.touch(fs, path);
+ final FileStatus touchStatus = fs.getFileStatus(path);
+ LOG.info("0-byte file has been created: {}", touchStatus);
+ fs.delete(path, false);
+ // now complete the upload
+ mpu.complete(path, handles, upload1);
Review comment:
That you can use it to open the file again and read it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
