[ https://issues.apache.org/jira/browse/HADOOP-15576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565100#comment-16565100 ]
Ewan Higgs commented on HADOOP-15576: ------------------------------------- I wonder if I've misconfigured the test because they actually run 3 times for me: {code:java} $ mvn install -DskipShade -Dmaven.javadoc.skip=true -Pdist,parallel-tests -DtestsThreadCount=8 -Djava.awt.headless=true -DS3Guard -DAuth -Dit.test=ITestS3AContractMultipartUploader -pl :hadoop-aws -Dtest=wibble {code} {code:java} [INFO] --- maven-failsafe-plugin:2.21.0:integration-test (default-integration-test) @ hadoop-aws --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.541 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader [INFO] [INFO] Results: [INFO] [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] --- maven-failsafe-plugin:2.21.0:integration-test (sequential-integration-tests) @ hadoop-aws --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.987 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader [INFO] [INFO] Results: [INFO] [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] --- maven-failsafe-plugin:2.21.0:integration-test (default) @ hadoop-aws --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.966 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader [INFO] [INFO] Results: [INFO] [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 [INFO] {code} {quote}tests take a v long time.{quote} Out of interest, how long do they take for you? {quote} I'm going to do a revision of the patch, so I'll see if I can identify the problem. I'll do what I can about scaling the patch down to be viable, and make it a -Dscale test only. {quote} Thanks > S3A Multipart Uploader to work with S3Guard and encryption > ----------------------------------------------------------- > > Key: HADOOP-15576 > URL: https://issues.apache.org/jira/browse/HADOOP-15576 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.2 > Reporter: Steve Loughran > Assignee: Ewan Higgs > Priority: Blocker > Attachments: HADOOP-15576.001.patch, HADOOP-15576.002.patch, > HADOOP-15576.003.patch, HADOOP-15576.004.patch > > > The new Multipart Uploader API of HDFS-13186 needs to work with S3Guard, with > the tests to demonstrate this > # move from low-level calls of S3A client to calls of WriteOperationHelper; > adding any new methods are needed there. > # Tests. the tests of HDFS-13713. > # test execution, with -DS3Guard, -DAuth > There isn't an S3A version of {{AbstractSystemMultipartUploaderTest}}, and > even if there was, it might not show that S3Guard was bypassed, because > there's no checks that listFiles/listStatus shows the newly committed files. > Similarly, because MPU requests are initiated in S3AMultipartUploader, > encryption settings are't picked up. Files being uploaded this way *are not > being encrypted* -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org