Steve Loughran created HADOOP-18339: ---------------------------------------
Summary: S3A storage class option only picked up when buffering writes to disk Key: HADOOP-18339 URL: https://issues.apache.org/jira/browse/HADOOP-18339 Project: Hadoop Common Issue Type: Improvement Components: fs/s3 Affects Versions: 3.3.9 Reporter: Steve Loughran when you switch s3a output stream buffering to heap or byte buffer, the storage class option isn't added to the put request {code} <property> <name>fs.s3a.fast.upload.buffer</name> <value>bytebuffer</value> </property> {code} and the ITestS3AStorageClass tests fail. {code} java.lang.AssertionError: [Storage class of object s3a://stevel-london/test/testCreateAndCopyObjectWithStorageClassGlacier/file1] Expecting: <null> to be equal to: <"glacier"> ignoring case considerations at org.apache.hadoop.fs.s3a.ITestS3AStorageClass.assertObjectHasStorageClass(ITestS3AStorageClass.java:215) at org.apache.hadoop.fs.s3a.ITestS3AStorageClass.testCreateAndCopyObjectWithStorageClassGlacier(ITestS3AStorageClass.java:129) {code} we noticed this in a code review; the request factory only sets the option when the source is a file, not memory. proposed: parameterize the test suite on disk/byte buffer, then fix -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org