Gargi-jais11 opened a new pull request, #10786: URL: https://github.com/apache/ozone/pull/10786
## What changes were proposed in this pull request? Ozone does not persist standard S3 object headers on PUT or return them on HEAD/GET. s3-tests fail for: - Cache-Control (no-cache from NoCacheFilter instead of stored value) - test_object_write_cache_control, - Expires (off by 6000s because filter sets Expires=now) - test_object_write_expires, and - Content-Encoding (KeyError on HEAD) -test_object_content_encoding_aws_chunked. Implement persist-on-PUT and return-on-HEAD/GET for these headers, strip aws-chunked from Content-Encoding per AWS semantics, and ensure object headers override HDDS NoCacheFilter defaults on S3 object responses. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-15606 ## How was this patch tested? Added IT and UT. **Before Fix:** ``` bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object \ --bucket testbucket \ --key foo \ --body /tmp/foo-body \ --cache-control "public, max-age=14400" { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } bash-5.1$ aws --endpoint-url $ENDPOINT s3api head-object \ --bucket testbucket \ --key foo \ --query 'CacheControl' "no-cache" <---------------- wrong output bash-5.1$ EXPIRES=$(date -u -v+6000S '+%a, %d %b %Y %H:%M:%S GMT' 2>/dev/null \ || date -u -d '+6000 seconds' '+%a, %d %b %Y %H:%M:%S GMT') aws --endpoint-url $ENDPOINT s3api put-object \ --bucket testbucket \ --key foo-expires \ --body /tmp/foo-body \ --expires "$EXPIRES" { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } bash-5.1$ aws --endpoint-url $ENDPOINT s3api head-object \ --bucket testbucket \ --key foo-expires \ --query 'Expires' "Thu, 16 Jul 2026 11:58:46 GMT" <---------------- wrong output bash-5.1$ KEY=encoding aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding gzip aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } null <---------------- wrong output bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding "deflate, gzip" aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } null <---------------- wrong output bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding "gzip, aws-chunked" aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } null <---------------- wrong output bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding "aws-chunked" aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } null ``` **After Fix:** ``` bash-5.1$ ozone sh bucket create /s3v/testbucket bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object \ --bucket testbucket \ --key foo \ --body /tmp/foo-body \ --cache-control "public, max-age=14400" { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } bash-5.1$ aws --endpoint-url $ENDPOINT s3api head-object \ --bucket testbucket \ --key foo \ --query 'CacheControl' "public, max-age=14400" <-------------Fixed ------------ ------------ bash-5.1$ aws --endpoint-url http://s3g:9878/ s3api put-object --bucket testbucket --key foo --body /tmp/foo-body --expires "Wed, 21 Oct 2026 07:50:00 GMT" { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } bash-5.1$ aws --endpoint-url http://s3g:9878/ s3api head-object \ --bucket testbucket --key foo --query Expires "Wed, 21 Oct 2026 07:50:00 GMT" <-------------Fixed ---------- ---------- bash-5.1$ KEY=encoding bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding gzip { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } bash-5.1$ aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' "gzip" <-------------Fixed bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding "deflate, gzip" aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } "deflate, gzip" <-------------Fixed bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding "gzip, aws-chunked" aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } "gzip" <-------------Fixed bash-5.1$ aws --endpoint-url $ENDPOINT s3api put-object --bucket testbucket --key $KEY --body /tmp/foo-body --content-encoding "aws-chunked" aws --endpoint-url $ENDPOINT s3api head-object --bucket testbucket --key $KEY --query 'ContentEncoding' { "ETag": "\"c157a79031e1c40f85931829bc5fc552\"" } null <-------------Fixed ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
