[
https://issues.apache.org/jira/browse/CAMEL-25903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25903 started by shashank.
----------------------------------------
> camel-aws2-s3 - with multiPartUpload=true, an object uploaded in parts is
> stored without the user metadata of the CamelAwsS3Metadata header
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25903
> URL: https://issues.apache.org/jira/browse/CAMEL-25903
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-s3
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> With {{multiPartUpload=true}}, {{AWS2S3Producer.processMultiPart}} (main
> 74ee085f7d64, lines 172-347) uploads a body that is at least {{partSize}}
> long (default 25 MB) with {{CreateMultipartUpload}} / {{UploadPart}} /
> {{CompleteMultipartUpload}}, and a smaller body with a single {{PutObject}}
> ({{doPutObject}}). Both start from the same user metadata, computed from the
> {{CamelAwsS3Metadata}} header:
> {code:java}
> Map<String, String> objectMetadata = determineMetadata(exchange);
> ...
> objectMetadata.put("Content-Length", "" + contentLength);
> ...
> CreateMultipartUploadRequest.Builder createMultipartUploadRequest
> =
> CreateMultipartUploadRequest.builder().bucket(bucketName).key(keyName);
> {code}
> {{doPutObject}} sets it on the request
> ({{putObjectRequest...metadata(objectMetadata)}}), but the multipart path
> never puts the map on the {{CreateMultipartUploadRequest}}: an object
> uploaded in parts is stored without any user metadata. The upload succeeds
> and nothing is logged. Storage class, ACL, content type, cache control,
> content disposition, content encoding and the encryption options are all
> applied on that path; only the metadata is left out.
> So with {{multiPartUpload=true}} whether an object gets its metadata depends
> on its size. The S3 consumer sets {{CamelAwsS3Metadata}} to the user metadata
> of the object it read, so a route that copies objects from one bucket to
> another with multipart upload keeps the metadata of the small objects and
> silently drops it for the large ones. CAMEL-17713 added user metadata to the
> producer for integrations that rely on it; the multipart path has never had
> it (the same lines are in 3.22.x).
> *Fix*: {{processMultiPart}} sets the metadata on the
> {{CreateMultipartUploadRequest}}. The {{Content-Length}} entry that the
> multipart path added to the map (never sent anywhere) is removed, so it does
> not become an {{x-amz-meta-content-length}} entry. No option or documentation
> change; no upgrade note (objects now get the metadata the route asked for).
> *Tests*: new {{AWS2S3ProducerMultipartMetadataTest}} (2 tests, Mockito
> {{S3Client}} as the other producer tests of the module, {{partSize=5}}).
> {{multipartUploadStoresTheUserMetadata}} (a 12 byte body, 3 parts) fails on
> main with {{expected: {"checksum"="abc123", "origin"="camel"} but was: {}}}.
> {{bodySmallerThanThePartSizeStoresTheUserMetadata}} is a control: the single
> put keeps the metadata on main and with the fix. camel-aws2-s3 unit tests:
> 118 run, 0 failures (run with the AWS SDK 2.55.15 jars of the local
> repository, {{-Daws-java-sdk2-version=2.55.15}}; main uses 2.56.0).
> *How it was found*: comparing, per producer, the request options applied on
> each code path of one operation (single vs multipart, sync vs async, batch vs
> single). The JUnit tests above are the evidence; nothing was run against AWS
> S3.
> *Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x
> (the same {{CreateMultipartUploadRequest}} without metadata; read, not run).
> Not changed, possible follow-ups: with {{useCustomerKey=true}} (SSE-C) the
> {{UploadPartRequest}} of the multipart path does not carry the customer key
> either, which S3 requires for each part, so such uploads fail (loudly).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
