shashank created CAMEL-25898:
--------------------------------
Summary: camel-ibm-cos - the producer stores a consumed object
back in the bucket it was consumed from, so a route from one bucket to another
loses the object
Key: CAMEL-25898
URL: https://issues.apache.org/jira/browse/CAMEL-25898
Project: Camel
Issue Type: Bug
Reporter: shashank
Assignee: shashank
{{IBMCOSProducer.determineBucketName}} (main 74ee085f7d64, line 357) takes the
bucket of every producer operation from the {{CamelIBMCOSBucketName}} header,
and only falls back to the bucket of the endpoint when the header is not set:
{code:java}
private String determineBucketName(Exchange exchange) {
String bucketName = exchange.getIn().getHeader(IBMCOSConstants.BUCKET_NAME,
String.class);
if (bucketName == null) {
bucketName = getConfiguration().getBucketName();
}
return bucketName;
}
{code}
{{IBMCOSConsumer}} (line 223) sets the same header on every object it consumes,
to the bucket the object was consumed from. So a route that moves objects from
one bucket to another:
{code:java}
from("ibm-cos://source?cosClient=#cos")
.to("ibm-cos://target?cosClient=#cos");
{code}
puts every object back into {{source}}, under the same key (the producer also
takes the key from the {{CamelIBMCOSKey}} header the consumer set), and nothing
reaches {{target}}. With the default {{deleteAfterRead=true}} the consumer then
deletes the object from {{source}} when the exchange completes: the object is
in neither bucket, and the exchange completed without an error. With
{{deleteAfterRead=false}} each object is rewritten in place and the target
bucket stays empty. The same applies to every other producer operation that
follows such a consumer ({{copyObject}} source bucket, {{getObject}},
{{deleteObject}}, ...).
camel-aws2-s3, camel-minio and camel-google-storage had the same problem and
fixed it in 4.10 (CAMEL-21680, CAMEL-21678, CAMEL-21682): the producer reads a
separate {{...OverrideBucketName}} header, and the bucket header the consumer
sets is no longer used by the producer. camel-ibm-cos was added after that
(CAMEL-22598) with the earlier pattern. CAMEL-25888 (open) proposes the same
split for camel-pubnub ({{CamelPubNubOverrideChannel}}).
*Fix*: the same as in camel-aws2-s3: a new producer header
{{CamelIBMCOSOverrideBucketName}} ({{IBMCOSConstants.OVERRIDE_BUCKET_NAME}})
selects another bucket than the one of the endpoint; {{CamelIBMCOSBucketName}}
is documented as a consumer header and no longer read by the producer. The
{{createBucket}} example of the component page, the integration test that
creates and deletes a bucket, and the generated metadata (component and catalog
JSON, endpoint DSL header names) use the new header. Upgrade note in the 4.24
guide, marked as a breaking change: a route that sets {{CamelIBMCOSBucketName}}
to choose the bucket of a producer operation must set
{{CamelIBMCOSOverrideBucketName}} instead (code that uses the Java constant has
to switch to the new constant).
*Tests*: new {{IBMCOSProducerBucketHeaderTest}} (2 tests).
{{consumedObjectIsStoredInTheBucketOfTheProducerEndpoint}}: the real consumer
reads {{a.txt}} from bucket {{source}} of the module's in-process fake S3
server, the producer endpoint {{ibm-cos://target}} uses a recording client; on
main it fails with {{expected: <[target/a.txt]> but was: <[source/a.txt]>}}.
{{overrideBucketNameHeaderSelectsTheBucket}}: without a header the endpoint
bucket is used, with the override header the named one; on main {{expected:
<[target/b.txt, other/c.txt]> but was: <[target/b.txt, target/c.txt]>}}.
camel-ibm-cos unit tests: 8 run, 0 failures.
*How it was found*: a sweep of the producers for headers that their own
component's consumer sets and the producer reads as a destination (bucket,
topic, queue, channel), after the CAMEL-21680 family. The JUnit tests above are
the evidence; nothing was run against IBM Cloud Object Storage.
*Affected*: main, camel-4.23.x, camel-4.22.x and camel-4.18.x (the same
{{determineBucketName}} and the same header set by the consumer; read, not run).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
