oscerd commented on issue #1849:
URL:
https://github.com/apache/camel-kafka-connector/issues/1849#issuecomment-6442061643
Thanks @aonamrata, the full trace pinpointed it, and I can reproduce it with
`cp-kafka-connect:8.2.4` and your config. It isn't Java-related.
The image's `connect-distributed` puts `/usr/share/java/kafka-serde-tools/*`
on the worker classpath. That folder includes AWS SDK 2.29.39's
`apache-client`, which registers
`software.amazon.awssdk.http.apache.ApacheSdkHttpService`. Since Camel 4.22.0
(CAMEL-23866), the connector uses the SDK's Apache5 client instead, so it no
longer contains that class. Kafka Connect still lets the plugin see the
worker's registration, and the class then loads from the worker's own copy of
the SDK. The result is `not a subtype`. 4.18.x hid this because it shipped the
classic client itself.
**Workaround on 4.22.0** (verified in that image). Add the classic client,
at the same SDK version, to the connector folder. The SDK still picks Apache5;
this jar only overrides the conflicting registration.
```dockerfile
RUN cd /usr/share/camel/camel-aws-s3-source-kafka-connector && \
for a in
software/amazon/awssdk/apache-client/2.50.2/apache-client-2.50.2.jar \
org/apache/httpcomponents/httpclient/4.5.14/httpclient-4.5.14.jar \
org/apache/httpcomponents/httpcore/4.4.16/httpcore-4.4.16.jar \
commons-logging/commons-logging/1.2/commons-logging-1.2.jar; do
\
curl -sSfLO "https://repo1.maven.org/maven2/$a";; done
```
Two alternatives, each with a caveat:
- Setting
`KAFKA_OPTS=-Dsoftware.amazon.awssdk.http.service.impl=software.amazon.awssdk.http.apache5.Apache5SdkHttpService`
applies to the whole JVM. It can break other AWS SDK v2 users on the worker.
- Deleting `/usr/share/java/kafka-serde-tools/apache-client-2.29.39.jar` is
only safe if you don't use CSFLE with AWS KMS.
I could only verify the workaround as far as the first S3 call, so please
let us know whether it works with your credentials.
The proper fix belongs in Camel, and affects all 4.22.0 AWS connectors on
Confluent images:
- [CAMEL-26343](https://issues.apache.org/jira/browse/CAMEL-26343),
apache/camel#28563: the AWS components set their HTTP client themselves instead
of relying on the SDK's classpath lookup. The backport to Camel 4.22.x is
apache/camel#28564.
- #1851 documents the workarounds in the troubleshooting guide.
This issue stays open until CKC moves to a Camel 4.22.x release that
contains the fix.
---
_Claude Code on behalf of Andrea Cosentino_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
