davsclaus opened a new pull request, #30296:
URL: https://github.com/apache/camel/pull/30296

   Follow-up to #30295 
([CAMEL-27130](https://issues.apache.org/jira/browse/CAMEL-27130)): Camel JBang 
downloads the serde jars of the chosen `schemaRegistryVendor`, so no `--dep` is 
needed. For Confluent, there is now a `--repos=confluent` shorthand.
   
   > **Stacked on #30295.** This branch is built on the CAMEL-27130 branch, so 
until #30295 is merged its diff includes that PR. Review only the last two 
commits.
   
   ### 1. Camel JBang downloads the serdes
   
   **Mappings.** All 16 serializer and deserializer classes of the four vendors 
are mapped by class name in 
`dsl/camel-kamelet-main/src/main/known-third-party-libraries.properties`. That 
is the existing source that `prepare-kamelet-main` turns into 
`camel-thirdparty-known-dependencies.properties` with the versions from the 
parent pom.
   
   **How the class reaches Kafka.** No change was needed for this part:
   
   - camel-kafka already resolves `value.serializer` and `value.deserializer` 
through the Camel `ClassResolver` (`KafkaEndpoint.updateClassProperties`), and 
passes the resolved `Class` to Kafka.
   - Under JBang, that resolver is `DependencyDownloaderClassResolver`, which 
downloads the mapped artifact.
   
   **Several artifacts per mapping.** A mapping can now list several artifacts, 
separated by commas, which are downloaded together as one dependency graph.
   
   - Azure needs this. `azure-schemaregistry-kafka-avro` pulls in `azure-core` 
1.35.0, while its `azure-identity` needs 1.54.1 or later. The first send then 
failed with `NoSuchMethodError: TokenRequestContext.isCaeEnabled()`.
   - The Azure classes now map to the serde plus `com.azure:azure-core` from 
the Azure SDK BOM.
   - `KnownDependenciesResolver` gets `mavenGavsForClass`, `mavenGavsForImport` 
and `download(...)`. The class resolver, the Java imports, the properties 
component and the property-binding listener now download every artifact of a 
mapping.
   
   **A failed download is remembered.** The class resolver now remembers a 
known dependency whose download failed, so that it is not attempted again for 
each class loader. Without the Confluent repository, this gives one error 
instead of five stack traces.
   
   **camel-kafka checks the serde.**
   
   - When `schemaRegistryVendor` is set and its serde class cannot be loaded, 
the producer, consumer and share consumer fail at start, naming the Maven 
artifact to add.
   - For Confluent, the message also gives the repository URL and `camel run 
--repos=confluent`.
   - With a custom `KafkaClientFactory`, which may supply the class another 
way, it only logs a warning.
   - An explicit `valueSerializer` or `valueDeserializer` is never checked.
   
   ### 2. Versions in `parent/pom.xml`
   
   | Property | Version | Why |
   |---|---|---|
   | `confluent-serializer-version` | 8.3.3 | Confluent Platform 8.3.x goes 
with Kafka 4.3.x, which Camel uses (`kafka-version` 4.3.2). 8.4.1 is the newest 
|
   | `aws-glue-schema-registry-version` | 2.0.2 | Newest on Maven Central |
   | `azure-schemaregistry-kafka-avro-version` | 1.1.4 | Newest on Maven 
Central |
   | `apicurio-registry-sdk-version` | 3.3.3 (reused) | The serdes come from 
the same Apicurio Registry release as the SDK |
   
   `azure-core` comes from `azure-sdk-bom-version`.
   
   `PrepareKameletMainMojo` now:
   
   - accepts several artifacts per line;
   - verifies the class keys, not only packages;
   - adds extra repositories while verifying, with 
`camel.known-dependencies.verify.repos`, which defaults to the Confluent 
repository.
   
   `-Dcamel.known-dependencies.verify=true` checked 167 entries against 154 
jars.
   
   ### 3. The `confluent` repository alias
   
   - `known-maven-repos.properties` gets `confluent = 
https://packages.confluent.io/maven/`, next to `redhat`.
   - The aliases were not documented, so `camel-jbang-running.adoc` now lists 
all six.
   - The Confluent repository is not added automatically: the user asks for it 
with `--repos=confluent`.
   - The camel-kafka "Schema registry" docs now say that Camel JBang downloads 
the jars.
   
   ### End to end
   
   Run with the Camel JBang launcher built from this branch, against Kafka 4.4 
in Docker, without `--dep`:
   
   | Vendor | Result |
   |---|---|
   | Confluent, no `--repos` | Fails at start, naming 
`io.confluent:kafka-avro-serializer`, the repository and `--repos=confluent` |
   | Confluent, `--repos=confluent` | Serde 8.3.3 downloaded. An Avro 
`GenericRecord` was sent and received, and the subject registered |
   | Apicurio Registry 3.3.4 | Serde 3.3.3 downloaded. Sent and received, and 
the artifact registered |
   | AWS Glue | Serde 2.0.2 downloaded and configured. Fails only on the fake 
credentials |
   | Azure | Serde 1.1.4 and `azure-core` 1.57.0 downloaded. Fails only on the 
fake tenant, with no `NoSuchMethodError`. With `--dep` on the serde only, the 
`NoSuchMethodError` comes back, which confirms that the new path is what fixes 
it |
   
   ### Tests
   
   New tests:
   
   - `DefaultRepositoryResolverTest`
   - `KnownDependenciesResolverTest`: the 16 classes against the parent pom 
versions, plus several artifacts per mapping
   - `MavenDependencyDownloaderBatchTest`
   - `KafkaShareSchemaRegistrySerdeTest`
   - more cases in `KafkaSchemaRegistryVendorTest`
   
   `mvn test` passed for camel-package-maven-plugin, camel-tooling-maven, 
camel-kafka-common, camel-kafka (1405 tests), camel-kafka-share and 
camel-kamelet-main (1033 tests).
   
   ### Notes
   
   - The new version properties are only read by the generator, so Dependabot 
will not bump them.
   - If the Azure serde is added with `--dep` instead, JBang does not bring 
`azure-core`. The docs keep the note about it.
   - A dependency whose download failed is not retried during the same run.
   - JSON Schema and Protobuf were not run end to end; their class names are 
verified in the jars.
   
   _Claude Code on behalf of Claus Ibsen_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to