ysymi opened a new issue, #652: URL: https://github.com/apache/flink-agents/issues/652
### Search before asking - [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar. ### Description ## Motivation The current `flink-agents-dist-*.jar` bundles a large number of third-party dependencies without relocating them: - `jackson-databind` 2.18.2 - `kafka-clients` 4.0.0 - `kotlin-stdlib` 1.x (transitive from `openai-java`) - `anthropic-java`, `openai-java`, etc. When users drop this fat jar into the Flink cluster's `lib/` directory, or include it via `--jars` at job submission time, these classes land in the same classloader as the user's job code. If the user's job also depends on e.g. `kafka-clients` (a very common scenario in Flink streaming jobs) or `jackson-databind` with a different version, runtime errors like `NoSuchMethodError` or `ClassCastException` are likely. Note that Flink itself already relocates its own bundled dependencies (e.g. `flink-shaded-jackson`, `flink-shaded-guava`) precisely to avoid this problem. flink-agents-dist should follow the same approach. ## Analysis The top offenders by size and conflict risk: | Dependency | Size | Risk | Notes | |---|---|---|---| | `kafka-clients` 4.0.0 | ~22 MB | **High** | Many Flink jobs use Kafka; major version gap | | `openai-java` + `kotlin-stdlib` | ~80 MB | Medium | Uncommon in user code but large | | `jackson-databind` 2.18.2 | ~6 MB | Medium | Flink uses shaded jackson, user code may not | | `anthropic-java` | ~17 MB | Low | Rarely used alongside | ## Proposed Solution Relocate in shade plugin Add `<relocation>` rules in `dist/pom.xml` to move third-party packages under an `org.apache.flink.agents.shaded.*` namespace: ```xml <relocations> <relocation> <pattern>com.fasterxml.jackson</pattern> <shadedPattern>org.apache.flink.agents.shaded.jackson</shadedPattern> </relocation> <relocation> <pattern>org.apache.kafka</pattern> <shadedPattern>org.apache.flink.agents.shaded.kafka</shadedPattern> </relocation> <!-- etc. --> </relocations> ``` This requires verifying that internal usages of these libraries (especially Jackson annotations on public API classes, Kafka SPI configurations) still work correctly after relocation. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
