Hi everyone, Following on from Ryan's thread in April [1], where it was agreed to keep the pre-built Kafka Connect connector out of releases until its LICENSE/NOTICE issues were fixed, and Kevin's suggestion in May [2] to start from runtime-deps.txt, I've opened a PR that I believe fixes them: https://github.com/apache/iceberg/pull/18019
It finishes the work Ryan started in #16147 and: - regenerates runtime-deps.txt, guarded by checkRuntimeDeps, the same way as the cloud bundles and Spark/Flink runtimes - has the runtime declare the same cloud SDK dependencies as the aws/gcp/azure bundles, so its LICENSE and NOTICE can be kept in sync with the bundles' files - reconciles LICENSE and NOTICE against every jar in the built zip, including shaded code Two things to flag for awareness/comment: 1. It removes the Hive distribution. It could come back later with its own runtime-deps.txt if people still want it. 2. The distribution grows a little (third-party jars 243 → 257, zip 184 → 192 MB), because the bundles also declare the SDK modules behind Iceberg's optional cloud features (S3 Access Grants, the S3 analytics accelerator, CloudWatch metrics publishing, Google Cloud KMS, GCS analytics, Azure Key Vault), which the previous hand-picked list left out. The Hadoop tree is unchanged. There are full details in the PR. Reviews welcome, particularly on the licensing judgement calls listed under "Notes for reviewers" in the PR; I'm not a licensing expert and would rather those were checked than taken on trust. Full disclosure: Claude helped me a lot with this :) I drove and checked the work. `check` passes, including the Kafka Connect integration tests against the installed distribution, and every jar in the zip was opened and compared against LICENSE. This isn't aimed at 1.12. Once it lands, I'd like to pick #15212 back up so the connector can ship as part of a future release. thanks, Robin. [1] https://lists.apache.org/thread/3p7nrxfv3p1hclfww4hjdtbmfwklgok7 [2] https://lists.apache.org/thread/rvp2wh1m91h5lvz5zvxqp96f7pdxrzqr
