[I] Kafka Connect config providers are not working when CVE-2024-31141 fix is applied [iceberg]

[via GitHub]

AnatolyPopov opened a new issue, #12221:
URL: https://github.com/apache/iceberg/issues/12221

   ### Apache Iceberg version
   
   None
   
   ### Query engine
   
   Kafka Connect
   
   ### Please describe the bug 🐞
   
   When a Kafka Connect worker is configured with secret providers and the fix 
for CVE-2024-31141 is applied—setting the JVM system property 
org.apache.kafka.automatic.config.providers to "none"—the connector crashes 
with the following exception:
   
   ```
   org.apache.kafka.common.config.ConfigException: 
io.lenses.connect.secrets.providers.AWSSecretProvider is not allowed. Update 
System property 'org.apache.kafka.automatic.config.providers' to allow 
io.lenses.connect.secrets.providers.AWSSecretProvider
        at 
org.apache.kafka.common.config.AbstractConfig.instantiateConfigProviders(AbstractConfig.java:609)
        at 
org.apache.kafka.common.config.AbstractConfig.resolveConfigVariables(AbstractConfig.java:541)
        at 
org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:113)
        at 
org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:147)
        at 
org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:292)
        at org.apache.kafka.clients.admin.Admin.create(Admin.java:147)
        at 
org.apache.iceberg.connect.channel.KafkaClientFactory.createAdmin(KafkaClientFactory.java:66)
        at 
org.apache.iceberg.connect.channel.CommitterImpl.start(CommitterImpl.java:64)
        at 
org.apache.iceberg.connect.IcebergSinkTask.open(IcebergSinkTask.java:58)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask.openPartitions(WorkerSinkTask.java:660)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask.access$1300(WorkerSinkTask.java:77)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask$HandleRebalance.onPartitionsAssigned(WorkerSinkTask.java:751)
        at 
org.apache.kafka.clients.consumer.internals.ConsumerRebalanceListenerInvoker.invokePartitionsAssigned(ConsumerRebalanceListenerInvoker.java:64)
        at 
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:424)
        at 
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:503)
        at 
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:414)
        at 
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:510)
        at 
org.apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.updateAssignmentMetadataIfNeeded(LegacyKafkaConsumer.java:652)
        at 
org.apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.poll(LegacyKafkaConsumer.java:611)
        at 
org.apache.kafka.clients.consumer.internals.LegacyKafkaConsumer.poll(LegacyKafkaConsumer.java:591)
        at 
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:874)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask.pollConsumer(WorkerSinkTask.java:497)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:339)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:246)
        at 
org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:215)
        at 
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:225)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:280)
        at 
org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:237)
        at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
   ```
   This issue occurs because the connector task reads the worker.properties 
file and applies those configurations to all control topic clients. However, 
the CVE fix prevents any config providers from being used in the client 
configuration unless they are explicitly included in the 
org.apache.kafka.automatic.config.providers system property.
   
   See the relevant implementation in Kafka: 
[AbstractConfig.java#L554](https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java#L554).
   
   
   ### Willingness to contribute
   
   - [x] I can contribute a fix for this bug independently
   - [ ] I would be willing to contribute a fix for this bug with guidance from 
the Iceberg community
   - [ ] I cannot contribute a fix for this bug at this time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org