[jira] [Created] (KAFKA-14980) MirrorMaker consumers don't get configs prefixed with source.cluster

[Mickael Maison (Jira)]

Mickael Maison created KAFKA-14980:
--------------------------------------

             Summary: MirrorMaker consumers don't get configs prefixed with 
source.cluster
                 Key: KAFKA-14980
                 URL: https://issues.apache.org/jira/browse/KAFKA-14980
             Project: Kafka
          Issue Type: Bug
          Components: mirrormaker
    Affects Versions: 3.5.0
            Reporter: Mickael Maison


As part of KAFKA-14021, we made a change to 
MirrorConnectorConfig.sourceConsumerConfig() to grab all configs that start 
with "source.". Previously it was grabbing configs prefixed with 
"source.cluster.". 

This means existing connector configuration stop working, as configurations 
such as bootstrap.servers are not passed to source consumers.

For example, the following connector configuration was valid in 3.4 and now 
makes the connector tasks fail:

{code:json}
{
    "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
    "name": "source",
    "topics": "test",
    "tasks.max": "30",
    "source.cluster.alias": "one",
    "target.cluster.alias": "two",
    "source.cluster.bootstrap.servers": "localhost:9092",
   "target.cluster.bootstrap.servers": "localhost:29092"
}
{code}


The connector attempts to start source consumers with bootstrap.servers = [] 
and the task crash with 


{noformat}
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at 
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:837)
        at 
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:671)
        at 
org.apache.kafka.connect.mirror.MirrorUtils.newConsumer(MirrorUtils.java:59)
        at 
org.apache.kafka.connect.mirror.MirrorSourceTask.start(MirrorSourceTask.java:103)
        at 
org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.initializeAndStart(AbstractWorkerSourceTask.java:274)
        at 
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)
        at 
org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:75)
        at 
org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:181)
        at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable 
bootstrap urls given in bootstrap.servers
{noformat}






--
This message was sent by Atlassian Jira
(v8.20.10#820010)