Hi, I am using Kafka Connect Sink Connector to consume messages from a Kafka topic in a secure Kafka cluster. I have provided the group.id in connect-distributed.properties. I am using security.protocol as SASL_PLAINTEXT.
Here is definition of group id in connect-distributed.properties - bootstrap.servers=<server>:6667 *group.id <http://group.id>*=alert key.converter=org.apache.kafka.connect.storage.StringConverter value.converter=org.apache.kafka.connect.json.JsonConverter key.converter.schemas.enable=true value.converter.schemas.enable=true internal.key.converter=org.apache.kafka.connect.json.JsonConverter internal.value.converter=org.apache.kafka.connect.json.JsonConverter internal.key.converter.schemas.enable=false internal.value.converter.schemas.enable=false offset.storage.topic=connect-offsets offset.flush.interval.ms=10000 config.storage.topic=connect-configs security.protocol=SASL_PLAINTEXT sasl.mechanism=GSSAPI sasl.kerberos.service.name=kafka producer.sasl.kerberos.service.name=kafka producer.security.protocol=SASL_PLAINTEXT producer.sasl.mechanism=GSSAPI consumer.sasl.kerberos.service.name=kafka consumer.security.protocol=SASL_PLAINTEXT consumer.sasl.mechanism=GSSAPI In the Kafka Connect log, its picking up the group.id in the properties file - 2016-11-28 15:49:40 INFO DistributedConfig : 165 - DistributedConfig values: request.timeout.ms = 40000 retry.backoff.ms = 100 ...... group.id = alert metric.reporters = [] ssl.truststore.type = JKS cluster = connect But, when its instantiating a sink connector task, its using a new consumer group - 2016-11-28 15:49:43 INFO ConsumerConfig : 165 - ConsumerConfig values: request.timeout.ms = 40000 check.crcs = true retry.backoff.ms = 100 ...... group.id = connect-sink-connector enable.auto.commit = false metric.reporters = [] ssl.truststore.type = JKS send.buffer.bytes = 131072 Why is this happening? I have not faced this issue on an unsecure cluster. Is there any configuration property I am missing? Thanks in advance for your help. -Sri On Mon, Nov 28, 2016 at 6:52 PM, Srikrishna Alla <[email protected]> wrote: > Hi, > > I am using Kafka Connect Sink Connector to consume messages from a Kafka > topic in a secure Kafka cluster. I have provided the group.id in > connect-distributed.properties. I am using security.protocol as > SASL_PLAINTEXT. > > Here is definition of group id in connect-distributed.properties - > [sa9726@clpd355 conf]$ less connect-distributed.properties |grep -i group > group.id=alert > > In the Kafka Connect log, its picking up the group.id in the properties > file - > 2016-11-28 15:49:40 INFO DistributedConfig : 165 - DistributedConfig > values: > request.timeout.ms = 40000 > retry.backoff.ms = 100 > > > >
