[ https://issues.apache.org/jira/browse/KAFKA-14000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xinyu Zou updated KAFKA-14000: ------------------------------ Attachment: kafka-connect-trace.log > Kafka-connect standby server shows empty tasks list > --------------------------------------------------- > > Key: KAFKA-14000 > URL: https://issues.apache.org/jira/browse/KAFKA-14000 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect > Affects Versions: 2.6.0 > Reporter: Xinyu Zou > Priority: Major > Attachments: kafka-connect-trace.log > > > I'm using Kafka-connect distributed mode. There're two servers. One active > and one standby. The standby server sometimes shows empty tasks list in > status rest API response. > curl host:8443/connectors/name1/status > {code:java} > { > "connector": { > "state": "RUNNING", > "worker_id": "1.2.3.4:10443" > }, > "name": "name1", > "tasks": [], > "type": "source" > } {code} > I enabled TRACE log and checked. As required, the connect-status topic is set > to cleanup.policy=compact. But messages in the topic won't be compacted > timely. They will be compacted in a specific interval. So usually there're > more than one messages with same key. E.g. When kafka-connect is launched > there's no connector running. And then we start a new connector. Then there > will be two messages in connect-status topic: > status-task-name1 : state=RUNNING, workerId='10.251.170.166:10443', > generation=100 > status-task-name1 : _<emtpy>_ > > When reading status from connect-status topic, it doesn't sort messages by > generation. > [https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java] > So I think this could be improved. We can either sort the messages after poll > or compare generation value before we choose correct status message. -- This message was sent by Atlassian Jira (v8.20.7#820007)