Xinyu Zou created KAFKA-14000: --------------------------------- Summary: Kafka-connect standby server shows empty tasks list Key: KAFKA-14000 URL: https://issues.apache.org/jira/browse/KAFKA-14000 Project: Kafka Issue Type: Bug Components: KafkaConnect Affects Versions: 2.6.0 Reporter: Xinyu Zou
I'm using Kafka-connect distributed mode. There're two servers. One active and one standby. The standby server sometimes shows empty tasks list in status rest API response. curl host:8443/connectors/name1/status {code:java} { "connector": { "state": "RUNNING", "worker_id": "1.2.3.4:10443" }, "name": "name1", "tasks": [], "type": "source" } {code} I enabled TRACE log and checked. As required, the connect-status topic is set to cleanup.policy=compact. But messages in the topic won't be compacted timely. They will be compacted in a specific interval. So usually there're more than one messages with same key. E.g. When kafka-connect is launched there's no connector running. And then we start a new connector. Then there will be two messages in connect-status topic: status-task-name1 : state=RUNNING, workerId='10.251.170.166:10443', generation=100 status-task-name1 : _<emtpy>_ When reading status from connect-status topic, it doesn't sort messages by generation. [https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecords.java] So I think this could be improved. We can either sort the messages after poll or compare generation value before we choose correct status message. -- This message was sent by Atlassian Jira (v8.20.7#820007)