Chris Egerton created KAFKA-17105: ------------------------------------- Summary: Unnecessary connector restarts after being newly created Key: KAFKA-17105 URL: https://issues.apache.org/jira/browse/KAFKA-17105 Project: Kafka Issue Type: Bug Components: connect Affects Versions: 3.7.1, 3.6.2, 3.6.1, 3.7.0, 3.5.2, 3.5.1, 3.6.0, 3.4.1, 3.5.0, 3.3.2, 3.3.1, 3.2.3, 3.2.2, 3.4.0, 3.2.1, 3.1.2, 3.0.2, 3.3.0, 3.1.1, 3.2.0, 3.0.1, 3.0.0, 3.1.0, 3.8.0, 3.9.0 Reporter: Chris Egerton
When a connector is created, it may be restarted unnecessarily immediately after it is first started by the worker to which it has been assigned: # Connector config is written to the config topic # A worker reads the new record from the config topic, and adds the connector to its connectorConfigUpdates field (see [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L2445]) # The worker participates in the ensuing rebalance (see [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L419]) and is assigned the connector # After the rebalance is over (see [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L422]), the worker starts all of the connectors it has been newly-assigned (see [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L1843] and [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L1963-L2001]) # Once finished with that, the worker checks for new connector configs (see [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L535]) and restarts all connectors in the connectorConfigUpdates field (see [here|https://github.com/apache/kafka/blob/43676f7612b2155ecada54c61b129d996f58bae2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L717-L726]). -- This message was sent by Atlassian Jira (v8.20.10#820010)