[
https://issues.apache.org/jira/browse/KAFKA-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511504#comment-17511504
]
Chris Egerton commented on KAFKA-13764:
---------------------------------------
Discussed earlier during review of [https://github.com/apache/kafka/pull/10367]
> Potential improvements for Connect incremental rebalancing logic
> ----------------------------------------------------------------
>
> Key: KAFKA-13764
> URL: https://issues.apache.org/jira/browse/KAFKA-13764
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Chris Egerton
> Assignee: Chris Egerton
> Priority: Minor
>
> There are a few small changes that we might make to the incremental
> rebalancing logic for Kafka Connect to improve distribution of connectors and
> tasks across a cluster and address potential bugs:
> # During assignment, assign new connectors and tasks across the cluster
> before calculating revocations that may be necessary in order to balance the
> cluster. This way, we can potentially skip a round of revocation by using
> newly-created connectors and tasks to balance out the cluster.
> # Perform connector and task revocation in more cases, such as when one or
> more connectors are reconfigured to use fewer tasks, which can possibly lead
> to an imbalanced cluster.
> # Fix [this
> line|https://github.com/apache/kafka/blob/06ca4850c5b2b12e972f48e03fe4f9c1032f9a3e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java#L248]
> to use the same aggregation logic that's used
> [here|https://github.com/apache/kafka/blob/06ca4850c5b2b12e972f48e03fe4f9c1032f9a3e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java#L273-L281]
> in order to avoid overwriting map values when they should be combined
> instead.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
