The docs say: “Each task is assigned to a thread. Each task is capable of handling multiple Kafka partitions, but a single partition must be handled by only one task.” From what I understand additional tasks would sit idle.
From: Yeikel Santana <em...@yeikel.com> Date: Thursday, May 30, 2024 at 7:43 AM To: users@kafka.apache.org <users@kafka.apache.org> Subject: [EXTERNAL] Regarding Kafka connect task to partition relationship for both source and sink connectors Hi everyone, From my understanding, if a topic has n partitions, we can create up to n tasks for both the source and sink connectors to achieve the maximum parallelism. Adding more tasks would not be beneficial, as they would remain idle and be limited to the number of partitions of the topic Could you please confirm if this understanding is correct? If this understanding is incorrect could you please explain the relationship if any? Thank you! This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
