[
https://issues.apache.org/jira/browse/KAFKA-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986460#comment-16986460
]
Sophie Blee-Goldman commented on KAFKA-9173:
--------------------------------------------
I agree we should aim to fix this, but the workaround here is pretty simple –
you just need to decrease the number of threads to better match the actual
workload. If your app only has 10 tasks and you want to run 10 instances, each
instance only needs one thread. Even if we did "fix" the assignor so that it
spread the 10 tasks evenly across the 10 instances, 19 of the 20 threads would
have no tasks assigned and nothing to do.
You can change num.threads to 1 for each instance, and you will see each be
assigned one of the ten tasks
> StreamsPartitionAssignor assigns partitions to only one worker
> --------------------------------------------------------------
>
> Key: KAFKA-9173
> URL: https://issues.apache.org/jira/browse/KAFKA-9173
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.3.0, 2.2.1
> Reporter: Oleg Muravskiy
> Priority: Major
> Labels: user-experience
> Attachments: StreamsPartitionAssignor.log
>
>
> I'm running a distributed KafkaStreams application on 10 worker nodes,
> subscribed to 21 topics with 10 partitions in each. I'm only using a
> Processor interface, and a persistent state store.
> However, only one worker gets assigned partitions, all other workers get
> nothing. Restarting the application, or cleaning local state stores does not
> help. StreamsPartitionAssignor migrates to other nodes, and eventually picks
> up other node to assign partitions to, but still only one node.
> It's difficult to figure out where to look for the signs of problems, I'm
> attaching the log messages from the StreamsPartitionAssignor. Let me know
> what else I could provide to help resolve this.
> [^StreamsPartitionAssignor.log]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
