[
https://issues.apache.org/jira/browse/KAFKA-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989070#comment-16989070
]
Sophie Blee-Goldman commented on KAFKA-9173:
--------------------------------------------
That's a good point, it does need to be documented. We should consider making
this more flexible, either by making this user-customizable (for example by
adding a `separateNodeGroups` flag to the StreamBuilder.stream(Pattern)
overload or maybe by autoscaling according to some heuristic.
As far as workarounds for now, I wouldn't necessarily recommend implementing a
custom PartitionGrouper since that's been deprecated in 2.4. You could always
use normal topic subscription to source each individually and then do something
like for (KStream stream : inputTopics) { _processingTopology(stream)_ }
> StreamsPartitionAssignor assigns partitions to only one worker
> --------------------------------------------------------------
>
> Key: KAFKA-9173
> URL: https://issues.apache.org/jira/browse/KAFKA-9173
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.3.0, 2.2.1
> Reporter: Oleg Muravskiy
> Priority: Major
> Labels: user-experience
> Attachments: StreamsPartitionAssignor.log
>
>
> I'm running a distributed KafkaStreams application on 10 worker nodes,
> subscribed to 21 topics with 10 partitions in each. I'm only using a
> Processor interface, and a persistent state store.
> However, only one worker gets assigned partitions, all other workers get
> nothing. Restarting the application, or cleaning local state stores does not
> help. StreamsPartitionAssignor migrates to other nodes, and eventually picks
> up other node to assign partitions to, but still only one node.
> It's difficult to figure out where to look for the signs of problems, I'm
> attaching the log messages from the StreamsPartitionAssignor. Let me know
> what else I could provide to help resolve this.
> [^StreamsPartitionAssignor.log]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
