[
https://issues.apache.org/jira/browse/KAFKA-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472576#comment-13472576
]
Joel Koshy commented on KAFKA-550:
----------------------------------
The bug was not in the implicit function. I removed the implicit because it
could be easily avoided in this case.
The issue was here: (tt <- topicThreadIds; qs <- queuesAndStreams) yield (tt
-> qs)
If there were topic-threadIds t1, t2, and queues q1, q2, you would get (t1
-> q1) (t1, q2) (t2 -> q1) (t2 -> q2) and they were sequentially added to
the topicThreadIdAndQueues pool. So all the threads would feed into the same
queue.
Re: the computation is different because topicThreadIds in wildcard
consumption may share the same queue so you can't zip them directly. e.g.,
whether there are two or 100 allowed topics, they will all share the same
num-streams queues, and num-streams != the number of threads, so you can't
zip them. In the static topic count case, there is one-to-one correspondence
so you can zip them.
That said, I think the code can be made clearer here. I will upload another
patch in a minute.
> Wildcarded consumption is single-threaded
> -----------------------------------------
>
> Key: KAFKA-550
> URL: https://issues.apache.org/jira/browse/KAFKA-550
> Project: Kafka
> Issue Type: Bug
> Reporter: Joel Koshy
> Assignee: Joel Koshy
> Attachments: KAFKA-550-v1.patch
>
>
> It's surprising that we haven't noticed this before, but I was looking at a
> CPU usage profile on yourkit and It turns out that only one mirror maker
> thread is actually doing anything. Basically I suspect some bug in fetcher ->
> queue mapping. Only one queue seems to have any data. I'll look into this
> probably next week.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
