[
https://issues.apache.org/jira/browse/KAFKA-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472523#comment-13472523
]
Neha Narkhede commented on KAFKA-543:
-------------------------------------
Could you explain (1) a little more ? What I think is that if any correct
usage of this API needs the input to be deduped, there are two choices - (1)
Have this API do the de-duping (2) Have it input a Set so that callers will
definitely dedup. Right now, it still ends up taking a Seq as input and every
caller has to remember to dedup the sequence, which is dangerous.
> Metadata request from DefaultEventHandler.handle repeats same topic over and
> over
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-543
> URL: https://issues.apache.org/jira/browse/KAFKA-543
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Priority: Blocker
> Labels: bugs
> Attachments: KAFKA-543.patch
>
>
> It looks like we are calling BrokerPartitionInfo.updateInfo() with a list of
> the same topic repeated many times:
> Here is the line:
> Utils.swallowError(brokerPartitionInfo.updateInfo(outstandingProduceRequests.map(_.getTopic)))
> The outstandingProduceRequests can (and generally would) have many entries
> for the same topic.
> For example if I use the producer performance test with the default batch
> size on a topic "test" my metadata request will have the topic "test"
> repeated 200 times. On the server side we do several zk reads for each of
> these repetitions.
> This is causing the metadata api to timeout in my perf test periodically.
> I think the fix is simply to de-duplicate prior to the call (and perhaps
> again on the server in case of a misbehaving client).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
