[
https://issues.apache.org/jira/browse/KAFKA-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13847812#comment-13847812
]
Adrian Muraru commented on KAFKA-1183:
--------------------------------------
[~junrao] Jun, what's the impact of lowering the metadata refresh interval to
say 10 seconds, this would solve the partition stickiness in the producer but
would that have other performance side effects? Reading the email thread on
KAFKA-1067 I see that creating a customised random partioner is not possible at
this point as it's not aware of AVAILABLE partitions. Thanks
> DefaultEventHandler causes unbalanced distribution of messages across
> partitions
> --------------------------------------------------------------------------------
>
> Key: KAFKA-1183
> URL: https://issues.apache.org/jira/browse/KAFKA-1183
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.8.0
> Reporter: Dragos Dena
> Assignee: Jun Rao
> Fix For: 0.8.1, 0.9.0
>
> Attachments: KAFKA-1183-trunk.patch
>
>
> KAFKA-959 introduced an optimisation in {{DefaultEventHandler}} that was
> supposed to have the effect of sending all messages from the same batch to a
> single partition if no key is specified.
> The problem is that the {{sendPartitionPerTopicCache}} cache, which holds the
> current selected partition for each topic, isn't actually invalided at the
> start or end of each batch.
> The observed result is that, after the first request chooses a random
> partition, all subsequent messages from that producer land in the same
> partition. If you have a large number of producers, then it should be fine,
> but if your producer count is comparable to the partition count, then it will
> get unbalanced.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
