[
https://issues.apache.org/jira/browse/KAFKA-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092237#comment-14092237
]
Jun Rao commented on KAFKA-1586:
--------------------------------
One way to address this issue is introduce a new config
"partition.sticky.time.ms" in the new producer. The producer will then stick to
a partition for the configured amount of time before switching to another.
"partition.sticky.time.ms" can default to 0, which means every message will
switch to a new partition.
> support sticky partitioning in the new producer
> -----------------------------------------------
>
> Key: KAFKA-1586
> URL: https://issues.apache.org/jira/browse/KAFKA-1586
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.2
> Reporter: Jun Rao
>
> If a message doesn't specify a key or a partition, the new producer selects a
> partition for each message in a round-robin way. As a result, in a window of
> linger.ms, messages are spread around in all partitions of a topic. Compared
> with another strategy that assigns all messages to a single partition in the
> same time window, this strategy may not compress the message set as well
> since the batch is smaller. Another potential problem with this strategy is
> that the compression ratio could be sensitive to the change of # partitions
> in a topic. If # partitions are increased in a topic, the produced data may
> not be compressed as well as before.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
