[
https://issues.apache.org/jira/browse/KAFKA-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13723074#comment-13723074
]
Jay Kreps commented on KAFKA-925:
---------------------------------
Good point, type parameter is not needed in DefaultPartitioner.
Yes, mirror maker should always partition with the partition number rather than
trying to reverse engineer the client partitioning logic. This additional key
is specifically for the case where you want to partition by something OTHER
than the stored key.
For console producer, yes, we should default to not losing data. Not really
related to this issue, but a good change.
> Add optional partition key override in producer
> -----------------------------------------------
>
> Key: KAFKA-925
> URL: https://issues.apache.org/jira/browse/KAFKA-925
> Project: Kafka
> Issue Type: New Feature
> Components: producer
> Affects Versions: 0.8.1
> Reporter: Jay Kreps
> Assignee: Jay Kreps
> Fix For: 0.8.1
>
> Attachments: KAFKA-925-v1.patch, KAFKA-925-v2.patch
>
>
> We have a key that is used for partitioning in the producer and stored with
> the message. Actually these uses, though often the same, could be different.
> The two meanings are effectively:
> 1. Assignment to a partition
> 2. Deduplication within a partition
> In cases where we want to allow the client to take advantage of both of these
> and they aren't the same it would be nice to allow them to be specified
> separately.
> To implement this I added an optional partition key to KeyedMessage. When
> specified this key is used for partitioning rather than the message key. This
> key is of type Any and the parametric typing is removed from the partitioner
> to allow it to work with either key.
> An alternative would be to allow the partition id to specified in the
> KeyedMessage. This would be slightly more convenient in the case where there
> is no partition key but instead you know a priori the partition number--this
> case must be handled by giving the partition id as the partition key and
> using an identity partitioner which is slightly more roundabout. However this
> is inconsistent with the normal partitioning which requires a key in the case
> where the partition is determined by a key--in that case you would be
> manually calling your partitioner in user code. It seems best to me to either
> use a key or always a partition and since we currently take a key I stuck
> with that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
