[
https://issues.apache.org/jira/browse/KAFKA-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14957113#comment-14957113
]
ASF GitHub Bot commented on KAFKA-2649:
---
GitHub user rhauch opened a pull request:
https://github.com/apache/kafka/pull/309
KAFKA-2649 Add support for custom partitioning in topology sinks
Added option to use custom partitioning logic within each topology sink.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rhauch/kafka kafka-2649
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/309.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #309
commit 4a5a2adf6667bcaeccc7b49d63923b1b22e19d16
Author: Randall Hauch
Date: 2015-10-14T15:19:43Z
KAFKA-2649 Add support for custom partitioning in topology sinks
Added option to use custom partitioning logic within each topology sink.
> Add support for custom partitioner in sink nodes
>
>
> Key: KAFKA-2649
> URL: https://issues.apache.org/jira/browse/KAFKA-2649
> Project: Kafka
> Issue Type: Sub-task
> Components: kafka streams
>Reporter: Randall Hauch
>Assignee: Randall Hauch
> Fix For: 0.9.0.0
>
>
> The only way for Processor implementations to control partitioning of
> forwarded messages is to set the partitioner class as property
> {{ProducerConfig.PARTITIONER_CLASS_CONFIG}} in the StreamsConfig, which
> should be set to the name of a
> {{org.apache.kafka.clients.producer.Partitioner}} implementation. However,
> doing this requires the partitioner knowing how to properly partition *all*
> topics, not just the one or few topics used by the Processor.
> Instead, Kafka Streams should make it easy to optionally add a partitioning
> function for each sink used in a topology. Each sink represents a single
> output topic, and thus is far simpler to implement. Additionally, the sink is
> already typed with the key and value types (via serdes for the keys and
> values), so the partitioner can be also be typed with the key and value
> types. Finally, this also keeps the logic of choosing partitioning strategies
> where it belongs, as part of building the topology.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
