Mirrormaker only publishing to a single partition on destination cluster

2016-03-03 Thread Stephen Powis
Hey! I'm using kafka 0.9.0.1 and trying to replicate a cluster from one datacenter to another. mirror-maker properly connects to my source cluster and consumes messages, but for some reason is only publishing to a single partition for my topic in the destination cluster. So all of my partitions

Re: Mirrormaker only publishing to a single partition on destination cluster

2016-03-04 Thread Stephen Powis
Took me a bit but worked out my issue. Appears like all of the messages in my topics were being hashed to a single partition using the DefaultPartitioner. Ended up rolling a simple round robin partitioner and contributed that back via this ticket: https://issues.apache.org/jira/browse/KAFKA-