So it's not possible to have 1 topic with 1 partition and many consumers of that topic?
My intention is to have a topic with many consumers, but each consumer needs to be able to have access to all the messages in that topic. On 11 February 2015 at 20:42, Zijing Guo <alter...@yahoo.com.invalid> wrote: > Partition key is on producer level, that if you have multiple partitions > for a single topic, then you can pass in a key for the KeyedMessage object, > and base on different partition.class, it will return a partition number > for the producer, and producer will find the leader for that partition.I > don't know how kafka could handle time series case, but depends on how many > partitions for that topic. If you only have 1 partition, then you don't > need to worry about order at all, since each consumer group can only allow > 1 consumer instance to consume that data. if you have multiple partitions > (say 3 for example), then you can fire up 3 consumer instances under the > same consumer group, and each will only consume 1 partition's data. if > order in each partition matters, then you need to do some work on the > producer side.Hope this helpsEdwin > > On Wednesday, February 11, 2015 3:14 PM, Gary Ogden <gog...@gmail.com> > wrote: > > > I'm trying to understand how the partition key works and whether I need to > specify a partition key for my topics or not. What happens if I don't > specify a PK and I have more than one consumer that wants all messages in a > topic for a certain period of time? Will those consumers get all the > messages, but they just may not be ordered correctly? > > The current scenario is that we will have events going into a topic based > on customer and the data will remain in the topic for 24 hours. We will > then have multiple consumers reading messages from that topic. They will > want to be able to get them out over a time range (could be last hour, last > 8 hours etc). > > So if I specify the same PK for each subscriber, then each consumer will > get all messages in the correct order? If I don't specify the PK or use a > random one, will each consumer still get all the messages but they just > won't be ordered correctly? > > > >
