Another idea. If a set of messages arrive over a single TCP connection, route to a partition depending on TCP connection.
To be honest, these approaches, while they work, may not scale when the message rate is high. If at all possible, try to think of a way to remove this requirement from your design. For example, a design might have a sequence number assigned to each message before it goes into Kafka (a time-based UUID, for example), and something later in the pipe line sorts it all out. Kafka then does what is does best, IMHO, a high-performance reliable, message bus. Philip On Jun 14, 2013, at 7:37 AM, David Arthur <[email protected]> wrote: > Simple example of how to take advantage of this behavior: > > Suppose you're sending document updates through Kafka. If you use the > document ID as the message key and the default hash partitioner, the updates > for a given document will exist on the same partition and come into the > consumer in order. > > On 6/10/13 8:37 AM, Neha Narkhede wrote: >> Kafka guarantees order per topic partition per source client. >> >> Thanks, >> Neha >> On Jun 9, 2013 5:33 PM, "S Ahmed" <[email protected]> wrote: >> >>> I understand that there are no guarantees per say that a message may be a >>> duplicate (its the consumer's job to guarantee that), but when it comes to >>> message order, is kafka built in such a way that it is impossible to get >>> messages in the wrong order? >>> >>> Certain use cases might not be sensitive to order, but when order is very >>> important, is kafka the wrong tool for the job or is there a way to get >>> this requirement? >
