This behavior might be expected, quoting description in apache kafka documentation here <http://kafka.apache.org/documentation.html>
Kafka only provides a total order over messages within a partition, not > between different partitions in a topic. Per-partition ordering combined > with the ability to partition data by key is sufficient for most > applications. However, if you require a total order over messages this can > be achieved with a topic that has only one partition, though this will mean > only one consumer process per consumer group. On Wed, Nov 25, 2015 at 8:54 AM, Guozhang Wang <wangg...@gmail.com> wrote: > With ack=1, if your code is does sth. like: > > producer1.send(msg1).get() // get() blocks until a response is received. > producer2.send(msg2).get() // get() blocks until a response is received. > > Then the ordering is guaranteed though under a broker failure the acked > messages may be lost if they have not been replicated. > > Guozhang > > > > On Mon, Nov 23, 2015 at 11:08 PM, Yonghui Zhao <zhaoyong...@gmail.com> > wrote: > > > Thanks Guozhang， Most params are not set in our config . So max retries > > should be 3 by default > > > > In your explanation: > > > > a. msg1 sent, produce() returned. > > b. msg2 sent. > > c. msg1 failed, and retried. > > d. msg2 acked. > > e. msg1 acked. > > > > But is acks is 0, "the retries configuration will not take effect (as > > the client won't generally know of any failures). " > > > > > > And I find in new producer to be released acks default value is already > > changed to 1. > > > > > > If acks is set to 1, the order will be maintained even they are sent from > > different producers? > > > > > > > > 2015-11-24 2:47 GMT+08:00 Guozhang Wang <wangg...@gmail.com>: > > > > > Yonghui, > > > > > > With ack = 0 and retries > 0 this could happen since producer.send() > > > returns before it gets acked from the broker, so you could get the > > > following trace: > > > > > > a. msg1 sent, produce() returned. > > > b. msg2 sent. > > > c. msg1 failed, and retried. > > > d. msg2 acked. > > > e. msg1 acked. > > > > > > Assuming you said "msg1 and msg2 may notbe sent by one producer", out > of > > > ordering can even happen more likely as they could arrive to the broker > > > from different sockets at arbitrary ordering. > > > > > > Guozhang > > > > > > On Mon, Nov 23, 2015 at 4:31 AM, Yonghui Zhao <zhaoyong...@gmail.com> > > > wrote: > > > > > > > The ack mode is default value 0. > > > > > > > > > > > > - 0, which means that the producer never waits for an > > acknowledgement > > > > from the broker (the same behavior as 0.7). This option provides > the > > > > lowest > > > > latency but the weakest durability guarantees (some data will be > > lost > > > > when > > > > a server fails). > > > > > > > > And msg1 and msg2 may not be sent by one broker. > > > > > > > > > > > > 2015-11-21 7:56 GMT+08:00 Guozhang Wang <wangg...@gmail.com>: > > > > > > > > > Yonghui, > > > > > > > > > > What is the ack mode for the producer clients? And are msg1 and > msg2 > > > sent > > > > > by the same producer? > > > > > > > > > > Guozhang > > > > > > > > > > On Thu, Nov 19, 2015 at 10:59 PM, Yonghui Zhao < > > zhaoyong...@gmail.com> > > > > > wrote: > > > > > > > > > > > Broker setting is: 8 partitions, 1 replica, kafka version 0.8.1 > > > > > > > > > > > > We send 2 message at almost same time. > > > > > > > > > > > > Msg1 first, Msg2 second. > > > > > > > > > > > > We have more than 1 producers in sync mode. > > > > > > > > > > > > We may send msg1 in one broker, after *producer.send return > > response* > > > > > > And send msg2 in the other broker. > > > > > > > > > > > > Both 2 msg has same partition key. > > > > > > > > > > > > In consumer side, we find the 2 msg is in same partition as > > expected, > > > > > but > > > > > > the order is inverted as msg2, msg1. > > > > > > > > > > > > Is it possible for kafka? > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- Guozhang > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > -- > -- Guozhang > -- Muqtafi Akhmad Software Engineer Traveloka
