[
https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17345848#comment-17345848
]
NEERAJ VAIDYA commented on KAFKA-12776:
---------------------------------------
Thanks [~guozhang]
To conclude, am I correct in assuming that :
Kafka Producer#send API does not guarantee ordering of messages when triggered
by multiple threads. And there is no such KIP which intends to solve this
potential problem which could occur even in case when there is no cluster
failure or reachability issues.
I agree about the skew in threads just that the point I am trying to make is
that given this point about skewness, if send can push the message in the order
in which it was called (which in most cases should match the order of HTTP
requests) and basically return a Future right away, then atleast we don't run
into the dis-ordering caused when the cluster is down. The requests that we get
via the HTTP interface are spaced apart for a single user (mobile phone)
session, so I doubt their individual threads should contend for order if `send`
can append in order in which the HTTP requests are received.
Today, if the cluster is unreachable, the messages received in a 10 minute
period of cluster downtime are completely out-of-sequence.
> Producer sends messages out-of-order inspite of enabling idempotence
> --------------------------------------------------------------------
>
> Key: KAFKA-12776
> URL: https://issues.apache.org/jira/browse/KAFKA-12776
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 2.6.0, 2.7.0
> Environment: Linux RHEL 7.9 and Ubuntu 20.04
> Reporter: NEERAJ VAIDYA
> Priority: Major
> Attachments: mocker.zip
>
>
> I have an Apache Kafka 2.6 Producer which writes to topic-A (TA).
> My application is basically a Spring boot web-application which accepts JSON
> payloads via HTTP and then pushes each to a Kafka topic. I also use Spring
> Cloud Stream Kafka in the application to create and use a Producer.
> For one of my failure handling test cases, I shutdown the Kafka cluster while
> my applications are running. (Note : No messages have been published to the
> Kafka cluster before I stop the cluster)
> When the producer application tries to write messages to TA, it cannot
> because the cluster is down and hence (I assume) buffers the messages. Let's
> say it receives 4 messages m1,m2,m3,m4 in increasing time order. (i.e. m1 is
> first and m4 is last).
> When I bring the Kafka cluster back online, the producer sends the buffered
> messages to the topic, but they are not in order. I receive for example, m2
> then m3 then m1 and then m4.
> Why is that ? Is it because the buffering in the producer is multi-threaded
> with each producing to the topic at the same time ?
> My project code is attached herewith.
> I can confirm that I have enabled idempotence. I have also tried with
> ```max.in.flight.requests=1```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
