[
https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344150#comment-17344150
]
A. Sophie Blee-Goldman commented on KAFKA-12776:
------------------------------------------------
I don't think there's any way to prevent #send from blocking on the metadata at
the moment. That's the way the method is currently written, and what should
hopefully be improved by KIP-739. The reason it blocks for the metadata is that
this is an input to the Partitioner, so creating a custom partitioner wouldn't
help as the Producer thinks it needs to supply the metadata no matter what. You
could try to change how this works, but it will require a KIP of its own, so
you may as well wait for KIP-739.
Out of ordering is possible when you have
_max.in.flight.requests.per.connection_ set to greater than 1, even with the
idempotent producer. It's not a bug in the Producer API, that's just how this
config works.
> Producer sends messages out-of-order inspite of enabling idempotence
> --------------------------------------------------------------------
>
> Key: KAFKA-12776
> URL: https://issues.apache.org/jira/browse/KAFKA-12776
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 2.6.0, 2.7.0
> Environment: Linux RHEL 7.9 and Ubuntu 20.04
> Reporter: NEERAJ VAIDYA
> Priority: Major
> Attachments: mocker.zip
>
>
> I have an Apache Kafka 2.6 Producer which writes to topic-A (TA).
> My application is basically a Spring boot web-application which accepts JSON
> payloads via HTTP and then pushes each to a Kafka topic. I also use Spring
> Cloud Stream Kafka in the application to create and use a Producer.
> For one of my failure handling test cases, I shutdown the Kafka cluster while
> my applications are running. (Note : No messages have been published to the
> Kafka cluster before I stop the cluster)
> When the producer application tries to write messages to TA, it cannot
> because the cluster is down and hence (I assume) buffers the messages. Let's
> say it receives 4 messages m1,m2,m3,m4 in increasing time order. (i.e. m1 is
> first and m4 is last).
> When I bring the Kafka cluster back online, the producer sends the buffered
> messages to the topic, but they are not in order. I receive for example, m2
> then m3 then m1 and then m4.
> Why is that ? Is it because the buffering in the producer is multi-threaded
> with each producing to the topic at the same time ?
> My project code is attached herewith.
> I can confirm that I have enabled idempotence. I have also tried with
> ```max.in.flight.requests=1```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
