Hi there,
Finally we upgraded our producer configuration to ensure message order:
retries = 1
# note to ensure order enable.idempotence=true, which forcing to acks=all and
max.in.flight.requests.per.connection<=5
enable.idempotence = true
max.in.flight.requests.per.connection = 4
acks = "all"
However, recently we faced this exception:
org.apache.kafka.common.errors.OutOfOrderSequenceException: The broker received
an out of order sequence number..
Any idea why happened? Is that expected?
It could be related to retries configuration? It’s that configuration properly
set?
From official
documentation<https://kafka.apache.org/21/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html>
we found recommending unset retries, being default value to Integer.MAX_VALUE:
“To take advantage of the idempotent producer, it is imperative to avoid
application level re-sends since these cannot be de-duplicated. As such, if an
application enables idempotence, it is recommended to leave the retries config
unset, as it will be defaulted to Integer.MAX_VALUE. Additionally, if a
send(ProducerRecord) returns an error even with infinite retries (for instance
if the message expires in the buffer before being sent), then it is recommended
to shut down the producer and check the contents of the last produced message
to ensure that it is not duplicated. Finally, the producer can only guarantee
idempotence for messages sent within a single session.”
Thanks
[https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180]<http://www.williamhill.com/>
[https://www.williamhillplc.com/content/signature/senet.gif?width=180]<http://www.whenthefunstops.co.uk/>
Jose Manuel Vega Monroy
Java Developer / Software Developer Engineer in Test
Direct: +0035 0 2008038 (Ext. 8038)
Email: [email protected]<mailto:[email protected]>
William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
