Hi,
I tried google and found one possible related issue in [1]. Could you try to
remove the config
properties.setProperty(ProducerConfig.RETRIES_CONFIG, "5");
and try again ?
Best,
Yun
[1]
https://stackoverflow.com/questions/55192852/transactional-producer-vs-just-idempotent-producer-java-exception-outoforderseq
--Original Mail --
Sender:SmileSmile
Send Date:Fri Jun 4 23:24:06 2021
Recipients:Yun Gao
CC:user
Subject:Re:Re: open checkpoint, send message to kafka
OutOfOrderSequenceException
Hi,
after failover still OutOfOrderSequenceException. when I close checkpoint,
kafka broker still return OutOfOrderSequenceException to me .
At 2021-06-04 17:52:22, "Yun Gao" wrote:
Hi,
Have you checked if the error during normal execution, or right after failover?
Best,
Yun
--
From:SmileSmile
Send Time:2021 Jun. 4 (Fri.) 11:07
To:user
Subject:open checkpoint, send message to kafka OutOfOrderSequenceException
Dear all: flink version is 1.12.4,kafka version is 1.1.1。topology is
very simple ,source-->flatmap--->sink ,enable checkpoint,job will fail after a
few hours 。 the error message is Caused by:
org.apache.flink.streaming.connectors.kafka.FlinkKafkaException: Failed to send
data to Kafka: The broker received an out of order sequence number.at
org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.checkErroneous(FlinkKafkaProducer.java:1392)
at
org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.invoke(FlinkKafkaProducer.java:850)
at
org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.invoke(FlinkKafkaProducer.java:99)
product's properties is simle , using defalut AT_LEAST_ONCE semantic
properties.setProperty(ProducerConfig.COMPRESSION_TYPE_CONFIG, "lz4");
properties.setProperty(ProducerConfig.RETRIES_CONFIG, "5");
properties.setProperty(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION,
"1");properties.setProperty(ProducerConfig.RETRY_BACKOFF_MS_CONFIG,
"2000");properties.setProperty(ProducerConfig.BATCH_SIZE_CONFIG,
"524288");properties.setProperty(ProducerConfig.LINGER_MS_CONFIG,
"100");properties.setProperty(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG,
"6");My questions is how to deal with ?