Not 100% sure if librdkafka provides the exact some semantics as the
Kafka Java producer (but I believe it does for this case; but might be
worth to check the docs?).
If `send()` gives an error back, ordering would not be guaranteed any
longer if the retry happens at the application level. Order can only be
guaranteed as long as the producer retries internally, so you might want
to increase retires configs. And yes, enabling idempotent feature would
be requires, otherwise, even internal producer retries would not
guarantee ordering.
-Matthias
On 7/24/24 1:54 AM, Abhijit Dutta wrote:
Hi,
My apology if this is a question which is already been answered before.
Use Case:
I am reviewing an application (producer) which is using librdKafka as a 3rd
party library to interact with Kafka brokers (3= 1L+2F). This 3rd party lib has
been configured for retries. Its a single topic with multiple partitions.
Application suppose to maintain a strict ordering without any loss of messages.
What I found is if the library fails to deliver the message after retires and
notifies the application, it then again retries to send.
(idempotent feature is enabled.)
Question:
1.
Will this break the ordering of message, if the application retries?
2. Will enabling idempotent going to help here ?
3. Is the retry from application unnecessary overhead in ALL cases.
Thanx
Abhijit
