Thanks for the answer Matthias. I still have doubts about the meaning of "risks reordering of sent record". If I understood correctly the example you gave is something like this 1. Producer sends batch with sequence number X 2. That request gets lost in the network 3. Producer sends batch with sequence number X+1 4. Broker receives batch with sequence number X+1 and returns an error and the Producer throws a OutOfOrderSequenceException
In that situation we could keep retrying sending batch with sequence number X+1 but we will keep getting a OutOfOrderSequenceException, or we ideally also resend a batch with sequence number X, and after being accepted send the one with X+1. If what I'm saying is correct then I can't see how this can reorder the messages, I mean if both batches include a message being written to topic A, could messages from batch with sn X+1 end up being persisted with an offset lesser than the ones from the batch with sn X? Does this question make sense? El mar, 7 jun 2022 a las 16:13, Matthias J. Sax (<mj...@apache.org>) escribió: > Yes, the broker de-dupes using the sequence number. > > But for example, if a sequence number is skipped, you could get this > exception: the current batch of messages cannot be appended to the log, > as one batch is missing, and the producer would need to re-send the > previous/missing batch with lower sequence number before it can move to > the "next" (ie current) batch. > > Does this make sense? > > > -Matthias > > On 5/27/22 10:43 AM, Gabriel Giussi wrote: > > The docs say > > "This exception indicates that the broker received an unexpected sequence > > number from the producer, which means that data may have been lost. If > the > > producer is configured for idempotence only (i.e. if enable.idempotence > is > > set and no transactional.id is configured), it is possible to continue > > sending with the same producer instance, but doing so risks reordering of > > sent record" > > > > Isn't the broker using the monotonically increasing sequence number to > > dedup messages? So how can it break message ordering without breaking > > idempotency? > > I can't see an example scenario where this could happen, I guess > > the OutOfOrderSequenceException can only happen > > with max.in.flight.requests.per.connection > 1, but even in that case why > > are not going to keep getting an OutOfOrderSequenceException but instead > a > > success that broke message ordering? > > > > Thanks. > > >
