Shekhar Prasad Rajak created KAFKA-20458:
--------------------------------------------
Summary: AcknowledgeType.RENEW re-delivers records to application
via poll() in explicit acknowledgement mode, undocumented and inconsistent with
broker-side contract
Key: KAFKA-20458
URL: https://issues.apache.org/jira/browse/KAFKA-20458
Project: Kafka
Issue Type: Bug
Affects Versions: 4.2.0
Reporter: Shekhar Prasad Rajak
AcknowledgeType.RENEW is documented as a way to extend the acquisition lock on
a record that is still being processed, without changing the record's state on
the broker. However, the client-side implementation silently re-delivers the
same records back to the application on the next poll() call. This behavior is
nowhere documented, violates the explicit acknowledgement mode invariant, and
diverges from the broker-side contract.
RENEW simultaneously satisfies two contradictory contracts:
Context ----What RENEW implies
ensureInFlightAcknowledgedIfExplicitAcknowledgement() --- Record IS
acknowledged — gate passes
collect() / takeRenewedRecords() ----Record is NOT done — re-delivered on next
poll()
Two acceptable resolutions, depending on intended design:
Option A — RENEW means "extend lock only, no re-delivery":
Option B — RENEW means "extend lock and re-deliver to same consumer" (current
behavior, make it explicit).
Our usecase :
We are building an async pipeline where records are consumed, published to a
downstream system asynchronously, and then acknowledged. We need to:
1. Hold the broker acquisition lock while async publish is in flight.
2. Not lose the record if publish takes longer than one poll cycle.
3. Call ACCEPT once publish confirms, RELEASE if it fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
