Lucas Brutschy created KAFKA-20167:
--------------------------------------
Summary: Introduce CloseOptions.DEFAULT for Kafka Streams
Key: KAFKA-20167
URL: https://issues.apache.org/jira/browse/KAFKA-20167
Project: Kafka
Issue Type: Task
Components: streams
Reporter: Lucas Brutschy
Currently, {{KafkaStreams}} allows users to specify how to close the
application using {{{}CloseOptions{}}}. The existing options are
{{LEAVE_GROUP}} and {{{}REMAIN_IN_GROUP{}}}.
The default behavior (when no options are passed) is {{{}REMAIN_IN_GROUP{}}}.
However, with the introduction of the Streams Protocol (KIP-1071), the behavior
of {{REMAIN_IN_GROUP}} has become inconsistent:
* *Classic Protocol:* The member actually remains in the group.
* *Streams Protocol (KIP-1071):* The member leaves the group unconditionally
(as staying in the group for non-static member provides no benefits).
To resolve this ambiguity and provide a cleaner API for the long term, we
should introduce a new enum value: {{{}CloseOptions.DEFAULT{}}}. This allows
the "default" behavior to be adaptive based on the underlying protocol, while
strictly reserving {{REMAIN_IN_GROUP}} for when the user explicitly requests
that intent.
h3. *Proposed Changes*
*1. Update {{org.apache.kafka.streams.CloseOptions}}*
Add a new enum value {{{}DEFAULT{}}}.
*2. Update {{KafkaStreams.close()}} Behavior*
* *Current:* {{KafkaStreams.close()}} calls
{{{}close(CloseOptions.REMAIN_IN_GROUP){}}}.
* *Proposed:* {{KafkaStreams.close()}} calls
{{{}close(CloseOptions.DEFAULT){}}}.
*3. Define Logic for {{CloseOptions.DEFAULT}}*
The new {{DEFAULT}} option should act as a "smart" selection that adheres to
the best practices of the specific protocol version being used:
* *If Classic Protocol:* Map to Consumer {{CloseOptions.REMAIN_IN_GROUP}}
(Preserves legacy behavior).
* *If Streams Protocol:* Map to Consumer {{CloseOptions.DEFAULT}} (Typically
leaves the group for dynamic members).
*4. Redefine {{CloseOptions.REMAIN_IN_GROUP}}*
Currently, {{REMAIN_IN_GROUP}} is the implicit default. It should become an
explicit instruction.
* It should always map to Consumer {{{}CloseOptions.REMAIN_IN_GROUP{}}},
regardless of the protocol.
* _Note:_ Even if the Streams protocol currently forces a leave, passing this
explicit option ensures the intent is passed down to the consumer client,
rather than being intercepted by Streams logic.
h3. *Behavior Matrix*
|*Streams CloseOption*|*Classic Protocol Behavior*|*Streams Protocol (KIP-1071)
Behavior*|
|*DEFAULT* (New)|Uses Consumer {{REMAIN}}|Uses Consumer {{DEFAULT}}|
|*REMAIN_IN_GROUP*|Uses Consumer {{REMAIN}}|Uses Consumer {{REMAIN}}|
|*LEAVE_GROUP*|Uses Consumer {{LEAVE}}|Uses Consumer {{LEAVE}}|
h3. *Motivation*
* *Clarity:* {{REMAIN_IN_GROUP}} should mean exactly what it says. If the
underlying protocol cannot support it, it is better for the Consumer client to
handle that rejection/fallback rather than Kafka Streams masking it.
* *Future Proofing:* By using {{{}DEFAULT{}}}, we allow the "standard" close
behavior to evolve with future protocols (e.g., KIP-848/1071 evolutions)
without breaking users who explicitly asked to {{{}REMAIN_IN_GROUP{}}}.
h3. *Compatibility*
* This requires a KIP as it changes the {{CloseOptions}} public API.
* The default behavior for Classic Protocol users remains unchanged (they
still remain in the group).
* The default behavior for Streams Protocol users becomes cleaner, mapping to
the Consumer's default rather than a forced mismatch.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
