[
https://issues.apache.org/jira/browse/KAFKA-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869371#comment-17869371
]
David Jacot commented on KAFKA-17116:
-------------------------------------
Hi all, I was thinking about two options to resolve this issue:
# We could actually generate the member id on the client side. This is already
supported by the protocol. Back then, we rejected this option because it was
considered too difficult for non java clients such as librdkafka as some
languages do not have built-in uuid support. This may be better now. We can
check this. We also need to convince ourselves that letting the client generate
it is good enough.
# We could use the first HB send out by the client to generate the uuid
without adding the member to the group yet. So, the first HB would send back
the new member id, zero as the member epoch as, zero as the heartbeat interval
so the client HB immediately. This is basically a way to do 1. but without
relying on the client to generate it. Note that this does not require any
changes to the protocol too. An alternative to this would be to return a new
error code to make it explicit but I am not sure if it is worth it.
I am not a fan of the idea of passing a temporary ID on the client that we
would store on the server. If we believe that we could generate an id on the
client, we should just do 1.
I think that we could also improve the client state machine to better handle
those cases.
> New consumer may not send effective leave group if member ID received after
> close
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-17116
> URL: https://issues.apache.org/jira/browse/KAFKA-17116
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.8.0
> Reporter: Lianet Magrans
> Assignee: TengYao Chi
> Priority: Major
> Labels: kip-848-client-support
> Fix For: 3.9.0
>
>
> If the new consumer is closed after sending a HB to join, but before
> receiving the response to it, it will send a leave group request but without
> member ID (will simply fail with UNKNOWN_MEMBER_ID). This will make that the
> broker will have a registered new member, for which it will never receive a
> leave request for it.
> # consumer.subscribe -> sends HB to join, transitions to JOINING
> # consumer.close -> will transition to LEAVING and send HB with epoch -1
> (without waiting for in-flight requests)
> # consumer receives response to initial HB, containing the assigned member
> ID. It will simply ignore it because it's not in the group anymore
> (UNSUBSCRIBED)
> Note that the expectation, with the current logic, and main downsides of this
> are:
> # If the case was that the member received partitions on the first HB, those
> partitions won't be re-assigned (broker waiting for the closed consumer to
> reconcile them), until the rebalance timeout expires.
> # Even if no partitions were assigned to it, the member will remain in the
> group from the broker point of view (but not from the client POV). The member
> will be eventually kicked out for not sending HBs, but only when it's session
> timeout expires.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
