suzhiking commented on PR #23266: URL: https://github.com/apache/kafka/pull/23266#issuecomment-5530965908
> uhm interesting. I see the gap, just not sure about the approach to fix (from a first look) > > Fixing by keeping the member ID on the static member seems simple but the implications seems tricky. E.g., it makes that the case of a static member unsubscribe/subcsribe may be treated on the GC just as as a known member intentionally rejoining when metadata changes (to trigger a rebalance if leader). But we shouldn't rebalance when a static member/leader just unsubscribe/subscribe, and with this PR seems we are? @frankvicky raised the same point above, and we think it is an acceptable trade-off. The classic protocol has no way to tell "same member re-attaching after unsubscribe" from "member intentionally rejoining", so any fix that keeps the member id inherits this. The old code avoided it only by making the member impersonate a new instance, which is exactly what enables the self-fence. The cases that change are unsubscribe and poll timeout, but both are already disruptive: the client has given up its assignment before rejoining, and Streams has closed all tasks. The rejoining member pays the same recovery cost either way; the only new cost is one cooperative rebalance in which the other members keep their partitions under the sticky assignor. For Streams specifically, the assignor deliberately makes its subscription bytes unique on every join precisely so a rejoining thread triggers a rebalance, so the new behavior matches what Streams already asks for, and the old static-replacement shortcut was bypassing that. Plain-consumer followers with an unchanged subscription are unaffected. Also the old path did not act on a changed subscription; the client only noticed the mismatch on the returned assignment and rejoined, so the new path is more correct there too. Against all of this stands a fatal `SHUTDOWN_APPLICATION` with no recovery. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
