Hi all, Even though KIP-88 was recently approved, due to a limitation that comes with the proposed protocol change in KIP-88 I'll have to re-open it to address the problem. I'd like to thank Jason Gustafson for catching this issue.
I'll explain this in the KIP as well, but to summarize, KIP-88 suggests adding the option of passing a "null" array in FetchOffset request to query all existing offsets for a consumer group. It does not suggest any modification to FetchOffset response. In the existing protocol, group or coordinator related errors are reported along with each partition in the OffsetFetch response. If there are partitions in the request, they are guaranteed to appear in the response (there could be an error code associated with each). So if there is an error, it is reported back by being attached to some partition in the request. If an empty array is passed, no error is reported (no matter what the group or coordinator status is). The response comes back with an empty list. With the proposed change in KIP-88 we could have a scenario in which a null array is sent in FetchOffset request, and due to some errors (for example if coordinator just started and hasn't caught up yet with the offset topic), an empty list is returned in the FetchOffset response (the group may or may not actually be empty). The issue is in situations like this no error can be returned in the response because there is no partition to attach the error to. I'll update the KIP with more details and propose to add to OffsetFetch response schema an "error_code" at the top level that can be used to report group related errors (instead of reporting those errors with each individual partition). I apologize if this causes any inconvenience. Feedback and comments are always welcome. Thanks. --Vahid
