[ https://issues.apache.org/jira/browse/KAFKA-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao updated KAFKA-2687: --------------------------- Description: Since the new consumer currently has no persistence in Zookeeper (pending outcome of KAFKA-2017), there is no way for administrators to investigate group status including getting the list of members in the group and their partition assignments. We therefore propose to modify GroupMetadataRequest (previously known as ConsumerMetadataRequest) to return group metadata when received by the respective group's coordinator. When received by another broker, the request will be handled as before: by only returning coordinator host and port information. {code} GroupMetadataRequest => GroupId IncludeMetadata GroupId => String IncludeMetadata => Boolean GroupMetadataResponse => ErrorCode Coordinator GroupMetadata ErrorCode => int16 Coordinator => Id Host Port Id => int32 Host => string Port => int32 GroupMetadata => State ProtocolType Generation Protocol Leader Members State => String ProtocolType => String Generation => int32 Protocol => String Leader => String Members => [Member MemberMetadata MemberAssignment] Member => MemberIp ClientId MemberIp => String ClientId => String MemberMetadata => Bytes MemberAssignment => Bytes {code} The request schema includes a flag to indicate whether metadata is needed, which saves clients from having to read all group metadata when they are just trying to find the coordinator. This is important to reduce group overhead for use cases which involve a large number of topic subscriptions (e.g. mirror maker). Tools will use the protocol type to determine how to parse metadata. For example, when the protocolType is "consumer", the tool can use ConsumerProtocol to parse the member metadata as topic subscriptions and partition assignments. The detailed proposal can be found below. https://cwiki.apache.org/confluence/display/KAFKA/KIP-40%3A+ListGroups+and+DescribeGroup was: Since the new consumer currently has no persistence in Zookeeper (pending outcome of KAFKA-2017), there is no way for administrators to investigate group status including getting the list of members in the group and their partition assignments. We therefore propose to modify GroupMetadataRequest (previously known as ConsumerMetadataRequest) to return group metadata when received by the respective group's coordinator. When received by another broker, the request will be handled as before: by only returning coordinator host and port information. {code} GroupMetadataRequest => GroupId IncludeMetadata GroupId => String IncludeMetadata => Boolean GroupMetadataResponse => ErrorCode Coordinator GroupMetadata ErrorCode => int16 Coordinator => Id Host Port Id => int32 Host => string Port => int32 GroupMetadata => State ProtocolType Generation Protocol Leader Members State => String ProtocolType => String Generation => int32 Protocol => String Leader => String Members => [Member MemberMetadata MemberAssignment] Member => MemberIp ClientId MemberIp => String ClientId => String MemberMetadata => Bytes MemberAssignment => Bytes {code} The request schema includes a flag to indicate whether metadata is needed, which saves clients from having to read all group metadata when they are just trying to find the coordinator. This is important to reduce group overhead for use cases which involve a large number of topic subscriptions (e.g. mirror maker). Tools will use the protocol type to determine how to parse metadata. For example, when the protocolType is "consumer", the tool can use ConsumerProtocol to parse the member metadata as topic subscriptions and partition assignments. > Allow GroupMetadataRequest to return member metadata when received by group > coordinator > --------------------------------------------------------------------------------------- > > Key: KAFKA-2687 > URL: https://issues.apache.org/jira/browse/KAFKA-2687 > Project: Kafka > Issue Type: Improvement > Reporter: Jason Gustafson > Assignee: Jason Gustafson > Priority: Blocker > Fix For: 0.9.0.0 > > > Since the new consumer currently has no persistence in Zookeeper (pending > outcome of KAFKA-2017), there is no way for administrators to investigate > group status including getting the list of members in the group and their > partition assignments. We therefore propose to modify GroupMetadataRequest > (previously known as ConsumerMetadataRequest) to return group metadata when > received by the respective group's coordinator. When received by another > broker, the request will be handled as before: by only returning coordinator > host and port information. > {code} > GroupMetadataRequest => GroupId IncludeMetadata > GroupId => String > IncludeMetadata => Boolean > GroupMetadataResponse => ErrorCode Coordinator GroupMetadata > ErrorCode => int16 > Coordinator => Id Host Port > Id => int32 > Host => string > Port => int32 > GroupMetadata => State ProtocolType Generation Protocol Leader Members > State => String > ProtocolType => String > Generation => int32 > Protocol => String > Leader => String > Members => [Member MemberMetadata MemberAssignment] > Member => MemberIp ClientId > MemberIp => String > ClientId => String > MemberMetadata => Bytes > MemberAssignment => Bytes > {code} > The request schema includes a flag to indicate whether metadata is needed, > which saves clients from having to read all group metadata when they are just > trying to find the coordinator. This is important to reduce group overhead > for use cases which involve a large number of topic subscriptions (e.g. > mirror maker). > Tools will use the protocol type to determine how to parse metadata. For > example, when the protocolType is "consumer", the tool can use > ConsumerProtocol to parse the member metadata as topic subscriptions and > partition assignments. > The detailed proposal can be found below. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-40%3A+ListGroups+and+DescribeGroup -- This message was sent by Atlassian JIRA (v6.3.4#6332)