Sean McKibben created KAFKA-4047:
------------------------------------
Summary: Return more useful information from ConsumerGroupCommand
for consumer groups that are rebalancing or manually assigned
Key: KAFKA-4047
URL: https://issues.apache.org/jira/browse/KAFKA-4047
Project: Kafka
Issue Type: Improvement
Components: consumer, tools
Affects Versions: 0.10.0.1, 0.10.0.0, 0.9.0.1
Reporter: Sean McKibben
Priority: Minor
For consumers which are manually assigned topic partitions, or for offsets
which are otherwise manually committed to brokers using the new consumer API,
the existing ConsumerGroupCommand tools are operationally insufficient.
There are many use cases in production operation when it is important to be
able to easily retrieve the stored offsets and calculated lag of a consumer
group, even if the group is rebalancing or partition assignment information is
indeterminate. Often these cases involve a misbehaving or crashed client
application, and having the ConsumerGroupCommand return {{Consumer group
`myGroupID` does not exist or is rebalancing.}} instead of the information it
does know is not very helpful. Additionally, when manual offset commits are
used, or the automatic consumer group subscription management is not used, the
same message is returned by the tool. In these cases, the offsets are actually
stored and available even though the broker/coordinator doesn't have
information about partition ownership.
Previously, this was a non-issue as a ZK client could be leveraged relatively
easily to get important information like the last stored offset positions, or
consumer-offset-checker would return more information in its responses. With
the new consumer, however, the process for getting offset and lag is not as
straightforward, and tools to provide this information at the operational level
are necessary as part of a Kafka installation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
