It is definitely expected behavior that the new consumer version of kafka-consumer-groups.sh —describe only returns metadata for ‘active’ members. It will print an error message if the consumer group you provide has no active members.
https://github.com/confluentinc/kafka/blob/trunk/core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala#L88 <https://github.com/confluentinc/kafka/blob/trunk/core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala#L88> As to the reason for this I am not certain. My guess is that since offsets are now stored in a partitioned kafka topic (i.e. _offsets) and uses a key that includes the topic+partition+consumerID then it was deemed too problematic or resource intensive to support queries for inactive consumers because the tool would have to read every message in every partition of the _offset topic in order to find a complete list of the offsets for a given inactive consumer. -hans > On May 30, 2017, at 8:09 AM, Jerry George <jerr...@gmail.com> wrote: > > Hi Abhimanyu, > > No, actually waiting for someone with operational experience to reply on > the list. Thank you for bumping the question though :) > > If anyone in the list has experience increasing the retention or if this is > expected behaviour, could kindly suggest an alternative? > > > Regards, > Jerry > > On Sat, May 27, 2017 at 8:18 AM, Abhimanyu Nagrath < > abhimanyunagr...@gmail.com> wrote: > >> Hi Jerry, >> >> I am also facing the same issue. Did you found the solution? >> >> Regards, >> Abhimanyu >> >> On Fri, May 26, 2017 at 7:24 PM, Jerry George <jerr...@gmail.com> wrote: >> >>> Hi >>> >>> I had question about the new consumer APIs. >>> >>> I am having trouble retrieving the offsets once the consumers are >>> *disconnected* when using new consumer v2 API. Following is what I am >>> trying to do, >>> >>> *bin/kafka-consumer-groups.sh -new-consumer --bootstrap-server kafka:9092 >>> --group group --describe* >>> >>> If I query this when the consumers are connected, there is no problem. >>> However, once the consumers are disconnected it says there is no such >>> group, though the offsets are retained in __consumer_offsets. >>> >>> The offset retention policy is default; i.e. 1440 minutes, I believe. >>> >>> Once the consumers are reconnected, I am able to query the offsets once >>> again. >>> >>> Could anyone here please help me understand why this is? >>> >>> Kafka: 0.10.1 >>> Consumer Library: sarama golang library >>> >>> Regards, >>> Jerry >>> >>
