I can confirm that in 0.10.2.1 I get offset information for disconnected
consumers. The note in the output is a bit misleading because it also works
with non-Java clients as long as they implement the new consumer. For example
below is what I get when using the blizzard/node-rdkafka client which wraps
librdkafka.
[bin] $ ./kafka-consumer-groups --bootstrap-server localhost:9092 --describe
--group node-red-rdkafka-groupid
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
CONSUMER-ID HOST
CLIENT-ID
mytopic 0 4 4 0
-0b713303-e69e-47d9-baf7-70df49ff48a6 /10.20.0.1
-hans
> On May 30, 2017, at 10:57 AM, Jerry George <[email protected]> wrote:
>
> Thank you Hans and Vahid.
>
> That was definitely of great help. Much appreciated!
>
> Regards,
> Jerry
>
> On Tue, May 30, 2017 at 1:53 PM, Vahid S Hashemian <
> [email protected]> wrote:
>
>> Hi Jerry,
>>
>> The behavior you are expecting is implemented in 0.10.2 through KIP-88 (
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> 88%3A+OffsetFetch+Protocol+Update
>> ) and KAFKA-3853 (https://issues.apache.org/jira/browse/KAFKA-3853).
>> Starting from this release when you query a consumer group (new consumer
>> only) you'll see the stored offset corresponding to a topic partition even
>> if there is no active consumer consuming from it.
>>
>> I hope this helps.
>> --Vahid
>>
>>
>>
>>
>> From: Jerry George <[email protected]>
>> To: [email protected]
>> Date: 05/26/2017 06:55 AM
>> Subject: Trouble with querying offsets when using new consumer
>> groups API
>>
>>
>>
>> Hi
>>
>> I had question about the new consumer APIs.
>>
>> I am having trouble retrieving the offsets once the consumers are
>> *disconnected* when using new consumer v2 API. Following is what I am
>> trying to do,
>>
>> *bin/kafka-consumer-groups.sh -new-consumer --bootstrap-server kafka:9092
>> --group group --describe*
>>
>> If I query this when the consumers are connected, there is no problem.
>> However, once the consumers are disconnected it says there is no such
>> group, though the offsets are retained in __consumer_offsets.
>>
>> The offset retention policy is default; i.e. 1440 minutes, I believe.
>>
>> Once the consumers are reconnected, I am able to query the offsets once
>> again.
>>
>> Could anyone here please help me understand why this is?
>>
>> Kafka: 0.10.1
>> Consumer Library: sarama golang library
>>
>> Regards,
>> Jerry
>>
>>
>>
>>
>>
