Hi Hans, Thanks for the response!
However, I get this result for all topics, not just for the newly created topic. Situation sketch: 1. I have a consumer group 'X' subscribed to topics 'A' and 'B' with partition assignments and lag information. Consumer group 'X' is "Stable". 2a. Topic 'C' is (being) created. 2b. During this creation, I do not have a partition assignment for consumer group 'X' for topics 'A' and 'B' but the consumer group is still "Stable". 3. A second later: I have a partition assignment for consumer group 'X' for topics 'A' and 'B' again and the consumer group is still "Stable". I expected the state of consumer group 'X' during step 2b to be "PreparingRebalance" or "AwaitingSync". Best regards, Wouter On 4 February 2018 at 21:25, Hans Jespersen <h...@confluent.io> wrote: > I believe this is expected behavior. > > If there are no subscriptions to a new topic, and therefor no partition > assignments, and definitely no committed offsets, then lag is an undefined > concept. When the consumers subscribe to this new topic they may chose to > start at the beginning or end of the commit log so the lag cannot be > predicted in advance. > > -hans > > > On Feb 4, 2018, at 11:51 AM, Wouter Bancken <wouter.banc...@aca-it.be> > wrote: > > > > Can anyone clarify if this is a bug in Kafka or the expected behavior? > > > > Best regards, > > Wouter > > > > > > On 30 January 2018 at 21:04, Wouter Bancken <wouter.banc...@aca-it.be> > > wrote: > > > >> Hi, > >> > >> I'm trying to write an external tool to monitor consumer lag on Apache > >> Kafka. > >> > >> For this purpose, I'm using the kafka-consumer-groups tool to fetch the > >> consumer offsets. > >> > >> When using this tool, partition assignments seem to be unavailable > >> temporarily during the creation of a new topic even if the consumer > group > >> has no subscription on this new topic. This seems to match the > >> documentation > >> <https://cwiki.apache.org/confluence/display/KAFKA/ > Kafka+Client-side+Assignment+Proposal> > >> saying *"Topic metadata changes which have no impact on subscriptions > >> cause resync"*. > >> > >> However, when this occurs I'd expect the state of the consumer to be > >> "PreparingRebalance" or "AwaitingSync" but it is simply "Stable". > >> > >> Is this a bug in the tooling or is there a different way to obtain the > >> correct offsets for a consumer group during a rebalance? > >> > >> I'm using Kafka 10.2.1 but I haven't found any related issues in recent > >> changelogs. > >> Best regards, > >> Wouter > >> >
