I believe this is expected behavior. If there are no subscriptions to a new topic, and therefor no partition assignments, and definitely no committed offsets, then lag is an undefined concept. When the consumers subscribe to this new topic they may chose to start at the beginning or end of the commit log so the lag cannot be predicted in advance.
-hans > On Feb 4, 2018, at 11:51 AM, Wouter Bancken <wouter.banc...@aca-it.be> wrote: > > Can anyone clarify if this is a bug in Kafka or the expected behavior? > > Best regards, > Wouter > > > On 30 January 2018 at 21:04, Wouter Bancken <wouter.banc...@aca-it.be> > wrote: > >> Hi, >> >> I'm trying to write an external tool to monitor consumer lag on Apache >> Kafka. >> >> For this purpose, I'm using the kafka-consumer-groups tool to fetch the >> consumer offsets. >> >> When using this tool, partition assignments seem to be unavailable >> temporarily during the creation of a new topic even if the consumer group >> has no subscription on this new topic. This seems to match the >> documentation >> <https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Client-side+Assignment+Proposal> >> saying *"Topic metadata changes which have no impact on subscriptions >> cause resync"*. >> >> However, when this occurs I'd expect the state of the consumer to be >> "PreparingRebalance" or "AwaitingSync" but it is simply "Stable". >> >> Is this a bug in the tooling or is there a different way to obtain the >> correct offsets for a consumer group during a rebalance? >> >> I'm using Kafka 10.2.1 but I haven't found any related issues in recent >> changelogs. >> Best regards, >> Wouter >>
