Re: Cleanup partition offsets that exist for consumer groups but not in broker

2016-11-08 Thread Guozhang Wang
Hello Jeff, Generally speaking ZK's stored offset paths should not be used as the "source-of-truth" to determine which topic-partitions exist in the Kafka cluster, but instead the broker topics path should be treated as the "source-of-truth". More specifically, the common usage pattern would be:

Cleanup partition offsets that exist for consumer groups but not in broker

2016-11-03 Thread Jeff Widman
We hit an error in some custom monitoring code for our Kafka cluster where the root cause was zookeeper was storing for some partition offsets for consumer groups, but those partitions didn't actually exist on the brokers. Apparently in the past, some colleagues needed to reset a stuck cluster cau