Re: Best way to erase consumer state

[Jiangjie Qin]

One thing might worth checking is the offset.storage value in your
consumer config. Is it set to ³kafka² or ³zookeeper² for you consumer? If
the offset is set to kafka, then the offset will be stored in the offset
topic in Kafka instead of in zookeeper. So deleting zookeeper path won¹t
work for those consumers.
Actually the offsets for those consumers cannot be easily removed so far.
KAFKA-1784 is aiming to provide a library to make it easier.

‹Jiangjie (Becket) Qin

On 12/4/14, 10:55 PM, Ben Drees ben.dr...@gmail.com wrote:

Hi,

What is the best way to erase a stopped, multiprocess consumer's state so
that it can be reset (to smallest offsets, for example) when the
consumer
is restarted?  Deleting /consumers/group-name from Zookeeper has had the
desired effect in the past, but I ran into a situation today where some of
the consumer processes jumped to the desired offsets (in this case
largest) on restart but others did not.  I'm not sure whether the issue
was replication lag across the Zookeeper cluster or cached state in Kafka.
I¹m using kafka-python.

Thanks,
Ben

Best way to erase consumer state

[Ben Drees]

Hi,

What is the best way to erase a stopped, multiprocess consumer's state so
that it can be reset (to smallest offsets, for example) when the consumer
is restarted?  Deleting /consumers/group-name from Zookeeper has had the
desired effect in the past, but I ran into a situation today where some of
the consumer processes jumped to the desired offsets (in this case
largest) on restart but others did not.  I'm not sure whether the issue
was replication lag across the Zookeeper cluster or cached state in Kafka.
I’m using kafka-python.

Thanks,
Ben