This page describes what Kafka stores in Zookeeper:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper
It looks like the info for a particular consumer groupId is stored at:
/consumers//
According to
https://community.cloudera.com/t5/Cloudera-Labs/Kafka-Parcels
There’s no simple command. You’ll need to use either zookeeper-shell.sh or
zkCli.sh or something similar that lets you explore and edit Zookeeper and
do a recursive delete on the group name in the consumers tree. I’m not sure
how Cloudera’s interface differs, however, or if they provide a separate
What is the command to delete group from zookeeper? I dont find
/consumer/ directory? I am using cloudera, is there any place on cloudera
manager where I can delete the group?
Thanks
On Sat, Dec 19, 2015 at 11:47 PM, Todd Palino wrote:
> If what you want to do is reset to smallest, all you ne
If what you want to do is reset to smallest, all you need to do is stop the
consumer, delete the group from Zookeeper, and restart the consumer. It
will automatically create the group again.
You only need to export the offsets first if you later need to reset to
where you were in the partitions.
What is the process for deleting the consumer group from zookeeper? Should
I export offset, delete and then import?
Thanks,
Akhilesh
On Fri, Dec 18, 2015 at 11:32 PM, Todd Palino wrote:
> Yes, that’s right. It’s just work for no real gain :)
>
> -Todd
>
> On Fri, Dec 18, 2015 at 9:38 AM, Marko
Yes, that’s right. It’s just work for no real gain :)
-Todd
On Fri, Dec 18, 2015 at 9:38 AM, Marko Bonaći
wrote:
> Hmm, I guess you're right Tod :)
> Just to confirm, you meant that, while you're changing the exported file it
> might happen that one of the segment files becomes eligible for cle
If you don't like messing w/ ZK directly, another alternative is to
manually seek to offset 0 on all relevant topic-partitions (via
OffsetCommitRequest or your favorite client api) and change the
auto-offset-reset policy on your consumer to earliest/smallest. Bonus is
that this should also work for
Hmm, I guess you're right Tod :)
Just to confirm, you meant that, while you're changing the exported file it
might happen that one of the segment files becomes eligible for cleanup by
retention, which would then make the imported offsets out of range?
Marko Bonaći
Monitoring | Alerting | Anomaly D
That works if you want to set to an arbitrary offset, Marko. However in the
case the OP described, wanting to reset to smallest, it is better to just
delete the consumer group and start the consumer with auto.offset.reset set
to smallest. The reason is that while you can pull the current smallest
o
You can also do this:
1. stop consumers
2. export offsets from ZK
3. make changes to the exported file
4. import offsets to ZK
5. start consumers
e.g.
bin/kafka-run-class.sh kafka.tools.ExportZkOffsets --group group-name
--output-file /tmp/zk-offsets --zkconnect localhost:2181
bin/kafka-run-class.
Hi,
I noticed that a consumer in the new consumer API supports setting the
offset for a partition to beginning. I assume doing so also would update
the offset in Zookeeper eventually.
Cheers,
Jens
On Friday, December 18, 2015, Akhilesh Pathodia
wrote:
> Hi,
>
> I want to reset the kafka offset
The way to reset to smallest is to stop the consumer, delete the consumer
group from Zookeeper, and then restart with the property set to smallest.
Once your consumer has recreated the group and committed offsets, you can
change the auto.offset.reset property back to largest (if that is your
prefer
Hi,
I want to reset the kafka offset in zookeeper so that the consumer will
start reading messages from first offset. I am using flume as a consumer to
kafka. I have set the kafka property kafka.auto.offset.reset to "smallest",
but it does not reset the offset in zookeeper and that's why flume wil
13 matches
Mail list logo
