Thanks, Achanta - you are right, I should have used 'get' instead of 'ls' !And
this also answers my other question about setting the offset manually - I guess
I can just as well do
'set /consumers/elastic_search_group/offsets/my_log_topic/0 someValue'
thanks!
Marina
From: Achanta Vamsi Subhash <[email protected]>
To: "[email protected]" <[email protected]>; Marina
<[email protected]>
Sent: Saturday, May 23, 2015 1:10 PM
Subject: Re: How to verify /update offsets in 0.8.2.1 ?
Should you be doing:
get /consumers/elastic_search_group/offsets/my_log_topic/0
On Sat, May 23, 2015 at 1:42 AM, Marina <[email protected]> wrote:
Hi,
I would like to inspect current offsets for my topic/partitions from a command
line, and update them when needed.
I can use the kafka.tools.ConsumerOffsetChecker to view the offsets as
following:
./bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
elastic_search_group --zookeeper
localhost:2181 --topic my_log_topic
Group Topic Pid Offset logSize Lag Owner
elastic_search_group my_log_topic 0 34 35 1 none
[to make it more legible in the post, the values are: Offset=34; logSize=35,
Lag=1]
However, i cannot update the offsets using this tool (to my knowledge).
So, according to posts here:
http://stackoverflow.com/questions/14935755/how-to-get-data-from-old-offset-point-in-kafka
I could try to modify the offset directly in Zookeeper, using the ZkUtils
.updatePersistentPath(....)
However, when I try to first see what is the value in Zookeeper for this
topic/partition, I see the following [no value for offset at all]:
Using zookeeper shell: ./bin/zookeeper-shell.sh localhost:2181 :
ls /consumers/elastic_search_group/offsets/my_log_topic/0
[]
I expected to see the same number [34 , or 35 ??] as what ConsumerOffsetChecker
showed - but this is not the case.
Why? Is it because in 0.8.2.1 offsets are already stored in Kafka, and not
Zookeeper?
(I am using low-level simpleConsumer API)?
And in that case - how can I see what are the real offset values and modify
them?
thanks!
Marina
--
RegardsVamsi Subhash
