I'm using kafka 0.8.2.0
I'm working on a C++ client library and I'm adding consumer offset
management to the client. (https://github.com/bitbouncer/csi-kafka)
I know that the creation of zookeeper "paths" is not handled by kafkabroker
so I've manually created
/consumers/consumer_offset_sample/offsets
in zookeeper using a command line utility.
After this I'm able to get consumer metadata from kafka.
If I commit a consumer offset to an existing topic/partition
("perf-8-new/0") I see the following paths in zookeeper
/consumers/consumer_offset_sample
offsets
offsets/perf-8-new
offsets/perf-8-new/0
owners
I'm surprised as to why the committed values shows up in zookeeper since I
have no bindings to a zookeeper and the "offset.storage" property is a
consumer config. My initial understanding was that they were only written
to __consumer_offsets topic.
Finally, I change the previously committed value manually in zookeeper
/consumers/consumer_offset_sample/offsets/perf-8-new/0 -> 42
then that's what I get back in my get_consumer_offset() as well - so it
seems the zookeeper is involved in the offsets commits/fetches from brokers
point of view.
Is this the intended way or am I doing something wrong?
best regards
svante
