In Kafka, the consumers are responsible for maintaining state information
(offset) on what has been consumed (refer from kafka design
page).high-level consumer api will store its consumption state in
zookeeper, while simple consumer shoud deal with these things itself.
My doubt is what happened wh
getOffsetsBefore sends an RPC call to the Kafka brokers to find out the
earliest/latest offset for that topic, partition. In your example, it will
get you the latest offset at the time of the request.
Thanks,
Neha
On Thu, May 30, 2013 at 11:31 PM, shangan chen wrote:
> In Kafka, the consumers a
You mean kafka broker will maintain earliest/latest offset for each
partition of each topic ? Do these offsets information have anything to do
with consumers or just broker's own information? I mean, simple consumer
won't get the offset where it stopped, but the offset of brokers latest
offset.
An
I mean, simple consumer
won't get the offset where it stopped, but the offset of brokers latest
offset.
Correct. Also each consumer in Kafka reads data independently from the
broker.
Thanks,
Neha
On May 31, 2013 7:44 PM, "shangan chen" wrote:
> You mean kafka broker will maintain earliest/lates
