Re: Thread safety when committing from multiple threads

2015-09-16 Thread Ashish Shenoy
Hi, Can anyone please shed some light on this ? I couldn't find any pointers online other than this: http://stackoverflow.com/questions/26477585/when-calling-commitoffsets-on-kafka-high-level-consumerconnector-what-value-get Thanks, Ashish On Mon, Sep 14, 2015 at 2:06 PM, Ashish Shenoy

Re: New to Apache Kafka

2015-09-15 Thread Ashish Shenoy
Zookeeper is a distributed coordination service. Kafka uses Zookeeper for various things like leader election, storing consumer-partition offsets etc. More information on each service is available at http://kafka.apache.org/documentation.html and https://zookeeper.apache.org/ I highly recommend re

Thread safety when committing from multiple threads

2015-09-14 Thread Ashish Shenoy
Hi, I have a multi-threaded Kafka consumer. I have auto commit disabled and from each thread, I periodically call commitOffsets() to commit the offsets to Kafka. Is this thread safe ? What is the effect of calling commitOffsets() concurrently from multiple threads ? Should the consumer writer ens