Re: Starting from beginning of topic using Java API (kafka 0.7)

2014-03-10 Thread Neha Narkhede
Currently, the only way to restart from the beginning of the queue is by deleting the previous checkpoint for the group. The reason is that in real production deployments, a consumer application can go through process restarts or other interruptions but the expectation is that it can start reading

Re: Starting from beginning of topic using Java API (kafka 0.7)

2014-03-10 Thread Adam Phelps
Other than constantly using a new group id (and making a mess of zookeeper) or deleting the info for the group from zookeeper, is there any way to start from the beginning of the queue? It looks like this can be done from the underlying scala code, but I can't find anything in the Java API. - Ada

Re: Starting from beginning of topic using Java API (kafka 0.7)

2014-03-07 Thread Neha Narkhede
>From reading around it looked like setting "autooffset.reset" = "smallest" would do this, however I'm not actually seeing that behavior. The reason is that a consumer actually consults this config only if it doesn't find a previous offset stored for it's group in zookeeper. So, it will respect th

Starting from beginning of topic using Java API (kafka 0.7)

2014-03-07 Thread Adam Phelps
I've been trying to write a test consumer in Java for a new use of our Kafka cluster (currently used solely with Storm), however this use needs to always start from the earliest offset in the topic. From reading around it looked like setting "autooffset.reset" = "smallest" would do this, however I