Re: Commit Offset per topic

2013-11-08 Thread Roman Garcia
So, what would be that API for offset commit in 0.8 version? Are there any docs about it? On Thu, Nov 7, 2013 at 10:56 PM, 小宇 wrote: > offset commit API could solve your problem, it's for 0.8 version. > > ---Sent from Boxer | http://getboxer.com > > Thanks Neha! I guess auto-commit it is for no

Re: Commit Offset per topic

2013-11-07 Thread Joel Koshy
It covers any consumer rebalance - i.e., which could be caused by a consumer instance joining/leaving the group, session expiration, new partitions showing up, etc. On Thu, Nov 07, 2013 at 07:56:25PM -0800, Vadim Keylis wrote: > Joel. Just to be clear consumer re-balances you mentioning in the FAQ

Re: Commit Offset per topic

2013-11-07 Thread Vadim Keylis
Joel. Just to be clear consumer re-balances you mentioning in the FAQ refers to consumer restart, am I correct? On Wed, Nov 6, 2013 at 5:51 PM, Joel Koshy wrote: > This question seems to come up often - added this to the FAQ. > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Idon%

Re: Commit Offset per topic

2013-11-07 Thread 小宇
offset commit API could solve your problem, it's for 0.8 version. ---Sent from Boxer | http://getboxer.com Thanks Neha! I guess auto-commit it is for now... On Tue, Nov 5, 2013 at 5:08 AM, Neha Narkhede wrote: > Currently, the only way to achieve that is to use the SimpleConsumer API.

Re: Commit Offset per topic

2013-11-06 Thread Joel Koshy
This question seems to come up often - added this to the FAQ. https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Idon%27twantmyconsumer%27soffsetstobecommittedautomatically.CanImanuallymanagemyconsumer%27soffsets%3F On Tue, Nov 05, 2013 at 04:02:09PM +, Roman Garcia wrote: > Thanks Neh

Re: Commit Offset per topic

2013-11-05 Thread Roman Garcia
Thanks Neha! I guess auto-commit it is for now... On Tue, Nov 5, 2013 at 5:08 AM, Neha Narkhede wrote: > Currently, the only way to achieve that is to use the SimpleConsumer API. > We are considering the feature you mentioned for the 0.9 release - > > https://cwiki.apache.org/confluence/display/

Re: Commit Offset per topic

2013-11-04 Thread Neha Narkhede
Currently, the only way to achieve that is to use the SimpleConsumer API. We are considering the feature you mentioned for the 0.9 release - https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI Thanks, Neha On Mon, Nov 4, 2013 at 9:57 AM, Roman Garcia wrote

Re: Commit Offset per topic

2013-11-04 Thread Roman Garcia
Right Neha, thanks. I was aware of that particular API, but I process topics within separate threads (actors) so I cannot assume every topic/partition is supposed to commit its offset. Instead, I could use a per-stream (topic-partition) commit. Is this use case valid? Should there be a ticket to im

Re: Commit Offset per topic

2013-11-04 Thread Neha Narkhede
The API you are looking for is ConsumerConnector.commitOffsets(). However, not that it will commit offsets for all partitions that the particular consumer connector owns at that time. And the set of partitions owned by a particular consumer connector can change over time. Thanks, Neha On Mon, No

Commit Offset per topic

2013-11-04 Thread Roman Garcia
Hi, I'm still using Kafka 0.7, and was wondering if there's a way I can "manually" commit offsets per Topic/Partition. I was expecting something like KafkaMessageStream#commitOffset() I couldn't find an API for this. If there isn't any, is there a reason? Regards, Roman