Re: Commit offsets only work for subscribe(), not assign()

2015-11-23 Thread Guozhang Wang
Siyuan, When you say "I didn't see anything from the time the process was killed" did you mean your consumer did not get any data after it is restarted, or that the processed messages before killing the consumer does not show up again? Guozhang On Mon, Nov 23, 2015 at 8:13 AM, hsy...@gmail.com

Re: Commit offsets only work for subscribe(), not assign()

2015-11-23 Thread hsy...@gmail.com
Hey Jason, The test I did is very simple, I was using manual assignment with it's own groupid and clientid. I first started a process to consume data, then produce some data, then kill the process, continue produce more data and start the process again, I didn't see anything from the time the

Re: Commit offsets only work for subscribe(), not assign()

2015-11-20 Thread Jason Gustafson
Hey Siyuan, The commit API should work the same regardless whether subscribe() or assign() was used. Does this not appear to be working? Thanks, Jason On Wed, Nov 18, 2015 at 4:40 PM, hsy...@gmail.com wrote: > In the new API, the explicit commit offset method call only works

Re: Commit offsets only work for subscribe(), not assign()

2015-11-20 Thread Jason Gustafson
I suppose I should have added one qualification to that. The commit API will not work for a consumer using manual assignment if its groupId is shared with another consumer using automatic assignment (with subscribe()). When a consumer group is active, Kafka only allows commits from members of that

Commit offsets only work for subscribe(), not assign()

2015-11-18 Thread hsy...@gmail.com
In the new API, the explicit commit offset method call only works for subscribe consumer, not the assign consumer, correct? Best, Siyuan