Commit offsets only work for subscribe(), not assign()
In the new API, the explicit commit offset method call only works for subscribe consumer, not the assign consumer, correct? Best, Siyuan
Re: Commit offsets only work for subscribe(), not assign()
Hey Siyuan, The commit API should work the same regardless whether subscribe() or assign() was used. Does this not appear to be working? Thanks, Jason On Wed, Nov 18, 2015 at 4:40 PM, hsy...@gmail.com wrote: > In the new API, the explicit commit offset method call only works for > subscribe consumer, not the assign consumer, correct? > > Best, > Siyuan >
Re: Commit offsets only work for subscribe(), not assign()
I suppose I should have added one qualification to that. The commit API will not work for a consumer using manual assignment if its groupId is shared with another consumer using automatic assignment (with subscribe()). When a consumer group is active, Kafka only allows commits from members of that group. -Jason On Fri, Nov 20, 2015 at 3:41 PM, Jason Gustafson wrote: > Hey Siyuan, > > The commit API should work the same regardless whether subscribe() or > assign() was used. Does this not appear to be working? > > Thanks, > Jason > > On Wed, Nov 18, 2015 at 4:40 PM, hsy...@gmail.com > wrote: > >> In the new API, the explicit commit offset method call only works for >> subscribe consumer, not the assign consumer, correct? >> >> Best, >> Siyuan >> > >
Re: Commit offsets only work for subscribe(), not assign()
Hey Jason, The test I did is very simple, I was using manual assignment with it's own groupid and clientid. I first started a process to consume data, then produce some data, then kill the process, continue produce more data and start the process again, I didn't see anything from the time the process was killed. Do I have set "auto.offset.reset" as "none"? Thank On Fri, Nov 20, 2015 at 3:56 PM, Jason Gustafson wrote: > I suppose I should have added one qualification to that. The commit API > will not work for a consumer using manual assignment if its groupId is > shared with another consumer using automatic assignment (with subscribe()). > When a consumer group is active, Kafka only allows commits from members of > that group. > > -Jason > > On Fri, Nov 20, 2015 at 3:41 PM, Jason Gustafson > wrote: > > > Hey Siyuan, > > > > The commit API should work the same regardless whether subscribe() or > > assign() was used. Does this not appear to be working? > > > > Thanks, > > Jason > > > > On Wed, Nov 18, 2015 at 4:40 PM, hsy...@gmail.com > > wrote: > > > >> In the new API, the explicit commit offset method call only works for > >> subscribe consumer, not the assign consumer, correct? > >> > >> Best, > >> Siyuan > >> > > > > >
Re: Commit offsets only work for subscribe(), not assign()
Siyuan, When you say "I didn't see anything from the time the process was killed" did you mean your consumer did not get any data after it is restarted, or that the processed messages before killing the consumer does not show up again? Guozhang On Mon, Nov 23, 2015 at 8:13 AM, hsy...@gmail.com wrote: > Hey Jason, > > The test I did is very simple, I was using manual assignment with it's own > groupid and clientid. I first started a process to consume data, then > produce some data, then kill the process, continue produce more data and > start the process again, I didn't see anything from the time the process > was killed. Do I have set "auto.offset.reset" as "none"? Thank > > On Fri, Nov 20, 2015 at 3:56 PM, Jason Gustafson > wrote: > > > I suppose I should have added one qualification to that. The commit API > > will not work for a consumer using manual assignment if its groupId is > > shared with another consumer using automatic assignment (with > subscribe()). > > When a consumer group is active, Kafka only allows commits from members > of > > that group. > > > > -Jason > > > > On Fri, Nov 20, 2015 at 3:41 PM, Jason Gustafson > > wrote: > > > > > Hey Siyuan, > > > > > > The commit API should work the same regardless whether subscribe() or > > > assign() was used. Does this not appear to be working? > > > > > > Thanks, > > > Jason > > > > > > On Wed, Nov 18, 2015 at 4:40 PM, hsy...@gmail.com > > > wrote: > > > > > >> In the new API, the explicit commit offset method call only works for > > >> subscribe consumer, not the assign consumer, correct? > > >> > > >> Best, > > >> Siyuan > > >> > > > > > > > > > -- -- Guozhang
