Hello Jason, Thanks for reply!
About your proposal, in general case it might be helpful. In my case it will not help much - I'm allowing each ConsumerRecord or subset of ConsumerRecords to be processed and ACKed independently and out of HLC process/thread (not to block partition), and then committing largest consecutive ACKed processed offset (+1) since current last committed offset per partition. Kind regards, Stevo Slavic. On Mon, Jul 27, 2015 at 6:52 PM, Jason Gustafson <ja...@confluent.io> wrote: > Hey Stevo, > > I agree that it's a little unintuitive that what you are committing is the > next offset that should be read from and not the one that has already been > read. We're probably constrained in that we already have a consumer which > implements this behavior. Would it help if we added a method on > ConsumerRecords to get the next offset (e.g. nextOffset(partition))? > > Thanks, > Jason > > On Fri, Jul 24, 2015 at 10:11 AM, Stevo Slavić <ssla...@gmail.com> wrote: > > > Hello Apache Kafka community, > > > > Say there is only one topic with single partition and a single message on > > it. > > Result of calling a poll with new consumer will return ConsumerRecord for > > that message and it will have offset of 0. > > > > After processing message, current KafkaConsumer implementation expects > one > > to commit not offset 0 as processed, but to commit offset 1 - next > > offset/position one would like to consume. > > > > Does this sound strange to you as well? > > > > Wondering couldn't this offset+1 handling for next position to read been > > done in one place, in KafkaConsumer implementation or broker or whatever, > > instead of every user of KafkaConsumer having to do it. > > > > Kind regards, > > Stevo Slavic. > > >
