It is an know issue. You can increase the retention time for stored offsets via configs thought.
There is already an open PR to fix this issue: https://issues.apache.org/jira/browse/KAFKA-4682 -Matthias On 6/1/18 2:00 AM, Dinesh Subramanian wrote: > Hi M. Manna, > > Am planning to store outside kafka.. will this be a solution ? > > > *Thanks & Regards,* > > *Dinesh S* > > On Fri, Jun 1, 2018 at 2:05 PM, M. Manna <manme...@gmail.com> wrote: > >> This can happen for two reasons: >> >> 1) Your offsets are expired and removed. So your consumers don't know where >> to start from - earliest means "Start from the beginning" >> 2) You are actually starting as part of a totally new consumer group - in >> which case it's as designed too - start from the beginning. >> >> I would check your offset retention policy by size/time - and tune that if >> necessary. >> >> On 1 June 2018 at 09:03, Dinesh Subramanian <dsubraman...@apptivo.co.in> >> wrote: >> >>> Hi, >>> >>> Facing duplication in below scenario >>> >>> Last commit is happened in 3 days back in the consumer, after that no >>> messages produced in the topic. so no commits.. >>> so after 3 days am stopping and restarting the consumer.. this time i >>> faced duplication issue in the consumer as i have this consumer >>> property "*auto.offset.reset >>> = earliest*", It is consumed again from the beginning.. any helps will be >>> appreciated. >>> >>> *Thanks & Regards,* >>> >>> *Dinesh S* >>> >> >
signature.asc
Description: OpenPGP digital signature
