It is an know issue. You can increase the retention time for stored
offsets via configs thought.
There is already an open PR to fix this issue:
https://issues.apache.org/jira/browse/KAFKA-4682
-Matthias
On 6/1/18 2:00 AM, Dinesh Subramanian wrote:
> Hi M. Manna,
>
> Am planning to store
Hi M. Manna,
Am planning to store outside kafka.. will this be a solution ?
*Thanks & Regards,*
*Dinesh S*
On Fri, Jun 1, 2018 at 2:05 PM, M. Manna wrote:
> This can happen for two reasons:
>
> 1) Your offsets are expired and removed. So your consumers don't know where
> to start from -
This can happen for two reasons:
1) Your offsets are expired and removed. So your consumers don't know where
to start from - earliest means "Start from the beginning"
2) You are actually starting as part of a totally new consumer group - in
which case it's as designed too - start from the
You should just recommit the same offsets sooner than every 24 hours (or
whatever your commit topic retention period is set to). The expiry of offsets
is based on the timestamp of the commits.
-hans
> On Jun 1, 2018, at 1:03 AM, Dinesh Subramanian
> wrote:
>
> Hi,
>
> Facing duplication in
Hi,
Facing duplication in below scenario
Last commit is happened in 3 days back in the consumer, after that no
messages produced in the topic. so no commits..
so after 3 days am stopping and restarting the consumer.. this time i
faced duplication issue in the consumer as i have this consumer
