If the topic has not seen traffic for a while then Kafka will remove the stored offset. When your consumer reconnects Kafka no longer has the offset so it will reprocess from earliest.
Michael > On 12 Jan 2017, at 11:13, Mahendra Kariya <mahendra.kar...@go-jek.com> wrote: > > Hey All, > > We have a Kafka cluster hosted on Google Cloud. There was some network > issue on the cloud and suddenly, the offset for a particular consumer group > got reset to earliest and all of a sudden the lag was in millions. We > aren't able to figure out what went wrong. Has anybody faced the > same/similar issue? Does anybody have any debugging tips? > > Some relevant info: > > - The auto.offset.reset config for the consumer is set to earliest > - The offsets are stored on Kafka > - Total nodes on cluster: 4 > - Replication factor: 3 > - Partitions: 50
