This is similar to a problem I am also grappling with. We store the processed offset for each partition in state store. And after restarts we see that sometimes the start offset that Kafka Streams uses is a few thousands to a couple million behind per partition. To compound it, this is not repeatable.
Regards Sab On Tue, Jul 18, 2017 at 9:00 PM, Justin Maltat <justin.mal...@gmail.com> wrote: > Hi > > On a 3 brokers cluster when one of the broker come back after a restart > group rebalancing happens on our 2 consumers which make them restart to > consume from an old offset which is not the earliest. Looking at the > consumer offsets through kafka tools when running commits look good but on > rebalance offset changes > > If we cut our consumer it restarts at the right offset > > Do you have any idea of what could be happening? > > Regards > > Justin Maltat > > > > > >
