Hi,
I've been trying out the new consumer and have noticed that i get duplicate
messages when i stop the consumer and then restart (different processes,
same consumer group).
I consume all of the messages on the topic and commit the offsets for each
partition and stop the consumer. On the next run i expect to get 0
messages, however i get a batch of records from each partition - in this
case works out 1020 messages. Run it again and i get the same batch of
records.
My logging shows that i've received messages with offsets lower than were
previously committed.
committed:{damian_test_one-2=137669}
committed:{damian_test_one-0=139161}
committed:{damian_test_one-1=137663}
min offsets received: {damian_test_one-0=138824, damian_test_one-1=137321,
damian_test_one-2=137331}
I've debugged the initial fetch requests for offsets and the offsets match
up with what has been committed. Is this expected behaviour? Something to
do with batching of compression of message sets?
TIA,
Damian
