Coen Damen created KAFKA-6471:
---------------------------------
Summary: seekToEnd and seek give unclear results for Consumer with
read_committed isolation level
Key: KAFKA-6471
URL: https://issues.apache.org/jira/browse/KAFKA-6471
Project: Kafka
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Coen Damen
I am using the transactional KafkaProducer to send messages to a topic. This
works fine. I use a KafkaConsumer with read_committed isolation level and I
have an issue with the seek and seekToEnd methods. According to the
documentation, the seek and seekToEnd methods give me the LSO (Last Stable
Offset). But this is a bit confusing. As it gives me always the same value, the
END of the topic. No matter if the last entry is committed (by the Producer) or
part of an aborted transaction. Example, after I abort the last 5 tries to
insert 20_000 messages, the last 100_000 records should not be read by the
Consumer. But during a seekToEnd it moves to the end of the Topic (including
the 100_000 messages). But the poll() does not return them.
I am looking for a way to retrieve the Last Committed Offset (so the last
successful committed message by the Producer). There seems to be no proper API
method for this. So do I need to roll my own?
Option would be to move back and poll until no more records are retrieved, this
would result in the last committed message. But I would assume that Kafka
provides this method.
We use Kafka 1.0.0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
