rajinisivaram commented on a change in pull request #9382:
URL: https://github.com/apache/kafka/pull/9382#discussion_r534117135
##########
File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala
##########
@@ -426,21 +451,34 @@ abstract class AbstractFetcherThread(name: String,
warn(s"Partition $topicPartition marked as failed")
}
- def addPartitions(initialFetchStates: Map[TopicPartition, OffsetAndEpoch]):
Set[TopicPartition] = {
+ /**
+ * Returns initial partition fetch state based on current state and the
provided `initialFetchState`.
+ * From IBP 2.7 onwards, we can rely on truncation based on diverging data
returned in fetch responses.
+ * For older versions, we can skip the truncation step iff the leader epoch
matches the existing epoch.
+ */
+ private def partitionFetchState(tp: TopicPartition, initialFetchState:
InitialFetchState, currentState: PartitionFetchState): PartitionFetchState = {
+ if (currentState != null && currentState.currentLeaderEpoch ==
initialFetchState.currentLeaderEpoch) {
+ currentState
+ } else if (isTruncationOnFetchSupported && initialFetchState.initOffset >=
0 && initialFetchState.lastFetchedEpoch.nonEmpty &&
+ (currentState == null || currentState.state == Fetching)) {
+ PartitionFetchState(initialFetchState.initOffset, None,
initialFetchState.currentLeaderEpoch,
+ state = Fetching, initialFetchState.lastFetchedEpoch)
Review comment:
@hachikuji Thanks for the review! You mentioned in another comment below
that we could use latestEpoch instead of using the epoch from
InitialFetchOffset. I have removed lastFetchEpoch from InitialFetchOffset and
updated this logic to use latestEpoch. It does look neater now, hope I haven't
missed any cases. I had to update some of the unit tests which rely on the
initial fetch epoch request to use IBP 2.6, but all tests have passed in my
local run. I will kick off a system test run as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
