agali-09 commented on code in PR #20741:
URL: https://github.com/apache/druid/pull/20741#discussion_r4383015091
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -2459,6 +2459,56 @@ private void killTasksInGroup(TaskGroup taskGroup,
String reasonFormat, Object..
}
}
+ /**
+ * Resets the offsets of the given partitions so that they are re-read from
metadata storage, and kills the actively
Review Comment:
Could you share what happens if there's no committed offset in metadata
storage? Checked about kafka's
[reset](https://www.conduktor.io/kafka/consumer-auto-offsets-reset-behavior)
behavior and if policy is latest, wouldn't we still have the gap?
An example:
```
Original task started at offset: 100
Its first publish fails: 100 -> 150 remains unpublished
Committed metadata: no offset for this partition yet
Kafka’s latest offset: 300
Configured starting policy: latest
Replacement reader starts from 300.
```
Would it help to preserve the failed group’s original starting offset as the
fallback when no committed offset exists?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
