cccs-jc commented on code in PR #8980:
URL: https://github.com/apache/iceberg/pull/8980#discussion_r1402704727
##########
spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java:
##########
@@ -392,8 +392,15 @@ public Offset latestOffset(Offset startOffset, ReadLimit
limit) {
// if everything was OK and we consumed complete snapshot then move to
next snapshot
if (shouldContinueReading) {
+ Snapshot nextValid = nextValidSnapshot(curSnapshot);
+ if (nextValid == null) {
+ // nextValide is null, this implies all the remaining snapshots
should be skipped.
+ shouldContinueReading = false;
+ break;
+ }
+ // we found the next available snapshot, continue from there.
Review Comment:
it makes it explicit we skipped some snapshots and we are continuing from
nextValid
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
