cccs-jc commented on code in PR #8980:
URL: https://github.com/apache/iceberg/pull/8980#discussion_r1402703648
##########
spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java:
##########
@@ -406,6 +413,30 @@ public Offset latestOffset(Offset startOffset, ReadLimit
limit) {
return latestStreamingOffset.equals(startingOffset) ? null :
latestStreamingOffset;
}
+ /**
+ * Get the next snapshot skiping over rewrite and delete snapshots.
+ *
+ * @param curSnapshot the current snapshot
+ * @return the next valid snapshot (not a rewrite or delete snapshot),
returns null if all
+ * remaining snapshots should be skipped.
+ */
+ private Snapshot nextValidSnapshot(Snapshot curSnapshot) {
+ Preconditions.checkArgument(
+ curSnapshot != null, "Sanity check, curSnapshot should not be null");
+
+ Snapshot nextSnapshot = SnapshotUtil.snapshotAfter(table,
curSnapshot.snapshotId());
+ // skip over rewrite and delete snapshots
+ while (!shouldProcess(nextSnapshot)) {
Review Comment:
i think SnapshotUtil.snapshotAfter never returns null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
