zinking commented on code in PR #9454: URL: https://github.com/apache/iceberg/pull/9454#discussion_r1462652137
########## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ########## @@ -289,13 +321,38 @@ private void invalidateFilteredCache() { cleanUncommitted(SnapshotProducer.EMPTY_SET); } + private void recordPartitionMinDataSequenceNumber(ManifestFile manifest) { Review Comment: > I had implemented a version of this as a Spark job. I had thought at the time to integrate it as RewriteAction, something like a finally action after all the compact jobs are run. It may be better as its Spark job and more scalable. #6581 @aokolnychyi should we proceed with it? seems solving the issue too, it's just there are too many such kind of actions, I'm really not a big fan of many separate spark actions. I remember there's another one that removed pos delete etc. can we just put these in rewrite so that the end users can be assured that after rewrite everything returns to normal? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org