UrsSchoenenbergerNu commented on issue #15846: URL: https://github.com/apache/iceberg/issues/15846#issuecomment-4169012028
Inspecting the history of the code there, is it possible that https://github.com/apache/iceberg/pull/10526 introduced a bug here? The flow previous to this PR did not use the upstream (Writer's) idea of the checkpoint ID belonging to a `WriteResult` and basically had its own "pending tracking" between two snapshotState() calls in the committer, so conceptually, it treated files that were part of checkpoint N for the writer, as part of checkpoint N+1 for the committer. The idea of "records arriving after their barrier, dropping previous state" therefore did not exist. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
