gabeiglio opened a new pull request, #16205: URL: https://github.com/apache/iceberg/pull/16205
This PR is a follow up to https://github.com/apache/iceberg/pull/15193 for Session-Level time travel in Spark. The previous PR works for Spark 4.0 and below since Iceberg still performs snapshot selection at scan time. But, in the new DSV2 that is not possible anymore, therefore, I am proposing a Spark extension that post-planning replaces DSV2 with the correct timestamp if there is one set in the spark conf, this allows us to still have this very useful time-travel conf. This rule, follows the same pattern as ResolveBranch which suffers from the same changes in DSV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
