manuzhang opened a new pull request, #18061: URL: https://github.com/apache/iceberg/pull/18061
A rollback between dangling-delete discovery and commit can restore data files that still need the candidate deletes. The cleanup rewrite can then rebase onto the restored snapshot and remove valid deletes, making deleted rows visible again. Pin all metadata scans to the planning snapshot and validate that it is still current inside every commit attempt, including retries. Cleanup fails with a `ValidationException` if the snapshot changes; tables without a snapshot remain a no-op. This change is limited to Spark 4.2 and uses the existing core validation API. Adds regression coverage for rollback before commit, rollback during a commit retry, snapshot changes during planning, and empty tables. Related to #13120. Validation: - Spark 4.2 `TestRemoveDanglingDeleteAction`: 24 tests, 23 passed and 1 expected skip. - All 9 concurrency regression cases fail against the original implementation. - Spark 4.2 `spotlessCheck`. --- **AI Disclosure** - Model: GPT-6 - Platform/Tool: Codex - Human Oversight: unreviewed - Prompt Summary: Fix dangling-delete cleanup races in Spark 4.2 using core commit-time snapshot validation, add regression tests, and open a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
