amogh-jahagirdar opened a new pull request, #9102: URL: https://github.com/apache/iceberg/pull/9102
Fixes #7635 . The Iceberg Spark integration has logic to surface to the Spark Optimizer whether a metadata delete (deleting a data file) should be performed or not. When there are branches, this logic reads the main branch's metadata and surfaces to the optimizer that metadata deletes are possible even though they are not. This leads to failure at the time of the commit validation since there are residual rows in the file that don't match the deletion criteria. This fixes the problem by making sure the scan gets built correctly. Also this change also makes sure the metadata delete tests actually exercise branches. Previously the tests weren't exercising metadata deletes on branches because all the branch tests were using Avro and Avro doesn't support metadata deletes so those tests would get skipped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
