dramaticlly commented on code in PR #17359:
URL: https://github.com/apache/iceberg/pull/17359#discussion_r3654334632
##########
core/src/main/java/org/apache/iceberg/BaseRewriteFiles.java:
##########
@@ -152,5 +152,12 @@ private void validateReplacedAndAddedFiles() {
Preconditions.checkArgument(
deletesDeleteFiles() || !addsDeleteFiles(),
"Delete files to add must be empty because there's no delete file to
be rewritten");
+
+ for (DataFile added : addedDataFiles()) {
+ Preconditions.checkArgument(
+ !replacedDataFiles.contains(added),
+ "Cannot add and delete the same file in the same rewrite: %s",
+ added.location());
+ }
Review Comment:
I am wondering if this is the right place to place the guard, if the goal is
to prevent the same file being added and removed in the same snapshot? I think
both overwrite-files and row-delta can be abused in a way to achieve the same
goal.
On a separate note, knowing this produce incorrect result for the change
tracking detection, sometimes such surgical operation can be helpful for stats
backfill as to rectify a previously written file with incorrect column level
stats, want to see if we shall encourage to use dedicated repair actions such
as proposed in #10784
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
