amogh-jahagirdar commented on code in PR #17359:
URL: https://github.com/apache/iceberg/pull/17359#discussion_r3685951368
##########
core/src/main/java/org/apache/iceberg/BaseRewriteFiles.java:
##########
@@ -152,5 +152,12 @@ private void validateReplacedAndAddedFiles() {
Preconditions.checkArgument(
deletesDeleteFiles() || !addsDeleteFiles(),
"Delete files to add must be empty because there's no delete file to
be rewritten");
+
+ for (DataFile added : addedDataFiles()) {
+ Preconditions.checkArgument(
+ !replacedDataFiles.contains(added),
+ "Cannot add and delete the same file in the same rewrite: %s",
+ added.location());
+ }
Review Comment:
Yeah I was looking at this in the interim, I think you're right that
Overwrite and Row Delta technically allow this and we should block them too.
>On a separate note, knowing this produce incorrect result for the change
tracking detection, sometimes such surgical operation can be helpful for stats
backfill as to rectify a previously written file with incorrect column level
stats, want to see if we shall encourage to use dedicated repair actions such
as proposed in https://github.com/apache/iceberg/pull/10784
Yup! I think rewriting manifests (just like Repair would do) is the right
way. After some thought, I don't think updating these operations to infer that
add + remove = existing is the right thing to do because I think these APIs
have largely been designed around the fact that added = "something new", and
removed = "something that existed and we need to remove".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
