aokolnychyi commented on a change in pull request #3069:
URL: https://github.com/apache/iceberg/pull/3069#discussion_r712630098
##########
File path: core/src/main/java/org/apache/iceberg/BaseOverwriteFiles.java
##########
@@ -127,8 +131,17 @@ protected void validate(TableMetadata base) {
}
}
+ // validating concurrently added data files is optional and is only needed
to achieve serializable isolation
if (conflictDetectionFilter != null && base.currentSnapshot() != null) {
validateAddedDataFiles(base, startingSnapshotId,
conflictDetectionFilter, caseSensitive);
}
+
+ // validating concurrently added delete files is required whenever we
overwrite specific data files
+ // if we find a new delete file matching one of the data files we are
trying to overwrite,
+ // we must fail this operation as it would undelete rows that were removed
concurrently
+ if (deletedDataFiles.size() > 0) {
+ validateNoNewDeletesForDataFiles(
+ base, startingSnapshotId, conflictDetectionFilter, deletedDataFiles,
caseSensitive);
Review comment:
I have updated the PR to match what I described above.
##########
File path: core/src/main/java/org/apache/iceberg/BaseRewriteFiles.java
##########
@@ -101,7 +101,7 @@ public RewriteFiles validateFromSnapshot(long snapshotId) {
protected void validate(TableMetadata base) {
if (replacedDataFiles.size() > 0) {
// if there are replaced data files, there cannot be any new row-level
deletes for those data files
- validateNoNewDeletesForDataFiles(base, startingSnapshotId,
replacedDataFiles);
+ validateNoNewDeletesForDataFiles(base, startingSnapshotId, null,
replacedDataFiles, true);
Review comment:
Added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
