CTTY commented on code in PR #17764:
URL: https://github.com/apache/iceberg/pull/17764#discussion_r3876578629
##########
core/src/main/java/org/apache/iceberg/DeleteFileIndex.java:
##########
@@ -485,6 +486,24 @@ Builder ignoreResiduals() {
return this;
}
+ /**
+ * Indexes deletion vectors as file-scoped position deletes, allowing a
data file to have more
+ * than one DV.
+ *
+ * <p>A data file has at most one live DV in a single snapshot, so by
default {@link #build()}
+ * rejects a second DV for the same data file. That invariant does not
hold across snapshots,
+ * and commit validation indexes delete manifests from every snapshot in
the validation window,
+ * so it requires an index that accepts them.
+ *
+ * <p>The resulting index only answers whether deletes exist for a data
file, not which deletes
+ * apply to it: {@code forDataFile} no longer suppresses partition-scoped
position deletes when
+ * a DV is present. Scans must use the default so that corrupt live
metadata is still detected.
+ */
+ Builder indexDVsAsPositionDeletes() {
Review Comment:
> I think the right way to fix this is in
MergingSnapshotProducer#validateNoNewDeletesForDataFiles to build a
DeleteFileIndex per concurrently added snapshot. For every data file we're
validating we go through each delete index and do the validation
+1. I think in `validateNoDeletesForDataFiles` we can have:
```
for (DataFile dataFile : dataFiles) {
for (DeleteFileIndex deletes : deleteIndexes) {
DeleteFile[] deleteFiles =
deletes.forDataFile(startingSequenceNumber, dataFile);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
