ashvina commented on code in PR #627:
URL: https://github.com/apache/incubator-xtable/pull/627#discussion_r1929496620
##########
xtable-api/src/main/java/org/apache/xtable/model/TableChange.java:
##########
@@ -18,22 +18,39 @@
package org.apache.xtable.model;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
import lombok.Builder;
import lombok.Value;
import org.apache.xtable.model.storage.DataFilesDiff;
+import org.apache.xtable.model.storage.InternalDeletionVector;
/**
* Captures the changes in a single commit/instant from the source table.
*
* @since 0.1
*/
@Value
-@Builder(toBuilder = true)
+@Builder(toBuilder = true, builderClassName = "Builder")
public class TableChange {
// Change in files at the specified instant
DataFilesDiff filesDiff;
+ // A commit can add deletion vectors when some records are deleted. New
deletion vectors can be
+ // added even if no new data files are added. However, as deletion vectors
are always associated
+ // with a data file, they are implicitly removed when a corresponding data
file is removed.
+ List<InternalDeletionVector> deletionVectorsAdded;
Review Comment:
Yes. In Delta Lake format, the `DeletionVectorDescriptor` (metadata of
delete vectors) is embedded inside `AddFile` (data file representation).
The integration test for delete vector reuses
`TestSparkDeltaTable.getAllActiveFiles` which reads all files in a snapshot
`deltaLog.snapshot().allFiles()`. The data file objects returned by this method
contain the deletion vector metadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
