wirybeaver opened a new issue, #12628: URL: https://github.com/apache/gluten/issues/12628
### Description Add native Iceberg format-version 3 deletion-vector support to the Velox backend. Today Gluten falls back when reading Puffin deletion vectors and row-level DELETE, UPDATE, and MERGE operations do not use the enhanced native Iceberg writer. Velox already contains the required deletion-vector writer support. Proposed scope: - Pass Puffin deletion-vector metadata through Substrait and map it to Velox `FileContent::kDeletionVector`. - Validate deletion vectors per delete file while continuing to fall back for equality deletes and malformed metadata. - Offload Iceberg V3 `WriteDeltaExec` operations for DELETE, UPDATE, and MERGE when enhanced native writes are enabled. - Preserve Iceberg's existing driver-side commit, validation, conflict detection, and abort handling. - Seed native writes with existing deletion vectors so repeated mutations union deleted positions and replace only the prior active DV. - Keep Spark 3.3 and Iceberg V2 behavior unchanged. - Cover Spark 3.4, 3.5, 4.0, and 4.1 compatibility, native writer behavior, partitioned tables, and split-boundary reads. ### Gluten version main branch This issue description was written with assistance from generative AI tooling. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
