malinjawi commented on PR #12218:
URL: https://github.com/apache/gluten/pull/12218#issuecomment-4646755326
> Could you add a DV-enabled CDF regression test to confirm the behavior?
Something like:
>
> ```
> CREATE TABLE t (id INT, name STRING) USING delta
> TBLPROPERTIES ('delta.enableChangeDataFeed'='true',
'delta.enableDeletionVectors'='true');
> INSERT INTO t VALUES (1,'v1'),(2,'v2'),(3,'v3');
> DELETE FROM t WHERE id = 2;
> SELECT id, name, _change_type FROM table_changes('t', 0);
> ```
>
> compared against vanilla Spark (checkAnswer)
Thanks @felipepessoto good point! I agree this is worth covering explicitly.
I’ll add a focused DV-enabled CDF regression with
`delta.enableChangeDataFeed=true` and `delta.enableDeletionVectors=true`,
comparing `table_changes(...)` against vanilla Spark via the existing
`runAndCompare` / `checkAnswer` path.
One important nuance: this PR is not intended to claim full native DV
support. DV-backed Delta scan execution is still a separate active area in
Gluten/Velox and may continue to fall back where native DV handling is not
available. The goal of the regression here is to make sure enabling DV does not
break the CDF planning path or result correctness.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
