andygrove opened a new pull request, #3698:
URL: https://github.com/apache/datafusion-comet/pull/3698
## Which issue does this PR close?
Closes #3320.
## Rationale for this change
Two ParquetFilterSuite tests were disabled for native_datafusion mode due to
a `CometNativeExec should not be executed directly without a serialized plan`
crash. This crash was fixed by #3511 which added a dedicated
`doExecuteColumnar()` to `CometNativeScanExec`, so these tests no longer need
to be skipped.
## What changes are included in this PR?
Regenerated `dev/diffs/3.5.8.diff` to remove `IgnoreCometNativeDataFusion`
tags from two ParquetFilterSuite tests:
- "SPARK-31026: Parquet predicate pushdown for fields having dots in the
names"
- "Filters should be pushed down for Parquet readers at row group level"
Both tests have their `stripSparkFilter` assertions already guarded by `if
(!isCometEnabled || isCometScanOnly)`, so they pass safely when Comet native
execution is active.
The other two tests from the issue were already resolved:
- "filter pushdown - StringPredicate" is tagged with
`IgnoreCometNativeScan("cannot be pushed down")` for a separate valid reason
- "SPARK-41017" already has Comet scan type pattern matching in the existing
diff
## How are these changes tested?
By running the Spark SQL tests with Comet enabled in native_datafusion mode,
which exercises the re-enabled ParquetFilterSuite tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
