andygrove opened a new issue, #3309: URL: https://github.com/apache/datafusion-comet/issues/3309
## Summary With the introduction of `native_datafusion` in auto scan mode (PR #3307), several test helpers that check the scan implementation config are broken when running in `auto` mode. The root cause is that helpers like `usingDataSourceExec` check if the config string is literally `native_datafusion` or `native_iceberg_compat`, but in `auto` mode the config reads as `"auto"` even though it resolves to `native_datafusion` at plan time. ## Failing Tests (in auto mode) - **"schema evolution"** (`ParquetReadSuite.scala:1256`) — expects `SparkException` but `native_datafusion` handles type widening gracefully - **"row group skipping doesn't overflow when reading into larger type"** (`ParquetReadSuite.scala:1523`) — same issue ## Proposed Fix Since `native_comet` is deprecated and the default path is now DataSource-based (via auto), invert the check: - Rename `usingDataSourceExec` → `usingLegacyNativeCometScan` which returns `true` only when config is explicitly `native_comet` - Flip all ~40 call sites accordingly - Update `usingDataSourceExecWithIncompatTypes` similarly - Fix the explicit `SCAN_NATIVE_DATAFUSION` check in the schema evolution test This avoids needing to enumerate all non-legacy modes and is forward-compatible with future scan implementations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
