liupoyi-1031 opened a new pull request, #6046:
URL: https://github.com/apache/datafusion-comet/pull/6046
## Which issue does this PR close?
Closes #3886.
## Rationale for this change
Comet validates the temporary row-index column type while deciding whether
to fall back from a native scan. An invalid type therefore leaks a raw
`RuntimeException` before Spark's Parquet reader can wrap it as
`FAILED_READ_FILE`.
## What changes are included in this PR?
- Detect the temporary row-index column by name without validating its type
in `CometScanRule`.
- Fall back to Spark so its Parquet reader preserves the expected error
handling.
- Re-enable the affected Spark 4.0.4 and 4.1.3 SQL tests.
## How are these changes tested?
- `./mvnw -ntp package -DskipTests`
- `./mvnw -ntp test -Dtest=none
-Dsuites="org.apache.comet.rules.CometScanRuleSuite" -Dscalastyle.skip=true`
- 5 tests passed.
- Spark 4.0.4 `ParquetRowIndexSuite` tests matching `invalid row index
column type`
- DataSource V1 passed.
- DataSource V2 passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
