sunchao commented on code in PR #5161:
URL: https://github.com/apache/datafusion-comet/pull/5161#discussion_r4104921866
##########
spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala:
##########
@@ -1117,6 +1117,10 @@ case class CometScanTypeChecker() extends
DataTypeSupport with CometTypeShim {
false
case s: StructType if s.fields.isEmpty =>
false
+ case _: YearMonthIntervalType | _: DayTimeIntervalType =>
Review Comment:
[P2] Update the existing interval fallback fixtures alongside these support
gates. `expressions/aggregate/collect_list.sql:292,295` still requires interval
scans to fall back, and `expressions/math/abs.sql:62` still requires a null
year-month literal to fall back. These queries now execute through Comet, so
`CometSqlFileTestSuite` fails instead of accepting the newly supported
behavior. This breaks every Spark-version expression job and the required CI
gate. Change those records to normal `query` assertions, update their obsolete
comments, and rerun both fixtures.
Evidence: Exact-head run 34703461345 reproduces this on Spark 3.4, 3.5, 4.0,
4.1 and 4.2. Each job reports three failures: `abs.sql` and both dictionary
configurations of `collect_list.sql`. Spark 4.1 job 103583366054 reports
`Expected fallback reason 'Unsupported data type YearMonthIntervalType' but no
fallback reasons were found` for `SELECT abs(CAST(NULL AS INTERVAL YEAR TO
MONTH))`, and the analogous missing `Unsupported ym of type
YearMonthIntervalType` reason for `collect_list(ym)`. Log:
https://github.com/apache/datafusion-comet/actions/runs/34703461345/job/103583366054
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
