comphead commented on code in PR #6671:
URL: https://github.com/apache/datafusion-comet/pull/6671#discussion_r4232154735
##########
spark/src/test/scala/org/apache/comet/CometNativeCastSuite.scala:
##########
@@ -2032,6 +2032,35 @@ class CometNativeCastSuite extends CometTestBase with
AdaptiveSparkPlanHelper {
}
}
+ test("cast to a struct with duplicate field names falls back") {
+ val sourceType = StructType(Seq(StructField("p", IntegerType),
StructField("q", IntegerType)))
+ val duplicateTarget =
+ StructType(Seq(StructField("x", IntegerType), StructField("x",
IntegerType)))
+ val uniqueTarget =
+ StructType(Seq(StructField("x", IntegerType), StructField("y",
IntegerType)))
+ val reason = CometCast.duplicateStructFieldNamesReason
+
+ Seq(
+ sourceType -> duplicateTarget,
+ ArrayType(sourceType) -> ArrayType(duplicateTarget),
+ MapType(StringType, sourceType) -> MapType(StringType,
duplicateTarget)).foreach {
+ case (fromType, toType) =>
+ assert(
+ CometCast.isSupported(fromType, toType, None, CometEvalMode.LEGACY)
==
+ Unsupported(Some(reason)))
+ }
+ assert(
+ CometCast
+ .isSupported(sourceType, uniqueTarget, None, CometEvalMode.LEGACY)
+ .isInstanceOf[Compatible])
+
+ val input = spark
+ .range(3)
+ .selectExpr("named_struct('p', CAST(id AS INT), 'q', CAST(id + 10 AS
INT)) AS s")
+
+
checkSparkAnswerAndFallbackReason(input.select(col("s").cast(duplicateTarget)),
reason)
Review Comment:
Could the end-to-end half of this test move to a SQL file fixture under
`sql-tests/expressions/cast/`? `create_map.sql:36` already uses `query
expect_fallback(...)` for the same duplicate-name struct shape, and
`CometSqlFileTestSuite` runs `ExpectFallback` through
`checkSparkAnswerAndFallbackReason`. A fixture could run the three spellings
from #6591 (`CAST(s AS STRUCT<x: INT, x: INT>)` and the `ARRAY<...>` and
`MAP<STRING, ...>` wrappers) as real queries, plus a unique-name `query` that
stays native. Today only the struct spelling runs as a query, and the array and
map spellings are checked only through `isSupported`. I have not run it, but
from reading the harness I expect the SQL `CAST` syntax from the issue to work
as written.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
