kazuyukitanimura commented on code in PR #6320:
URL: https://github.com/apache/datafusion-comet/pull/6320#discussion_r4225575662
##########
docs/source/user-guide/latest/compatibility/expressions/_category_template/cast.md:
##########
@@ -161,6 +165,38 @@ Spark's behavior of using Java `BigDecimal.toString()`
semantics, which produces
notation (e.g. a value of 12300 stored as `Decimal(7,-2)` with unscaled value
123 is rendered
as `"1.23E+4"`).
+## Complex Types
+
+Comet casts arrays, structs, and maps by casting each element, field, key, and
value with the
+rules for primitive types, using the table for the same eval mode. A
complex-type cast is
+therefore compatible only when every cast it contains is compatible. For
example, casting
+`ARRAY<INT>` to `ARRAY<BIGINT>` has the same support level as casting `INT` to
`BIGINT`, and a
+struct cast with one unsupported field cast is unsupported. The rules apply at
every level of
+nesting, so casting `ARRAY<STRUCT<a: INT>>` to `ARRAY<STRUCT<a: BIGINT>>` also
has the same
+support level as casting `INT` to `BIGINT`.
+
+| From | To | Support level
|
+| ----------- | ------------- |
---------------------------------------------------------------------------- |
+| `ARRAY<T>` | `ARRAY<U>` | Same as `T` to `U`
|
+| `ARRAY<T>` | `STRING` | Same as `T` to `STRING`
|
+| `STRUCT` | `STRUCT` | Compatible if every field cast is compatible.
Fields are matched by position |
+| `STRUCT` | `STRING` | Compatible if the cast from every field to
`STRING` is compatible |
+| `MAP<K, V>` | `MAP<K2, V2>` | Compatible if `K` to `K2` and `V` to `V2` are
both compatible |
+| `MAP<K, V>` | `STRING` | Unsupported
|
+
+The following casts are exceptions to these rules:
+
+- `ARRAY<DATE>` can only be cast natively to `ARRAY<INT>` or `ARRAY<STRING>`.
Casts to other array
+ types are unsupported.
+- A cast from a `DATE` struct field or map value to a numeric or boolean type
is unsupported. In
+ Legacy mode these casts always return `NULL` (see [Date to Numeric
Types](#date-to-numeric-types)).
+- When `spark.sql.legacy.castComplexTypesToString.enabled=true`, casts from
arrays, structs, and
+ maps to `STRING` are unsupported, because Comet's native cast only produces
Spark's default
+ string format.
+
+In Try mode, a cast between maps fails with an error when a key cannot be
cast, where Spark returns
Review Comment:
One coordination question about this paragraph. #6178 is open and makes this
cast avoid the failing path, but it doesn't touch cast.md. Could we make sure
whichever PR lands second removes or updates this note? Otherwise the page will
keep listing a known issue that's already been fixed. It might also help to say
that this only affects key casts that Spark accepts under TRY_CAST, such as
narrowing BIGINT to INT. Spark already rejects a STRING key cast at analysis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
