andygrove opened a new pull request, #6320:
URL: https://github.com/apache/datafusion-comet/pull/6320
## Which issue does this PR close?
Closes #2743.
## Rationale for this change
The cast compatibility page only covers casts between primitive types. #2760
was closed until the
complex-type casts had test coverage, which the #4248 work has since added.
## What changes are included in this PR?
A new Complex Types section in the cast compatibility template, which is
copied into every Spark
version's page:
- The rule: arrays, structs and maps are cast element by element, field by
field, and key and
value by key and value, so a complex-type cast is compatible only when
every cast it contains
is compatible.
- A table covering array to array, array to string, struct to struct (fields
matched by position),
struct to string, map to map, and map to string, which has no native path.
- The exceptions: `ARRAY<DATE>`, `DATE` struct fields and map values cast to
numeric or boolean
types, and `spark.sql.legacy.castComplexTypesToString.enabled`.
- A known-issue note for a Try-mode map cast whose key cast fails (#5995).
- Pointers to the new section from the page intro and from Date to Numeric
Types.
The `DATE` struct field and map value bullet describes the behavior after
#6319, so this should
merge after that PR.
## How are these changes tested?
Documentation only. I checked each claim against `CometCast.isSupported`,
and against Spark with a
probe over every container shape (struct field, map value, array element,
array of struct) for
each primitive target type in Legacy mode. That probe is what turned up
#6316. The #5995 failure
reproduces on `main`. The page renders through `GenerateDocs` for Spark 4.1,
and prettier passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
