Gengliang Wang created SPARK-38407:
--------------------------------------
Summary: ANSI Cast: loosen the limitation of casting non-null
complex types
Key: SPARK-38407
URL: https://issues.apache.org/jira/browse/SPARK-38407
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.3.0, 3.2.2
Reporter: Gengliang Wang
Assignee: Gengliang Wang
When ANSI mode is off, `ArrayType(DoubleType, containsNull = false)` can't cast
as `ArrayType(IntegerType, containsNull = false)` since there can be overflow
thus result in null results and breaks the non-null constraint.
When ANSI mode is on, currently Spark SQL has the same behavior. However, this
is not correct since the non-null constraint won't be break. Spark SQL can just
execute the cast and throw runtime error on overflow, just like casting
DoubleType as IntegerType.
This applies to MapType and StructType as well.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
