Jan-Ole Sasse created SPARK-44420: ------------------------------------- Summary: CheckOverflowInTableInsert does not handle cases where the cast was moved by the optimizer Key: SPARK-44420 URL: https://issues.apache.org/jira/browse/SPARK-44420 Project: Spark Issue Type: Bug Components: Optimizer Affects Versions: 3.4.0 Reporter: Jan-Ole Sasse
CheckOverflowInTableInsert is supposed to rewrite error messages that come from overflow exceptions on casts that are implicitly inserted for INSERTs to match the output schema. This does not work in all cases, for example when the cast is moved by the optimizer {code:java} CREATE TABLE source(value BIGINT) USING parquet INSERT INTO source values (${Long.MaxValue.toString}) CREATE TABLE target(value INT) USING parquet INSERT INTO target SELECT CASE WHEN value < 0 THEN 0 ELSE value END FROM source{code} In this case, the expression will no nothing because it only becomes effective if the direct child is a cast ([matching code|https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L2186]) -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org