[ https://issues.apache.org/jira/browse/SPARK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kousuke Saruta updated SPARK-36081: ----------------------------------- Description: sql-migration-guide.md mentions about the behavior of cast like as follows. {code} In Spark 3.0, when casting string value to integral types(tinyint, smallint, int and bigint), datetime types(date, timestamp and interval) and boolean type, the leading and trailing whitespaces (<= ASCII 32) will be trimmed before converted to these type values, for example, `cast(' 1\t' as int)` results `1`, `cast(' 1\t' as boolean)` results `true`, `cast('2019-10-10\t as date)` results the date value `2019-10-10`. In Spark version 2.4 and below, when casting string to integrals and booleans, it does not trim the whitespaces from both ends; the foregoing results is `null`, while to datetimes, only the trailing spaces (= ASCII 32) are removed. {code} In fact, select cast('2019-10-10\b' as date); returns 2019-10-10 in Spark 3.0.0. But after 3.0.1, the query returns NULL and this behavior doesn't comply with the specification. was: sql-migration-guide.md mentions about the behavior of cast like as follows. {code} In Spark 3.0, when casting string value to integral types(tinyint, smallint, int and bigint), datetime types(date, timestamp and interval) and boolean type, the leading and trailing whitespaces (<= ASCII 32) will be trimmed before converted to these type values, for example, `cast(' 1\t' as int)` results `1`, `cast(' 1\t' as boolean)` results `true`, `cast('2019-10-10\t as date)` results the date value `2019-10-10`. In Spark version 2.4 and below, when casting string to integrals and booleans, it does not trim the whitespaces from both ends; the foregoing results is `null`, while to datetimes, only the trailing spaces (= ASCII 32) are removed. {code} In fact, select cast('2019-10-10\b' as date); returns 2019-10-10 with Spark 3.0.0. But after 3.0.1, the query returns NULL and this behavior doesn't comply with the specification. The root cause seems to be > The implementation of cast doesn't comply with its specification > ---------------------------------------------------------------- > > Key: SPARK-36081 > URL: https://issues.apache.org/jira/browse/SPARK-36081 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.3, 3.1.2, 3.2.0, 3.3.0 > Reporter: Kousuke Saruta > Assignee: Kousuke Saruta > Priority: Major > > sql-migration-guide.md mentions about the behavior of cast like as follows. > {code} > In Spark 3.0, when casting string value to integral types(tinyint, smallint, > int and bigint), datetime types(date, timestamp and interval) and boolean > type, the leading and trailing whitespaces (<= ASCII 32) will be trimmed > before converted to these type values, for example, `cast(' 1\t' as int)` > results `1`, `cast(' 1\t' as boolean)` results `true`, `cast('2019-10-10\t as > date)` results the date value `2019-10-10`. In Spark version 2.4 and below, > when casting string to integrals and booleans, it does not trim the > whitespaces from both ends; the foregoing results is `null`, while to > datetimes, only the trailing spaces (= ASCII 32) are removed. > {code} > In fact, select cast('2019-10-10\b' as date); returns 2019-10-10 in Spark > 3.0.0. > But after 3.0.1, the query returns NULL and this behavior doesn't comply with > the specification. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org