comphead commented on code in PR #6445:
URL: https://github.com/apache/datafusion-comet/pull/6445#discussion_r4146864056
##########
spark/src/test/resources/sql-tests/expressions/cast/cast_string_trim.sql:
##########
@@ -54,7 +54,9 @@ SELECT
cast(concat(pad, '1.5', pad) as float),
cast(concat(pad, '1.5', pad) as double),
cast(concat(pad, '1.5', pad) as decimal(10,2)),
- cast(concat(pad, '2020-01-01', pad) as date)
+ cast(concat(pad, '2020-01-01', pad) as date),
+ cast(concat(pad, '2020-01-01 12:34:56', pad) as timestamp),
Review Comment:
Nit: every column in this fixture is derived from `pad`, and none of the
`INSERT` rows above has a NULL `pad`. Would it be worth adding a row such as
`('i_null', NULL)` so the casts, including the new `timestamp` and
`timestamp_ntz` ones, also see a NULL input?
##########
spark/src/test/scala/org/apache/comet/CometNativeCastSuite.scala:
##########
@@ -1068,6 +1068,29 @@ class CometNativeCastSuite extends CometTestBase with
AdaptiveSparkPlanHelper {
.foreach(castTest(values, _))
}
+ test("cast StringType to DateType - whitespace trim parity") {
+ castTest(trimPaddedValues("2020-01-01").toDF("a"), DataTypes.DateType)
+ }
+
+ test("cast StringType to timestamp types - whitespace trim parity") {
+ withSQLConf(SQLConf.SESSION_LOCAL_TIMEZONE.key -> "UTC") {
+ val values = trimPaddedValues("2020-01-01 12:34:56").toDF("a")
+ Seq(DataTypes.TimestampType,
DataTypes.TimestampNTZType).foreach(castTest(values, _))
+ }
+ }
+
+ test("cast StringType to timestamp types - ANSI rejects a value that trims
to nothing") {
Review Comment:
Nit: would it make sense to move these into a SQL file test with `-- Config:
spark.sql.ansi.enabled=true`, using one `expect_error(CAST_INVALID_INPUT)`
query per value over a Parquet column and a valid-row query as a
native-execution sentinel, like `cast_string_to_date_ansi.sql`? Each `castTest`
call here also re-runs the non-ANSI and `try_cast` checks that the parity test
above already covers for these values, and the loop makes ten calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
