dongjoon-hyun commented on code in PR #60043:
URL: https://github.com/apache/spark/pull/60043#discussion_r4460026401
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala:
##########
@@ -215,6 +215,45 @@ class FileIndexSuite extends SharedSparkSession {
}
}
+ test("SPARK-60825: validate datetime partition values regardless of ANSI
mode") {
+ // A TIMESTAMP_NTZ value with a valid date and an invalid time must not
become the start of
+ // that day.
+ val cases = Seq(
+ DateType -> "foo",
+ TimeType(6) -> "foo",
+ TimestampNTZType -> "2020-01-01 25%3A00%3A00")
Review Comment:
Thank you. In 2dab06c465a, I added `TimestampType -> "foo"` to the cases,
and a positive check that reads `a=2020-01-01` as `TIMESTAMP_NTZ` in both ANSI
modes and expects the start of the day.
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala:
##########
@@ -588,19 +588,16 @@ object PartitioningUtils extends SQLConfHelper {
case LongType => JLong.parseLong(value)
case FloatType => JDouble.parseDouble(value).toFloat
case DoubleType => JDouble.parseDouble(value)
- // Cast in ANSI mode to fail on a value that the type cannot hold in any
session, so that
- // `spark.sql.sources.validatePartitionColumns` alone decides between an
error and null.
+ // Cast in ANSI mode, here and in the datetime cases below, to fail on a
value that the type
Review Comment:
Thank you. In 2dab06c465a, I passed `ansiEnabled = true` to the interval
cast as well and changed the comment to cover it. The string-to-interval cast
does not depend on the mode, so the behavior does not change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
