[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r429117869 ## File path: docs/sql-ref-datetime-pattern.md ## @@ -26,44 +26,128 @@ There are several common scenarios for datetime usage in Spark: - Datetime functions related to convert `StringType` to/from `DateType` or `TimestampType`. For example, `unix_timestamp`, `date_format`, `to_unix_timestamp`, `from_unixtime`, `to_date`, `to_timestamp`, `from_utc_timestamp`, `to_utc_timestamp`, etc. -Spark uses pattern letters in the following table for date and timestamp parsing and formatting: - -|Symbol|Meaning|Presentation|Examples| -|--|---||| -|**G**|era|text|AD; Anno Domini; A| -|**y**|year|year|2020; 20| -|**D**|day-of-year|number|189| -|**M/L**|month-of-year|number/text|7; 07; Jul; July; J| -|**d**|day-of-month|number|28| -|**Q/q**|quarter-of-year|number/text|3; 03; Q3; 3rd quarter| -|**Y**|week-based-year|year|1996; 96| -|**w**|week-of-week-based-year|number|27| -|**W**|week-of-month|number|4| -|**E**|day-of-week|text|Tue; Tuesday; T| -|**u**|localized day-of-week|number/text|2; 02; Tue; Tuesday; T| -|**F**|week-of-month|number|3| -|**a**|am-pm-of-day|text|PM| -|**h**|clock-hour-of-am-pm (1-12)|number|12| -|**K**|hour-of-am-pm (0-11)|number|0| -|**k**|clock-hour-of-day (1-24)|number|0| -|**H**|hour-of-day (0-23)|number|0| -|**m**|minute-of-hour|number|30| -|**s**|second-of-minute|number|55| -|**S**|fraction-of-second|fraction|978| -|**V**|time-zone ID|zone-id|America/Los_Angeles; Z; -08:30| -|**z**|time-zone name|zone-name|Pacific Standard Time; PST| -|**O**|localized zone-offset|offset-O|GMT+8; GMT+08:00; UTC-08:00;| -|**X**|zone-offset 'Z' for zero|offset-X|Z; -08; -0830; -08:30; -083015; -08:30:15;| -|**x**|zone-offset|offset-x|+; -08; -0830; -08:30; -083015; -08:30:15;| -|**Z**|zone-offset|offset-Z|+; -0800; -08:00;| -|**'**|escape for text|delimiter| | -|**''**|single quote|literal|'| -|**[**|optional section start| | | -|**]**|optional section end| | | - -The count of pattern letters determines the format. - -- Text: The text style is determined based on the number of pattern letters used. Less than 4 pattern letters will use the short form. Exactly 4 pattern letters will use the full form. Exactly 5 pattern letters will use the narrow form. Six or more letters will fail. +The following tables define how the pattern letters be used for date and timestamp parsing and formatting in Spark. + +## Date Fields + +Pattern letters to output a date: + +|Pattern|Count|Meaning|Presentation|Examples| +|---|---|---|---|---| +|**G**|1|era|text|AD| +|**GG**|2|era|text|AD| +|**GGG**|3|era|text|AD| Review comment: I tried this once, but some of the patterns are not able to be simplified because of different outputs. So I choose to stick to same layout. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r429110475 ## File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out ## @@ -750,3 +750,164 @@ select to_timestamp("2019 10:10:10", " hh:mm:ss") struct -- !query output 2019-01-01 10:10:10 + + +-- !query +select date_format(date '2020-05-23', 'G') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'G' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(date '2020-05-23', 'M') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'M' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(date '2020-05-23', 'L') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'L' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(timestamp '2020-05-23', 'E') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'E' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(timestamp '2020-05-23', 'u') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'u' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format('2020-05-23', 'Q') +-- !query schema +struct<> +-- !query output +java.lang.IllegalArgumentException +Too many pattern letters: Q + + +-- !query +select date_format('2020-05-23', 'q') +-- !query schema +struct<> +-- !query output +java.lang.IllegalArgumentException +Too many pattern letters: q + + +-- !query +select to_timestamp('2019-10-06 A', '-MM-dd G') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize '-MM-dd G' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select to_timestamp('22 05 2020 Friday', 'dd MM EE') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'dd MM EE' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select to_timestamp('22 05 2020 Friday', 'dd MM E') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'dd MM E' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select unix_timestamp('22 05 2020 Friday', 'dd MM E') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrad
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r429110475 ## File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out ## @@ -750,3 +750,164 @@ select to_timestamp("2019 10:10:10", " hh:mm:ss") struct -- !query output 2019-01-01 10:10:10 + + +-- !query +select date_format(date '2020-05-23', 'G') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'G' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(date '2020-05-23', 'M') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'M' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(date '2020-05-23', 'L') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'L' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(timestamp '2020-05-23', 'E') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'E' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format(timestamp '2020-05-23', 'u') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'u' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select date_format('2020-05-23', 'Q') +-- !query schema +struct<> +-- !query output +java.lang.IllegalArgumentException +Too many pattern letters: Q + + +-- !query +select date_format('2020-05-23', 'q') +-- !query schema +struct<> +-- !query output +java.lang.IllegalArgumentException +Too many pattern letters: q + + +-- !query +select to_timestamp('2019-10-06 A', '-MM-dd G') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize '-MM-dd G' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select to_timestamp('22 05 2020 Friday', 'dd MM EE') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'dd MM EE' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select to_timestamp('22 05 2020 Friday', 'dd MM E') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrading of Spark 3.0: Fail to recognize 'dd MM E' pattern in the new parser. 1) You can set spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before Spark 3.0. 2) You can form a valid datetime pattern with the guide from https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html + + +-- !query +select unix_timestamp('22 05 2020 Friday', 'dd MM E') +-- !query schema +struct<> +-- !query output +org.apache.spark.SparkUpgradeException +You may get a different result due to the upgrad
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r429093201 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -33,6 +34,8 @@ sealed trait DateFormatter extends Serializable { def format(days: Int): String def format(date: Date): String def format(localDate: LocalDate): String + + def initialize(): Unit = {} Review comment: Got it. SGTM. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428667113 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( extends DateFormatter with DateTimeFormatterHelper { @transient - private lazy val formatter = getOrCreateFormatter(pattern, locale) + private lazy val formatter = { Review comment: Hmm, this one and the others are transient, so the lazy keyword is required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428667113 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( extends DateFormatter with DateTimeFormatterHelper { @transient - private lazy val formatter = getOrCreateFormatter(pattern, locale) + private lazy val formatter = { Review comment: Hmm, this one and the other are transient, so the lazy keyword is required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428607225 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime-corrected.sql ## @@ -0,0 +1,2 @@ +--SET spark.sql.legacy.timeParserPolicy=CORRECTED Review comment: I've just come to understand what it means, I will rm this case. ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TimestampFormatter.scala ## @@ -61,8 +64,23 @@ class Iso8601TimestampFormatter( needVarLengthSecondFraction: Boolean) extends TimestampFormatter with DateTimeFormatterHelper { @transient - protected lazy val formatter: DateTimeFormatter = -getOrCreateFormatter(pattern, locale, needVarLengthSecondFraction) + protected lazy val formatter: DateTimeFormatter = { +try { Review comment: OK This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428606030 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( extends DateFormatter with DateTimeFormatterHelper { @transient - private lazy val formatter = getOrCreateFormatter(pattern, locale) + private lazy val formatter = { Review comment: Shall we remove this lazy to let it fail fast in the parse phase? @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428604890 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -77,14 +96,14 @@ trait LegacyDateFormatter extends DateFormatter { class LegacyFastDateFormatter(pattern: String, locale: Locale) extends LegacyDateFormatter { @transient - private lazy val fdf = FastDateFormat.getInstance(pattern, locale) Review comment: needed https://github.com/apache/spark/pull/28592/files#diff-7f589e01d3e5e5ea284c1622527d4984R52 here, to fail if invaild This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428000557 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ## @@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12", "-MM-dd'T'HH:mm:ss''"); -- select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); -- head but as single quote + +-- Unsupported narrow text style +select date_format(date '2020-05-23', 'G'); Review comment: For parsing and formatting, throw SparkUpgradeExeception to tell users how to restore the previous behavior or update their pattern via our online doc? How about the current approach This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428000557 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ## @@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12", "-MM-dd'T'HH:mm:ss''"); -- select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); -- head but as single quote + +-- Unsupported narrow text style +select date_format(date '2020-05-23', 'G'); Review comment: For parsing and formatting, throw SparkUpgradeExeception to tell users how to restore or update their pattern via our online doc? How about the current approach This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r427960736 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ## @@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12", "-MM-dd'T'HH:mm:ss''"); -- select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); -- head but as single quote + +-- Unsupported narrow text style +select date_format(date '2020-05-23', 'G'); Review comment: I just realize that returning NULL is also silent data change... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r427956371 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ## @@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12", "-MM-dd'T'HH:mm:ss''"); -- select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); -- head but as single quote + +-- Unsupported narrow text style +select date_format(date '2020-05-23', 'G'); Review comment: This question may beyond the scope of this PR. So far, using an invalid pattern for parsing always returns `NULL`, which is also compatible with 2.4. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org