[ https://issues.apache.org/jira/browse/SPARK-31212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098547#comment-17098547 ]
Apache Spark commented on SPARK-31212: -------------------------------------- User 'tianshizz' has created a pull request for this issue: https://github.com/apache/spark/pull/28443 > Failure of casting the '1000-02-29' string to the date type > ----------------------------------------------------------- > > Key: SPARK-31212 > URL: https://issues.apache.org/jira/browse/SPARK-31212 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.5 > Reporter: Maxim Gekk > Priority: Major > > The '1000-02-29' is valid date in the Julian calendar used in Spark 2.4.5 for > dates before 1582-10-15 but casting the string to the date type fails: > {code:scala} > scala> val df = > Seq("1000-02-29").toDF("dateS").select($"dateS".cast("date").as("date")) > df: org.apache.spark.sql.DataFrame = [date: date] > scala> df.show > +----+ > |date| > +----+ > |null| > +----+ > {code} > Creating a dataset from java.sql.Date w/ the same input string works > correctly: > {code:scala} > scala> val df2 = > Seq(java.sql.Date.valueOf("1000-02-29")).toDF("dateS").select($"dateS".as("date")) > df2: org.apache.spark.sql.DataFrame = [date: date] > scala> df2.show > +----------+ > | date| > +----------+ > |1000-02-29| > +----------+ > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org