MaxGekk commented on a change in pull request #25210: [SPARK-28432][SQL] Add
`make_date` function
URL: https://github.com/apache/spark/pull/25210#discussion_r305609274
##########
File path: sql/core/src/test/resources/sql-tests/results/pgSQL/date.sql.out
##########
@@ -508,8 +508,48 @@ struct<Days From 2K:int>
-- !query 47
-DROP TABLE DATE_TBL
+select make_date(2013, 7, 15)
-- !query 47 schema
-struct<>
+struct<make_date(2013, 7, 15):date>
-- !query 47 output
+2013-07-15
+
+
+-- !query 48
+select make_date(-44, 3, 15)
+-- !query 48 schema
+struct<make_date(-44, 3, 15):date>
+-- !query 48 output
+0045-03-15
Review comment:
Actually, the reason for the difference here is how Spark converts internal
type `DateType` to external one `java.sql.Date` (by default) or
`java.sql.LocalDate` (when `spark.sql.datetime.java8API.enabled` is set to
`true`). And how the external type converted to string. For example, to get the
same format as PostgreSQL, need to provide the appropriate formatter.
For `java.sql.Date`:
```Scala
spark.conf.set("spark.sql.datetime.java8API.enabled", false)
val date = spark.sql("select make_date(-44, 3,
15)").first.getAs[java.sql.Date](0)
val sdf = new java.text.SimpleDateFormat("MM-dd-yyyy G")
scala> sdf.format(date)
res18: String = 03-17-0045 BC
```
For Java8 `java.time.LocalDate`:
```Scala
spark.conf.set("spark.sql.datetime.java8API.enabled", true)
val formatter = DateTimeFormatter.ofPattern("MM-dd-yyyy G")
val localDate = spark.sql("select make_date(-44, 3,
15)").first.getAs[LocalDate](0)
localDate.format(formatter)
res16: String = 03-15-0045 BC
```
The difference in days due to difference calendars (Julian in the first
case, and Proleptic Gregorian in the second one).
I see Postgres formats the `-44` year as `44 BC` which is wrong according to
ISO 8601. See https://en.wikipedia.org/wiki/Year_zero , for example:
```
The "basic" format for year 0 ... year 1 BC. ... hence -0001 = 2 BC.
```
I don't think we should implement Postgres bugs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
