Github user ckadner commented on the issue:
https://github.com/apache/spark/pull/13652
I think this is a good "best effort" fix for the round-trip conversion
```int -> java.sql.Date -> int```.
It may also be worth testing the round-trip conversion ```java.sql.Date ->
int -> java.sql.Date```:
```scala
test("fromJavaDate toJavaDate in all timezones") {
val debug = true
val years = 1900 to 2020
val dates : Seq[String] = for {
year <- years
month <- 1 to 12
day <- 1 to 31
} yield new Date(year-1900, month-1, day).toString
def testDateIsEqual(date: String): Boolean = {
val d1 = Date.valueOf(date)
val d2 = toJavaDate(fromJavaDate(d1))
d2 === d1
}
def testStringEquals(date: String): Boolean = {
val d1 = Date.valueOf(date)
val d2 = toJavaDate(fromJavaDate(d1))
d2.toString === d1.toString
}
// make sure all converted Dates have a matching String representation
(ignores time component)
for (tz <- DateTimeTestUtils.ALL_TIMEZONES) {
DateTimeTestUtils.withDefaultTimeZone(tz) {
assert(dates.filterNot(testStringEquals _).isEmpty)
}
}
// find Dates that don't convert to an equal Date (non-matching time
component)
val nonEqualDatesWithTZ = for {
tz <- DateTimeTestUtils.ALL_TIMEZONES
badDate <- DateTimeTestUtils.withDefaultTimeZone(tz) {
dates.filterNot(testDateIsEqual _) }
} yield (badDate, tz)
// scalastyle:off println
if (debug) {
println(s"${nonEqualDatesWithTZ.size} incorrect conversions
${years.start}..${years.end}")
nonEqualDatesWithTZ.foreach { case (d, t) => println(s"${d} in
${t.getID}") }
}
// scalastyle:on println
}
```
Output when ```debug == true```
```
135 incorrect conversions 1900..2020:
1939-11-19 in Africa/Algiers
1900-01-01 in Africa/Ceuta
1943-03-21 in Africa/Windhoek
1922-01-01 in America/Bahia_Banderas
1922-01-01 in America/Cancun
1996-10-27 in America/Chihuahua
1997-10-26 in America/Chihuahua
1981-03-29 in America/Danmarkshavn
1982-03-28 in America/Danmarkshavn
1983-03-27 in America/Danmarkshavn
1984-03-25 in America/Danmarkshavn
1985-03-31 in America/Danmarkshavn
1986-03-30 in America/Danmarkshavn
1987-03-29 in America/Danmarkshavn
1988-03-27 in America/Danmarkshavn
1989-03-26 in America/Danmarkshavn
1990-03-25 in America/Danmarkshavn
1991-03-31 in America/Danmarkshavn
1992-03-29 in America/Danmarkshavn
1993-03-28 in America/Danmarkshavn
1994-03-27 in America/Danmarkshavn
1995-03-26 in America/Danmarkshavn
1965-10-31 in America/Dawson
1969-10-26 in America/Indiana/Tell_City
1970-10-25 in America/Indiana/Tell_City
1965-10-31 in America/Inuvik
1999-10-31 in America/Iqaluit
1945-09-30 in America/Juneau
1969-10-26 in America/Juneau
1970-10-25 in America/Juneau
1971-10-31 in America/Juneau
1972-10-29 in America/Juneau
1973-10-28 in America/Juneau
1974-10-27 in America/Juneau
1975-10-26 in America/Juneau
1976-10-31 in America/Juneau
1977-10-30 in America/Juneau
1978-10-29 in America/Juneau
1979-10-28 in America/Juneau
1981-10-25 in America/Juneau
1982-10-31 in America/Juneau
1922-01-01 in America/Mexico_City
1996-10-27 in America/Ojinaga
1997-10-26 in America/Ojinaga
1945-09-30 in America/Pangnirtung
1980-10-26 in America/Pangnirtung
1981-10-25 in America/Pangnirtung
1982-10-31 in America/Pangnirtung
1983-10-30 in America/Pangnirtung
1984-10-28 in America/Pangnirtung
1985-10-27 in America/Pangnirtung
1986-10-26 in America/Pangnirtung
1987-10-25 in America/Pangnirtung
1988-10-30 in America/Pangnirtung
1989-10-29 in America/Pangnirtung
1990-10-28 in America/Pangnirtung
1991-10-27 in America/Pangnirtung
1992-10-25 in America/Pangnirtung
1993-10-31 in America/Pangnirtung
1994-10-30 in America/Pangnirtung
1999-10-31 in America/Pangnirtung
1950-04-16 in America/Santarem
1945-09-30 in America/Sitka
1969-10-26 in America/Sitka
1970-10-25 in America/Sitka
1971-10-31 in America/Sitka
1972-10-29 in America/Sitka
1973-10-28 in America/Sitka
1974-10-27 in America/Sitka
1975-10-26 in America/Sitka
1976-10-31 in America/Sitka
1977-10-30 in America/Sitka
1978-10-29 in America/Sitka
1979-10-28 in America/Sitka
1980-10-26 in America/Sitka
1981-10-25 in America/Sitka
1982-10-31 in America/Sitka
1965-10-31 in America/Whitehorse
1900-01-01 in Antarctica/Casey
1900-01-01 in Antarctica/Davis
2009-10-18 in Antarctica/Davis
2011-10-28 in Antarctica/Davis
1900-01-01 in Antarctica/DumontDUrville
1900-01-01 in Antarctica/Mawson
1900-01-01 in Antarctica/Syowa
1900-01-01 in Antarctica/Vostok
2001-09-29 in Asia/Choibalsan
2002-09-28 in Asia/Choibalsan
2003-09-27 in Asia/Choibalsan
2004-09-25 in Asia/Choibalsan
2005-09-24 in Asia/Choibalsan
2006-09-30 in Asia/Choibalsan
1916-11-01 in Atlantic/Azores
1916-11-01 in Atlantic/Madeira
1917-10-21 in Atlantic/Reykjavik
1918-11-16 in Atlantic/Reykjavik
1919-11-16 in Atlantic/Reykjavik
1921-06-23 in Atlantic/Reykjavik
1965-10-31 in Canada/Yukon
1900-01-01 in Europe/Brussels
1900-01-01 in Europe/Gibraltar
1916-10-01 in Europe/Kaliningrad
1916-10-01 in Europe/Kaliningrad
1990-07-01 in Europe/Kiev
1990-07-01 in Europe/Kiev
1922-10-08 in Europe/Luxembourg
1924-10-05 in Europe/Luxembourg
1925-10-04 in Europe/Luxembourg
1926-10-03 in Europe/Luxembourg
1927-10-02 in Europe/Luxembourg
1928-10-07 in Europe/Luxembourg
1900-01-01 in Europe/Madrid
1992-09-27 in Europe/Minsk
1911-03-11 in Europe/Paris
1990-07-01 in Europe/Uzhgorod
1990-07-01 in Europe/Uzhgorod
1917-10-21 in Iceland
1918-11-16 in Iceland
1919-11-16 in Iceland
1921-06-23 in Iceland
1900-01-01 in Indian/Kerguelen
1922-01-01 in Mexico/General
1900-01-01 in Pacific/Apia
1950-01-02 in Pacific/Apia
2011-09-25 in Pacific/Apia
1900-01-01 in Pacific/Enderbury
1979-10-02 in Pacific/Enderbury
1900-01-01 in Pacific/Fakaofo
1901-01-02 in Pacific/Fakaofo
1900-01-01 in Pacific/Kiritimati
1979-10-02 in Pacific/Kiritimati
1911-03-11 in ECT
1900-01-01 in MIT
1950-01-02 in MIT
2011-09-25 in MIT
```
The extra debug stuff should be removed of course.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
