MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603976469 > So, does this happen only in vectorized reader @dongjoon-hyun Correct, the regular reader uses `DateTimeUtils.fromJavaDate`: https://github.com/apache/spark/blob/300ec1a74cb14867c22e616e657566d510426331/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcDeserializer.scala#L111 And the PRs #27807 and #27980 introduced date rebasing in `fromJavaDate()`. Also, I fixed ORC writer, and added a round trip test for both vectorized and non-vectorized readers.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
