Maxim Gekk created SPARK-31398: ---------------------------------- Summary: Speed up reading dates in ORC Key: SPARK-31398 URL: https://issues.apache.org/jira/browse/SPARK-31398 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.0.0 Reporter: Maxim Gekk
Currently, ORC datasource converts values of DATE type to java.sql.Date and the result to days since the epoch in Proleptic Gregorian calendar. ORC datasource does such conversion when spark.sql.orc.enableVectorizedReader is set to false. The conversion to java.sql.Date is not necessary because we can use DaysWritable which performs rebasing in much more optimal way. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org