[jira] [Created] (SPARK-31398) Speed up reading dates in ORC

[Maxim Gekk (Jira)]

Maxim Gekk created SPARK-31398:
----------------------------------

             Summary: Speed up reading dates in ORC
                 Key: SPARK-31398
                 URL: https://issues.apache.org/jira/browse/SPARK-31398
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.0.0
            Reporter: Maxim Gekk


Currently, ORC datasource converts values of DATE type to java.sql.Date and the 
result to days since the epoch in Proleptic Gregorian calendar. ORC datasource 
does such conversion when 
spark.sql.orc.enableVectorizedReader is set to false.

The conversion to java.sql.Date is not necessary because we can use 
DaysWritable which performs rebasing in much more optimal way.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org