[
https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-20682:
----------------------------
Labels: releasenotes (was: )
> Add new ORCFileFormat based on Apache ORC
> -----------------------------------------
>
> Key: SPARK-20682
> URL: https://issues.apache.org/jira/browse/SPARK-20682
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.4.1, 1.5.2, 1.6.3, 2.1.1, 2.2.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Labels: releasenotes
> Fix For: 2.3.0
>
>
> Since SPARK-2883, Apache Spark supports Apache ORC inside `sql/hive` module
> with Hive dependency. This issue aims to add a new and faster ORC data source
> inside `sql/core` and to replace the old ORC data source eventually. In this
> issue, the latest Apache ORC 1.4.0 (released yesterday) is used.
> There are four key benefits.
> - Speed: Use both Spark `ColumnarBatch` and ORC `RowBatch` together. This is
> faster than the current implementation in Spark.
> - Stability: Apache ORC 1.4.0 has many fixes and we can depend on ORC
> community more.
> - Usability: User can use `ORC` data sources without hive module, i.e,
> `-Phive`.
> - Maintainability: Reduce the Hive dependency and can remove old legacy code
> later.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
