[
https://issues.apache.org/jira/browse/SPARK-44677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44677:
----------------------------------
Parent: (was: SPARK-44111)
Issue Type: Task (was: Sub-task)
> Drop legacy Hive-based ORC file format
> --------------------------------------
>
> Key: SPARK-44677
> URL: https://issues.apache.org/jira/browse/SPARK-44677
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Cheng Pan
> Priority: Major
>
> Currently, Spark allows to use spark.sql.orc.impl=native/hive to switch the
> ORC FileFormat implementation.
> SPARK-23456(2.4) switched the default value of spark.sql.orc.impl from "hive"
> to "native". and prepared to drop the "hive" implementation in the future.
> > ... eventually, Apache Spark will drop old Hive-based ORC code.
> The native implementation works well during the whole Spark 3.x period, so
> it's a good time to consider dropping the "hive" one in Spark 4.0.
> Also, we should take care about the backward-compatibility during change.
> > BTW, IIRC, there was a different at Hive ORC CHAR implementation before.
> > So, we couldn't remove it for backward-compatibility issues. Since Spark
> > implements many CHAR features, we need to re-verify that {{native}}
> > implementation has all legacy Hive-based ORC features
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
