Cheng Pan created SPARK-44677:
---------------------------------
Summary: Drop legacy Hive-based ORC file format
Key: SPARK-44677
URL: https://issues.apache.org/jira/browse/SPARK-44677
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.0.0
Reporter: Cheng Pan
Currently, Spark allows to use spark.sql.orc.impl=native/hive to switch the ORC
FileFormat implementation.
SPARK-23456(2.4) switched the default value of spark.sql.orc.impl from "hive"
to "native". and prepared to drop the "hive" implementation in the future.
> ... eventually, Apache Spark will drop old Hive-based ORC code.
The native implementation works well during the whole Spark 3.x period, so it's
a good time to consider dropping the "hive" one in Spark 4.0.
Also, we should take care about the backward-compatibility during change.
> BTW, IIRC, there was a different at Hive ORC CHAR implementation before. So,
> we couldn't remove it for backward-compatibility issues. Since Spark
> implements many CHAR features, we need to re-verify that {{native}}
> implementation has all legacy Hive-based ORC features
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
