[
https://issues.apache.org/jira/browse/HIVE-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kokila N updated HIVE-28979:
----------------------------
Description:
The ORC library has been upgraded to version *2.1.2* in Hive.
Key changes in ORC 2.x:
* The *default compression codec* has changed from *ZLIB* to {*}ZSTD{*}.
→ [Related ORC PR: apache/orc#1733|https://github.com/apache/orc/pull/1733]
However, in Hive, after upgrading from ORC *1.9.4* to {*}2.1.2{*}, we have
explicitly retained *ZLIB* as the default compression codec, since we have not
yet performed performance benchmarking to justify switching to ZSTD.
Another notable change in ORC 2.x:
* *ZStandard (ZSTD)* compression has moved from using *aircompressor* to
{*}zstd-jni{*}.
** There is a fallback option to revert to {*}aircompressor{*}, which has been
*enabled in Hive* for compatibility.
→ [Related ORC PR: apache/orc#988|https://github.com/apache/orc/pull/988]
> ORC with ZSTD as default
> ------------------------
>
> Key: HIVE-28979
> URL: https://issues.apache.org/jira/browse/HIVE-28979
> Project: Hive
> Issue Type: Task
> Reporter: Kokila N
> Assignee: Kokila N
> Priority: Major
>
> The ORC library has been upgraded to version *2.1.2* in Hive.
> Key changes in ORC 2.x:
> * The *default compression codec* has changed from *ZLIB* to {*}ZSTD{*}.
> → [Related ORC PR: apache/orc#1733|https://github.com/apache/orc/pull/1733]
> However, in Hive, after upgrading from ORC *1.9.4* to {*}2.1.2{*}, we have
> explicitly retained *ZLIB* as the default compression codec, since we have
> not yet performed performance benchmarking to justify switching to ZSTD.
> Another notable change in ORC 2.x:
> * *ZStandard (ZSTD)* compression has moved from using *aircompressor* to
> {*}zstd-jni{*}.
> ** There is a fallback option to revert to {*}aircompressor{*}, which has
> been *enabled in Hive* for compatibility.
> → [Related ORC PR: apache/orc#988|https://github.com/apache/orc/pull/988]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
