[
https://issues.apache.org/jira/browse/HIVE-30198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30198:
----------------------------------
Description:
An Iceberg v3 table STORED AS ORC with a variant column can be created, but
every write fails when the data file is
closed:
{code}
java.lang.UnsupportedOperationException: Variant is not supported
at org.apache.iceberg.orc.OrcMetrics.statsColumns(OrcMetrics.java:377)
at org.apache.iceberg.orc.OrcMetrics.buildOrcMetrics(OrcMetrics.java:139)
at org.apache.iceberg.orc.OrcFileAppender.metrics(OrcFileAppender.java:109)
at org.apache.iceberg.io.DataWriter.close(DataWriter.java:92)
at
org.apache.iceberg.mr.hive.writer.HiveIcebergDefaultWriter.close(HiveIcebergDefaultWriter.java:91)
{code}
Cause: Iceberg's OrcMetrics.StatsColumnsVisitor has no variant override, and
the base OrcSchemaVisitor.variant
throws. Iceberg 1.12 has ORC variant readers and writers
(GenericOrcWriters.VariantWriter); only the metrics fail.
Iceberg main has the same gap.
Proposal: fix OrcMetrics upstream in Iceberg. Until Hive's Iceberg version has
the fix, reject a variant column
in an Iceberg ORC table at CREATE/ALTER TABLE with a clear error, instead of
failing every INSERT.
> Iceberg: writing a VARIANT column to an ORC table fails (OrcMetrics)
> --------------------------------------------------------------------
>
> Key: HIVE-30198
> URL: https://issues.apache.org/jira/browse/HIVE-30198
> Project: Hive
> Issue Type: Sub-task
> Reporter: Denys Kuzmenko
> Priority: Major
>
> An Iceberg v3 table STORED AS ORC with a variant column can be created, but
> every write fails when the data file is
> closed:
> {code}
> java.lang.UnsupportedOperationException: Variant is not supported
> at org.apache.iceberg.orc.OrcMetrics.statsColumns(OrcMetrics.java:377)
> at org.apache.iceberg.orc.OrcMetrics.buildOrcMetrics(OrcMetrics.java:139)
> at org.apache.iceberg.orc.OrcFileAppender.metrics(OrcFileAppender.java:109)
> at org.apache.iceberg.io.DataWriter.close(DataWriter.java:92)
> at
> org.apache.iceberg.mr.hive.writer.HiveIcebergDefaultWriter.close(HiveIcebergDefaultWriter.java:91)
> {code}
> Cause: Iceberg's OrcMetrics.StatsColumnsVisitor has no variant override, and
> the base OrcSchemaVisitor.variant
> throws. Iceberg 1.12 has ORC variant readers and writers
> (GenericOrcWriters.VariantWriter); only the metrics fail.
> Iceberg main has the same gap.
> Proposal: fix OrcMetrics upstream in Iceberg. Until Hive's Iceberg version
> has the fix, reject a variant column
> in an Iceberg ORC table at CREATE/ALTER TABLE with a clear error, instead of
> failing every INSERT.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
