[
https://issues.apache.org/jira/browse/HIVE-29617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-29617:
----------------------------------
Description:
Repro
# Create an Iceberg table with and older Hive version
# Insert some
# make sure that column stats are up-to-date. (DESCRIBE FORMATTED <table name>
<column name>)
# Upgrade Hive to the next version
# Try to run a query that requires column statistics
Exception is thrown
{code}
java.lang.UnsupportedOperationException: null
at java.util.AbstractList.add(AbstractList.java:153)
at java.util.AbstractList.add(AbstractList.java:111)
at java.util.AbstractCollection.addAll(AbstractCollection.java:336)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateStatsForMissingCols(StatsUtils.java:255)
{code}
But the cause of this exception is this other one located in hive.log
{code}
2026-05-06 19:21:28,653 WARN
org.apache.iceberg.mr.hive.HiveIcebergStorageHandler:
[6856cc4a-f953-48e9-94a2-c228d9a226ee HiveServer2-HttpHandler-Pool:
Thread-109]: Unable to read col stats:
org.apache.commons.lang3.SerializationException: java.io.InvalidClassException:
org.apache.hadoop.hive.metastore.api.ColumnStatistics; local class
incompatible: stream classdesc serialVersionUID = -2416986586064965120, local
class serialVersionUID = -7870173800499813820
at
org.apache.commons.lang3.SerializationUtils.deserialize(SerializationUtils.java:195)
at
org.apache.commons.lang3.SerializationUtils.deserialize(SerializationUtils.java:158)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.readColStats(IcebergTableUtil.java:822)
...
Caused by: java.io.InvalidClassException:
org.apache.hadoop.hive.metastore.api.ColumnStatistics; local class
incompatible: stream classdesc serialVersionUID = -2416986586064965120, local
class serialVersionUID = -7870173800499813820
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:597)
~[?:?]
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2051) ~[?:?]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1898)
~[?:?]
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2224) ~[?:?]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1733)
~[?:?]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:509)
~[?:?]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:467)
~[?:?]
at
org.apache.commons.lang3.SerializationUtils.deserialize(SerializationUtils.java:192)
{code}
was:
Repro
# Create an Iceberg table with and older Hive version
# Insert some
# make sure that column stats are up-to-date. (DESCRIBE FORMATTED <table name>
<column name>)
# Upgrade Hive to the next version
# Try to run a query that requires column statistics
> Unable to load column statistics of Iceberg table after upgrading Hive
> ----------------------------------------------------------------------
>
> Key: HIVE-29617
> URL: https://issues.apache.org/jira/browse/HIVE-29617
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration, stats
> Reporter: Krisztian Kasa
> Priority: Major
>
> Repro
> # Create an Iceberg table with and older Hive version
> # Insert some
> # make sure that column stats are up-to-date. (DESCRIBE FORMATTED <table
> name> <column name>)
> # Upgrade Hive to the next version
> # Try to run a query that requires column statistics
> Exception is thrown
> {code}
> java.lang.UnsupportedOperationException: null
> at java.util.AbstractList.add(AbstractList.java:153)
> at java.util.AbstractList.add(AbstractList.java:111)
> at java.util.AbstractCollection.addAll(AbstractCollection.java:336)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.estimateStatsForMissingCols(StatsUtils.java:255)
> {code}
> But the cause of this exception is this other one located in hive.log
> {code}
> 2026-05-06 19:21:28,653 WARN
> org.apache.iceberg.mr.hive.HiveIcebergStorageHandler:
> [6856cc4a-f953-48e9-94a2-c228d9a226ee HiveServer2-HttpHandler-Pool:
> Thread-109]: Unable to read col stats:
> org.apache.commons.lang3.SerializationException:
> java.io.InvalidClassException:
> org.apache.hadoop.hive.metastore.api.ColumnStatistics; local class
> incompatible: stream classdesc serialVersionUID = -2416986586064965120, local
> class serialVersionUID = -7870173800499813820
> at
> org.apache.commons.lang3.SerializationUtils.deserialize(SerializationUtils.java:195)
> at
> org.apache.commons.lang3.SerializationUtils.deserialize(SerializationUtils.java:158)
> at
> org.apache.iceberg.mr.hive.IcebergTableUtil.readColStats(IcebergTableUtil.java:822)
> ...
> Caused by: java.io.InvalidClassException:
> org.apache.hadoop.hive.metastore.api.ColumnStatistics; local class
> incompatible: stream classdesc serialVersionUID = -2416986586064965120, local
> class serialVersionUID = -7870173800499813820
> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:597)
> ~[?:?]
> at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2051) ~[?:?]
> at
> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1898) ~[?:?]
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2224)
> ~[?:?]
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1733)
> ~[?:?]
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:509)
> ~[?:?]
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:467)
> ~[?:?]
> at
> org.apache.commons.lang3.SerializationUtils.deserialize(SerializationUtils.java:192)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
