[
https://issues.apache.org/jira/browse/IMPALA-11795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17694055#comment-17694055
]
Csaba Ringhofer commented on IMPALA-11795:
------------------------------------------
I see, on hive-3.1.2 the "engine" field is not yet added to column statistics:
https://github.com/apache/hive/blob/4784df9a186733eae1794078bacfd610cdcfee09/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift#L584
But high/low values are already there:
https://github.com/apache/hive/blob/8190d2be7b7165effa62bd21b7d60ef81fb0e4af/standalone-metastore/src/main/thrift/hive_metastore.thrift#L493
When using hive-4 line, we don't normally see this issue because Hive and
Impala column stats are completely separated with the "engine" field.
I see now that high/low values are a >10 year old feature in Hive, but Impala
only started to use them recently (2021), so at that point the "engine" field
was already there and we didn't need to deal with stats generated by Hive.
>Would it be more elegant to also set the maximum minimum value for the
>timestamp type
I am not sure, min/max values can be quite tricky for timestamps due to
timezone handling and rounding (the min/max is in microsecond precision, while
Impala stores timestamps and naneseconds precision). It would be nice to
implement this one day, but it looks like a much bigger change, so I would use
the quick fix to avoid table loading problems.
> Failed to load table metadata from catalog due to EXCEPTION:Unsupported type
> encountered in setLowAndHighValue()
> ----------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-11795
> URL: https://issues.apache.org/jira/browse/IMPALA-11795
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 4.2.0
> Reporter: zhi tang
> Assignee: Csaba Ringhofer
> Priority: Major
> Fix For: Impala 4.3.0
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
