[
https://issues.apache.org/jira/browse/HIVE-29016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17984985#comment-17984985
]
Shohei Okumiya commented on HIVE-29016:
---------------------------------------
[~henrib] Thanks! I think #5882 is very promising!
However, I probably have a slightly different stance. This is the priority in
my mind.
|| ||Accurate||Inaccurate||
|Fast|P2|Probably, no one wants this|
|Non-fast|P1|No one wants this|
In fact, the REST Catalog currently has zero users, and some or all third-party
query engines have not been able to access it successfully so far. Therefore,
I'd say the current priority is to make it fully functional with a
straightforward architecture. Inventing lots in performance at this point can
potentially make it hard to achieve P1, I think :) Once we are confident in the
P1 area with sure test cases, we can move forward to P2 with your help.
> Disable caching on the Iceberg REST Catalog
> -------------------------------------------
>
> Key: HIVE-29016
> URL: https://issues.apache.org/jira/browse/HIVE-29016
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Standalone Metastore
> Reporter: Shohei Okumiya
> Priority: Major
> Labels: hive-4.1.0-must
>
> The current implementation caches entries based on the property `
> metastore.iceberg.catalog.cache.expiry`.
> [https://github.com/apache/hive/blob/4193a6db6243bfd8a5a2a3d142bef5acdb02e389/standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCachingCatalog.java]
>
> I observed that this made CREATE TABLE from Trino non-functional. With
> CachingCatalog, [a transaction is created as a
> CommitCallbackTransaction|https://github.com/apache/iceberg/blob/62d9ff5d043a5571efe020b9177998ae763a41a0/core/src/main/java/org/apache/iceberg/CachingCatalog.java#L286-L303].
> However, CatalogHandler can't process the class, as
> CommitCallbackTransaction is not a sub-class of BaseTransaction. Our
> implementation is probably wrong.
> *
> [https://github.com/apache/hive/blob/4193a6db6243bfd8a5a2a3d142bef5acdb02e389/standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java#L420]
> *
> [https://github.com/apache/iceberg/blob/62d9ff5d043a5571efe020b9177998ae763a41a0/core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java#L350-L356]
>
> Essentially, I believe that caching Iceberg objects is a form of premature
> optimization. What if a user accesses HMS via Iceberg REST API and another
> accesses the same table via Thrift API? What if Hive Metastore is configured
> with HA? It is likely causing a issue.
> In this ticket, I propse to remove the caching mechanism once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
