Shohei Okumiya created HIVE-29016:
-------------------------------------
Summary: Disable caching on the Iceberg REST Catalog
Key: HIVE-29016
URL: https://issues.apache.org/jira/browse/HIVE-29016
Project: Hive
Issue Type: Improvement
Components: Iceberg integration, Standalone Metastore
Reporter: Shohei Okumiya
The current implementation caches entries based on the property `
metastore.iceberg.catalog.cache.expiry`.
[https://github.com/apache/hive/blob/4193a6db6243bfd8a5a2a3d142bef5acdb02e389/standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCachingCatalog.java]
I observed that this made CREATE TABLE from Trino non-functional. With
CachingCatalog, [a transaction is created as a
CommitCallbackTransaction|https://github.com/apache/iceberg/blob/62d9ff5d043a5571efe020b9177998ae763a41a0/core/src/main/java/org/apache/iceberg/CachingCatalog.java#L286-L303].
However, CatalogHandler can't process the class, as CommitCallbackTransaction
is not a sub-class of BaseTransaction. Our implementation is probably wrong.
*
[https://github.com/apache/hive/blob/4193a6db6243bfd8a5a2a3d142bef5acdb02e389/standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java#L420]
*
[https://github.com/apache/iceberg/blob/62d9ff5d043a5571efe020b9177998ae763a41a0/core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java#L350-L356]
Essentially, I believe that caching Iceberg objects is a form of premature
optimization. What if a user accesses HMS via Iceberg REST API and another
accesses the same table via Thrift API? What if Hive Metastore is configured
with HA? It is likely causing a issue.
In this ticket, I propse to remove the caching mechanism once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
