[
https://issues.apache.org/jira/browse/HIVE-30150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raghav Aggarwal updated HIVE-30150:
-----------------------------------
Description:
In the Apache Iceberg REST Catalog (IRC) specification, loading a table
({{{}GET /v1/\{prefix}/namespaces/\{namespace}/tables/\{table}{}}}) returns a
{{LoadTableResponse}} payload containing the full {{TableMetadata}} JSON. For
large production tables with extensive snapshot histories, schemas, and
partition specs, this payload can grow significantly.
Without conditional HTTP request caching ({{{}ETag{}}} and {{If-None-Match}}
per RFC 9110), engine clients (Spark, Trino) must download and deserialize the
entire metadata payload on every refresh or query plan resolution, even when no
new commits have occurred. Furthermore, Hive's metastore-rest-catalog
previously lacked header plumbing and standard {{304 Not Modified}} / {{204 No
Content}} status code handling.
References:
# [https://github.com/apache/iceberg/pull/14398]
# [https://github.com/apache/iceberg/pull/12194]
#
[https://docs.google.com/document/d/1rnVSP_iv2I47giwfAe-Z3DYhKkKwWCVvCkC9rEvtaLA]
> Implement ETag-based freshness-aware table loading for HMS Iceberg REST
> Catalog
> -------------------------------------------------------------------------------
>
> Key: HIVE-30150
> URL: https://issues.apache.org/jira/browse/HIVE-30150
> Project: Hive
> Issue Type: Sub-task
> Reporter: Raghav Aggarwal
> Assignee: Raghav Aggarwal
> Priority: Major
>
> In the Apache Iceberg REST Catalog (IRC) specification, loading a table
> ({{{}GET /v1/\{prefix}/namespaces/\{namespace}/tables/\{table}{}}}) returns a
> {{LoadTableResponse}} payload containing the full {{TableMetadata}} JSON. For
> large production tables with extensive snapshot histories, schemas, and
> partition specs, this payload can grow significantly.
> Without conditional HTTP request caching ({{{}ETag{}}} and {{If-None-Match}}
> per RFC 9110), engine clients (Spark, Trino) must download and deserialize
> the entire metadata payload on every refresh or query plan resolution, even
> when no new commits have occurred. Furthermore, Hive's metastore-rest-catalog
> previously lacked header plumbing and standard {{304 Not Modified}} / {{204
> No Content}} status code handling.
> References:
> # [https://github.com/apache/iceberg/pull/14398]
> # [https://github.com/apache/iceberg/pull/12194]
> #
> [https://docs.google.com/document/d/1rnVSP_iv2I47giwfAe-Z3DYhKkKwWCVvCkC9rEvtaLA]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
