[
https://issues.apache.org/jira/browse/HIVE-29599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18109090#comment-18109090
]
Shohei Okumiya commented on HIVE-29599:
---------------------------------------
We may need to discuss the security model first. I heard Unity Catalog
(probably) offers the most restrictive security model. When column- or
row-level access control is enabled, it generates new Parquet files (or
something similar), and the REST endpoint returns FileIO that reads the
dynamically generated files rather than the original files.
# An Iceberg client accesses
`/v1/\{prefix}/namespaces/\{namespace}/tables/\{table}/plan`
# Unity Catalog materializes post-projected or filtered rows
# Unity Catalog makes a response
# The Iceberg client reads the files
I'd say this apparently requires a compute engine. I guess it is hard to start
with the perfect mode in this ticket.
If column- or row-level access control is not enabled,
`/v1/\{prefix}/namespaces/\{namespace}/tables/\{table}/plan` is likely to have
the same security mode as
`/v1/\{prefix}/namespaces/\{namespace}/tables/\{table}`; we have nothing new to
do.
I'm not confident what the server-side planning should return when column- or
row-level access control is enabled... 403 Forbidden could be safer. However,
can an Iceberg client fall back to /v1/tables? I think Credential Vending has
the same problem. We may check how other catalogs behave first.
> Support server-side scan planning of Iceberg
> --------------------------------------------
>
> Key: HIVE-29599
> URL: https://issues.apache.org/jira/browse/HIVE-29599
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Standalone Metastore
> Reporter: Shohei Okumiya
> Priority: Major
>
> Iceberg REST Catalog provides
> `/v1/\{prefix}/namespaces/\{namespace}/tables/\{table}/plan`, which pushes
> scan planning to the server side.
> [https://github.com/apache/iceberg/blob/apache-iceberg-1.10.1/open-api/rest-catalog-open-api.yaml#L605-L689]
> This effort could benefit users.
> - Read performance can improve if the catalog effectively caches the response
> - Clients can be thin; beneficial for new languages
> - With credential vending, we can give more fine-grained credentials
> In this ticket, we will try to implement the simplest mode of the server-side
> scan planning.
> - [https://lists.apache.org/thread/qq13468x6gk0vxnsckzc5xd02tjlvpkm]
> -
> [https://docs.google.com/document/d/1FdjCnFZM1fNtgyb9-v9fU4FwOX4An-pqEwSaJe8RgUg/edit?usp=sharing]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
