[
https://issues.apache.org/jira/browse/IMPALA-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-14896:
-------------------------------------
Description:
Currently loading an Iceberg table loads 5 cache object and lead to 5
GetPartialCatalogObject RPCs:
1. CatalogdMetaProvider.loadTable()
2. CatalogdMetaProvider.loadIcebergTable()
3. CatalogdMetaProvider.loadTableColumnStatistics()
4. CatalogdMetaProvider.loadPartitionList()
5. CatalogdMetaProvider.loadPartitionsByRefs()
4 and 5 return dummy result in Iceberg and could be skipped
https://gerrit.cloudera.org/#/c/24049/ is POC patch and implements this among
other things,
Note that IMPALA-14806 / https://gerrit.cloudera.org/#/c/24177/ adds an extra
call (CatalogdMetaProvider.loadIcebergContentFileStore)
was:
Currently loading an Iceberg table loads 5 cache object and lead to 5
GetPartialCatalogObject RPCs:
1. CatalogdMetaProvider.loadTable()
2. CatalogdMetaProvider.loadIcebergTable()
3. CatalogdMetaProvider.loadTableColumnStatistics()
4. CatalogdMetaProvider.loadPartitionList()
5. CatalogdMetaProvider.loadPartitionsByRefs()
4 and 5 return dummy result in Iceberg and could be skipped
Note that IMPALA-14806 / https://gerrit.cloudera.org/#/c/24177/ adds an extra
call (CatalogdMetaProvider.loadIcebergContentFileStore)
> Avoid extra GetPartialCatalogObject calls in Icebert table loading
> ------------------------------------------------------------------
>
> Key: IMPALA-14896
> URL: https://issues.apache.org/jira/browse/IMPALA-14896
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Frontend
> Reporter: Csaba Ringhofer
> Priority: Major
> Labels: iceberg
>
> Currently loading an Iceberg table loads 5 cache object and lead to 5
> GetPartialCatalogObject RPCs:
> 1. CatalogdMetaProvider.loadTable()
> 2. CatalogdMetaProvider.loadIcebergTable()
> 3. CatalogdMetaProvider.loadTableColumnStatistics()
> 4. CatalogdMetaProvider.loadPartitionList()
> 5. CatalogdMetaProvider.loadPartitionsByRefs()
> 4 and 5 return dummy result in Iceberg and could be skipped
> https://gerrit.cloudera.org/#/c/24049/ is POC patch and implements this
> among other things,
> Note that IMPALA-14806 / https://gerrit.cloudera.org/#/c/24177/ adds an extra
> call (CatalogdMetaProvider.loadIcebergContentFileStore)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
