[
https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-23282:
------------------------------------
Description:
ObjectStore::getPartitionsByExprInternal internally uses Table information for
getting partitionKeys, table, catalog name.
For this, it ends up populating entire table data from DB (including skew
column, parameters, sort, bucket cols etc). This makes it a lot more expensive
call. It would be good to check if MTable itself can be used instead of Table.
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327]
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669]
!image-2020-04-23-14-07-06-077.png|width=665,height=592!
was:
ObjectStore::getPartitionsByExprInternal internally uses Table information for
getting partitionKeys, table, catalog name.
For this, it ends up populating entire table data from DB (including skew
column, parameters, sort, bucket cols etc). This makes it a lot more expensive
call. It would be good to either have a lightweight object to have basic
information or reduce the payload on Table object itself.
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327]
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669]
!image-2020-04-23-14-07-06-077.png|width=665,height=592!
> Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
> ---------------------------------------------------------------------
>
> Key: HIVE-23282
> URL: https://issues.apache.org/jira/browse/HIVE-23282
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Rajesh Balamohan
> Priority: Major
> Attachments: image-2020-04-23-14-07-06-077.png
>
>
> ObjectStore::getPartitionsByExprInternal internally uses Table information
> for getting partitionKeys, table, catalog name.
>
> For this, it ends up populating entire table data from DB (including skew
> column, parameters, sort, bucket cols etc). This makes it a lot more
> expensive call. It would be good to check if MTable itself can be used
> instead of Table.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327]
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669]
>
> !image-2020-04-23-14-07-06-077.png|width=665,height=592!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
