Sergey Shelukhin created HIVE-5189:
--------------------------------------
Summary: make batching in partition retrieval in metastore
applicable to more methods
Key: HIVE-5189
URL: https://issues.apache.org/jira/browse/HIVE-5189
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
As indicated in HIVE-5158, Metastore can OOM if retrieving a large number of
partitions. For client-side partition filtering, the client applies batching
(that would avoid that) by sending parts of the filtered name list in separate
request according to configuration.
The batching is not used on filter pushdown path, and when retrieving all
partitions (e.g. when the pruner expression is not useful in non-strict mode).
HIVE-4914 and pushdown improvements will make this problem somewhat worse by
allowing more requests to go to the server.
There needs to be some batching scheme (ideally, a somewhat generic one) that
would be applicable to all these paths.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
