Todd Lipcon created IMPALA-7077:
-----------------------------------
Summary: Add a configuration for the maximum number of partitions
to load
Key: IMPALA-7077
URL: https://issues.apache.org/jira/browse/IMPALA-7077
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Affects Versions: Impala 2.12.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
The current design of the catalog server has some tables which are large enough
that they cannot be serialized in Thrift under 2GB (eg IMPALA-3910,
IMPALA-2648, etc). Anecdotally some users have some extremely large tables that
they use via Hive that, if they reference them in an Impala query, will bring
down the Impala cluster.
Long term we would like to support these without OOM, but a short term
improvement is to add a flag that limits the number of partitions to load to
some maximum. If the user references a table that is larger than this size,
they should get an error message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
