Vincent Tran created IMPALA-7034: ------------------------------------ Summary: Increase scalability of metadata handling Key: IMPALA-7034 URL: https://issues.apache.org/jira/browse/IMPALA-7034 Project: IMPALA Issue Type: Improvement Affects Versions: Impala 2.13.0 Reporter: Vincent Tran
Currently the practical limit for catalog topic update is in the neighborhood of 4GB - the fundamental limit of max thrift message size. This is an architectural limitation and not a resource limitation Larger enterprise clusters with high file counts can easily surpass this with normal usage. The high level ask here is for a more scalable implementation for metadata handling. The amount metadata that a cluster can handle should be proportional to the amount of hardware resource that a user is willing to allocate to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org