[ https://issues.apache.org/jira/browse/IMPALA-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755488#comment-16755488 ]
bharath v commented on IMPALA-7034: ----------------------------------- [~thundergun] This should be substantially improved with IMPALA-7127 and IMPALA-2649. Can we resolve this? > Increase scalability of metadata handling > ----------------------------------------- > > Key: IMPALA-7034 > URL: https://issues.apache.org/jira/browse/IMPALA-7034 > Project: IMPALA > Issue Type: Improvement > Components: Catalog > Affects Versions: Impala 2.13.0 > Reporter: Vincent Tran > Priority: Major > > Currently the practical limit for catalog topic update is in the neighborhood > of 4GB - the fundamental limit of max thrift message size. This is an > architectural limitation and not a resource limitation > Larger enterprise clusters with high file counts can easily surpass this with > normal usage. > The high level ask here is for a more scalable implementation for metadata > handling. The amount metadata that a cluster can handle should be > proportional to the amount of hardware resource that an user is willing to > allocate to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org