[ https://issues.apache.org/jira/browse/IMPALA-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
bharath v resolved IMPALA-5835. ------------------------------- Resolution: Cannot Reproduce Fix Version/s: Not Applicable Please try out the latest bits and reopen if needed. > Severe slowdown in catalogd startup after 2.1 → 2.5 upgrade with > 200,000 > databases > ------------------------------------------------------------------------------------ > > Key: IMPALA-5835 > URL: https://issues.apache.org/jira/browse/IMPALA-5835 > Project: IMPALA > Issue Type: Bug > Components: Catalog > Affects Versions: Impala 2.6.0, Impala 2.7.0, Impala 2.5.5, Impala 2.8.0, > Impala 2.9.0, Impala 2.10.0 > Reporter: Ben Breakstone > Assignee: bharath v > Priority: Major > Labels: performance > Fix For: Not Applicable > > > After an upgrade from Impala 2.1 (CDH 5.3.9) to Impala 2.5 (CDH 5.7.5), > starting up Catalog Server takes around eight to ten hours. It took around > twenty minutes before the upgrade. > There are over 200,000 databases in use. Looking in the catalogd log as it > starts up for hours, it says > "Loading native functions for database..." and then > "Loading Java functions for database..." for each database. Based on this, it > appears the introduction of persistent UDFs/UDAs is causing the slowdown. > Only one of the databases actually has any UDFs defined. > num_metadata_loading_threads is set to 64. Background loading of metadata is > disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)