Sahana Bhat created HIVE-24406: ---------------------------------- Summary: HiveConf creation in HiveMetastore's getMS() call adds ~100ms latency Key: HIVE-24406 URL: https://issues.apache.org/jira/browse/HIVE-24406 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 2.3.4 Reporter: Sahana Bhat Attachments: Screenshot 2020-11-17 at 5.17.50 PM.png
The changes to HMSHandler's getMSForConf introduced in the commit [https://github.com/apache/hive/commit/9a47cf9f92d1c8a4e72890e3dfe2d9567f12bfb5] makes getMSForConf and newRawStoreForConf static and hence adds an additional HiveConf object creation step for every metastore connection created. In a client like Presto, which creates a new metastore connection for every query, the object creation rate shoots up leading to higher heap usage, more frequent garbage collection, higher garbage collection times and increased latency in the service. We noticed a constant ~100ms increase in latency of all metastore calls during migration of HMS from 1x to 2x for Presto. PFA the latency difference of a get_table call between 1x and 2x. !Screenshot 2020-11-17 at 5.17.50 PM.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)