Rajkumar Singh created HIVE-22081:
-------------------------------------
Summary: Hivemetastore Performance: Compaction Initiator thread
overwhelmed if no there are too many Table/partitions are eligible for
compaction
Key: HIVE-22081
URL: https://issues.apache.org/jira/browse/HIVE-22081
Project: Hive
Issue Type: Improvement
Components: Transactions
Affects Versions: 3.1.1
Reporter: Rajkumar Singh
Assignee: Rajkumar Singh
if Automatic Compaction is turned on, Initiator thread check for potential
table/partitions which are eligible for compactions and run some checks in for
loop before requesting compaction for eligibles. Though initiator thread is
configured to run at interval 5 min default, in case of many objects it keeps
on running as these checks are IO intensive and hog cpu.
In the proposed changes, I am planning to do
1. passing less object to for loop by filtering out the objects based on the
condition which we are checking within the loop.
2. Doing Async call using future to determine compaction type(this is where we
do FileSystem calls)
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
