Eugene Koifman created HIVE-11987:
-------------------------------------
Summary: CompactionTxnHandler.createValidCompactTxnList() can use
much less memory
Key: HIVE-11987
URL: https://issues.apache.org/jira/browse/HIVE-11987
Project: Hive
Issue Type: Bug
Components: Metastore, Transactions
Affects Versions: 1.1.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Priority: Minor
This method only needs HWM and list of txn IDs in 'o' state.
It's currently implemented to get the list from TxnHandler.getOpenTxnsInfo(),
which returns (txn id, state, host, user) for each txn and includes Aborted
txns.
This can easily be 120 bytes or more per txn overhead (over 1 Java long) which
not an issue in general but when the system is misconfigured, the number of
opened/aborted txns can get into the millions. This creates unnecessary memory
pressure on metastore.
Should consider fixing this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
