zhangda created HIVE-29194:
------------------------------
Summary: large numbers of MNotificationLog causes OOM
Key: HIVE-29194
URL: https://issues.apache.org/jira/browse/HIVE-29194
Project: Hive
Issue Type: Improvement
Components: Standalone Metastore
Affects Versions: 3.1.2
Reporter: zhangda
Attachments: 1.patch
The current implementation MNotificationLog cleaner consists of 2 steps, first
selecting out all outdated logs, then deleting them. It causes problem in our
system when there are large numbers of MNotificationLog accumulated in the
database (not sure how it happened but we guess the cleaner thread stopped
working for some reason, or ppl dropped tables with lots of partitions,
re-created tables/added back big numbers of partitions etc.). Those
MNotificationLog objects are loaded into the memory, trigger continuous full gc
and oom. As the matter of fact, we can just delete those logs without selecting
them. Fix is in the attachment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
