[
https://issues.apache.org/jira/browse/HIVE-26898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682210#comment-17682210
]
Riju Trivedi commented on HIVE-26898:
-------------------------------------
[~tarak271] Do we see this notification_log table growing and OOM issues In
recent versions 717, SP1 and SP2? Improvements like loading notifications in
batches (HIVE-19430) and handling batches in different transactions
(HIVE-24432) are already included in these versions.
> Split Notification logging so that busy clusters can have better performance
> ----------------------------------------------------------------------------
>
> Key: HIVE-26898
> URL: https://issues.apache.org/jira/browse/HIVE-26898
> Project: Hive
> Issue Type: New Feature
> Reporter: Taraka Rama Rao Lethavadla
> Priority: Major
>
> For DDL & DML events are logged into notifications log table and those get
> cleaned as soon as ttl got expired.
> In most of the busy clusters, the notification log is growing even though
> cleaner is running and kept on cleaning the events. It means the rate of Hive
> db operations are very high compared to rate at which cleaning is happening.
> So any query on this table is becoming bottle neck at backend DB causing slow
> response
> The proposal is to split the notification log table in to multiple tables
> like
> notification_log_dml - for all DML queries
> notification_log_insert - for all insert queries
> ..
> etc.
>
> So that load on that single table gets reduced improving the performance of
> the backend db as well as Hive
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
