Rakshith C created HIVE-26937:
---------------------------------
Summary: Batch events during incremental replication to avoid O.O.M
Key: HIVE-26937
URL: https://issues.apache.org/jira/browse/HIVE-26937
Project: Hive
Issue Type: Improvement
Components: Hive
Reporter: Rakshith C
Assignee: Rakshith C
* Currently incremental replication flow of hive dumps all events read from
notification logs sequentially in staging directory.
* Repl Load loads all the event directories present in staging directory to a
list and processes them.
* This has caused O.O.M issues when number of events are large.
Hence introducing batching of events where Repl Dump dumps events in batches
and Repl Load processes events batch by batch to avoid O.O.M.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
