Prasanth Jayachandran created HIVE-20656:
--------------------------------------------
Summary: Map aggregation memory configs are too aggressive
Key: HIVE-20656
URL: https://issues.apache.org/jira/browse/HIVE-20656
Project: Hive
Issue Type: Bug
Affects Versions: 4.0.0, 3.2.0
Reporter: Prasanth Jayachandran
The defaults for the following configs seems to be too aggressive. In java this
can easily lead to several full GC pauses whose memory cannot be reclaimed.
{code:java}
HIVEMAPAGGRHASHMEMORY("hive.map.aggr.hash.percentmemory", (float) 0.99,
"Portion of total memory to be used by map-side group aggregation hash
table"),
HIVEMAPAGGRMEMORYTHRESHOLD("hive.map.aggr.hash.force.flush.memory.threshold",
(float) 0.9,
"The max memory to be used by map-side group aggregation hash table.\n" +
"If the memory usage is higher than this number, force to flush
data"),{code}
We can be little bit conservative for these configs to avoid getting into GC
pause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
