Koji Noguchi created PIG-5384:
---------------------------------
Summary: OOM while spilling large bag
Key: PIG-5384
URL: https://issues.apache.org/jira/browse/PIG-5384
Project: Pig
Issue Type: Improvement
Reporter: Koji Noguchi
Assignee: Koji Noguchi
One of the common OOM issue in Pig is, Pig hitting OOM while trying to spill a
large bag. Current solutions is to give higher heapsize or tweak
{noformat}
pig.spill.memory.usage.threshold.fraction
pig.spill.collection.threshold.fraction
pig.spill.unused.memory.threshold.size
{noformat}
and make sure spilling starts early enough. These params are still critical
but wondering if any improvement can be made to increase the chances of
avoiding OOM while spilling a single large bag.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
