Rohini Palaniswamy created PIG-4782:
---------------------------------------
Summary: OutOfMemoryError: GC overhead limit exceeded with
POPartialAgg
Key: PIG-4782
URL: https://issues.apache.org/jira/browse/PIG-4782
Project: Pig
Issue Type: Bug
Reporter: Rohini Palaniswamy
Assignee: Rohini Palaniswamy
Fix For: 0.16.0
In some cases, even though spill is triggered the main thread is still
executing some user UDF which constructs a DataBag which requires lot of
memory. Since we block on spill in POPartialAgg, there is kind of a deadlock as
there is no memory for the user DataBag to grow and spill does not happen and
job finally failing with OutOfMemoryError: GC overhead limit exceeded. So need
to make POPartialAgg non-blocking so that the user DataBag can be spilled first
for the code to reach POPartialAgg and spill.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
