Haitao Yao created PIG-2812:
-------------------------------
Summary: Spill InternalCachedBag into only 1 file
Key: PIG-2812
URL: https://issues.apache.org/jira/browse/PIG-2812
Project: Pig
Issue Type: Bug
Components: data
Reporter: Haitao Yao
I encountered a reducer's OOM because of java.io.DeleteOnExitHook. And I found
out that the InternalCachedBag creates a seperate tmp file, and the tmp files
is deleted on exit. So the file delete hook caused the OOM.
Why not just hold the tmp file handle and spill only one tmp file?
Too many tmp files may block the tasktracker start process, if the tmp files
are not cleaned on time and the tasktracker restarts at this specific time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
