[
https://issues.apache.org/jira/browse/PIG-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454671#comment-13454671
]
Dmitriy V. Ryaboy commented on PIG-2917:
----------------------------------------
Haitao, I am curious what your script is doing that causes it to allocate that
many bags?
I just created a ticket ripping out some unnecessary usage of Spillable
objects; suspect your usage is pointing us at another place where we don't
really need to be dealing with SMM in the first place.
Patch seems reasonable. Stylistically, I prefer the code below for iteration of
Iterables:
{code}
for (WeakReference<Spillable> ref : spillables) {
...
}
{code}
> SpillableMemoryManager memory leak for WeakReference
> ----------------------------------------------------
>
> Key: PIG-2917
> URL: https://issues.apache.org/jira/browse/PIG-2917
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.11
> Reporter: Haitao Yao
> Attachments: aa.jpg, SpillableMemoryManager.patch
>
>
> Sine the SpillableMemoryManager subscribed the memory notifications, and hold
> a weak reference of all the spillable, if there's too many weak reference and
> the memory exceeds the threshold, the current code will not clear the
> spillable weak references.
> the details is illustrated in the attached screenshot of a MemoryAnalyzer.
> So we need to clear the spillables when the memory notifications is received.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
