[
https://issues.apache.org/jira/browse/MXNET-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16824593#comment-16824593
]
Andrew Ayres commented on MXNET-1154:
-------------------------------------
Documenting where this issues stands for when it's eventually picked back up.
Initially when we attempted to update module with ResourceScope, we ran into
some problems with the update causing segmentation faults. Some time was spent
to try and resolve this but little progress was made. Since this task is
primarily about code maintenance and maintainability (i.e. no known bugs,
leaks, or other issues resulted from this), other tasks were given priority. At
the time, it was believed that the segmentation fault was the result of an
incorrect implementation and not due to any issues with ResourceScope itself.
Later, while fixing [https://github.com/apache/incubator-mxnet/issues/10867],
it was realized that the segmentation faults were occurring due to an
unforeseen complication with ResourceScope. Essentially, a native resource
would be created, then new ResourceScopes opened, and the native resource could
end up creating new dependencies that were also native resources. The result
was that the original native resource was at one scope level and it's
dependencies at another. Leading to the resources dependencies being
deallocated and leaving it in a bad state.
The PR which fixed the bug likely fixed the cause of the seg faults that we
were seeing as well. At the very least, it introduced the mechanisms to resolve
them (ResourceScope.usingIfScopeExists).
The original attempt to fix this is preserved at
[https://github.com/andrewfayres/incubator-mxnet/tree/moodule]. This can mostly
be copied but there will be a few sections which should be updated for the
changes to the code base since then. It may also be necessary to add the
usingIfScopeExists method inside module wherever native resources are created.
> Update Module.Scala to resolve Memory Leaks
> -------------------------------------------
>
> Key: MXNET-1154
> URL: https://issues.apache.org/jira/browse/MXNET-1154
> Project: Apache MXNet
> Issue Type: Sub-task
> Components: Apache MXNet Scala API
> Reporter: Naveen Swamy
> Priority: Major
>
> Module.Scala throws a quite a few warnings for undisposed NDArrays. Although
> they will be cleaned up at some point due to the Phantom Reference feature,
> it could be cause OOM if the GC does not run frequently and also it is bad
> experience to see a lot of Warnings with Stack traces.
> Reach out to me offline and I will discuss some of the pointers I have found
> and possible solutions.
> This is one of the issues that a Clojure user reported, which is applicable
> to both Scala and Java users.
> https://discuss.mxnet.io/t/dealing-with-memory-leaks-ndarray/1767
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
