[
https://issues.apache.org/jira/browse/SINGA-381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539716#comment-16539716
]
ASF subversion and git services commented on SINGA-381:
-------------------------------------------------------
Commit 81908a82f4c9ea01b1359ed3d8fb4118a5bfd147 in incubator-singa's branch
refs/heads/master from Wang Wei
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=81908a8 ]
SINGA-381 - Update the autograd API to yeild the gradients
yield gradients by backward() in autograd.py; this saves memory by releasing
gradients early
> Update the autograd API to yeild the gradients
> ----------------------------------------------
>
> Key: SINGA-381
> URL: https://issues.apache.org/jira/browse/SINGA-381
> Project: Singa
> Issue Type: Improvement
> Reporter: wangwei
> Priority: Major
>
> This ticket is going to update one API in autograd.py
> {code:java}
> def backward(y, dy=None)
> # returns the gradient tensor one by one using yield
> def gradients(y, dy=None)
> # returns a dictionary: param tensor -> gradient tensor{code}
>
> With the backward() API, we can update the param immediately after its
> gradient is available. Then, the gradient tensor can be deleted and the
> memory is released.
>
> The gradients() API keeps all gradient tensors in the memory, which incurs
> memory overhead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
