[
https://issues.apache.org/jira/browse/SINGA-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582206#comment-16582206
]
ASF subversion and git services commented on SINGA-387:
-------------------------------------------------------
Commit 6c28abdeb6929334ba10d327fe1fd80e2d0b604c in incubator-singa's branch
refs/heads/master from xuewanqi
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=6c28abd ]
SINGA-387 Modified the design of autograd backward engine and correct some
mistakes in it
one of alternative solutions, the simplest one:
- output every intermediate gradient once it is calcualted and sum these
gradients togather in the funciton gradients().
> Modified the design of autograd backward engine and correct some mistakes in
> it
> -------------------------------------------------------------------------------
>
> Key: SINGA-387
> URL: https://issues.apache.org/jira/browse/SINGA-387
> Project: Singa
> Issue Type: Improvement
> Reporter: Xue Wanqi
> Priority: Major
>
> Old design of autograd engine will return intermediate gradients. We should
> avoid the happen of that. For the parameters(tensors) need to store
> gradients, they or their creator should know whether the gradients are
> calculated readily. and only when gradients are prepared then they can be
> output.
> Old design of autograd engine will store some unused gradient in dictionary
> not_ready. this should be avoid as well in the new design.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
