[
https://issues.apache.org/jira/browse/SINGA-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582208#comment-16582208
]
ASF subversion and git services commented on SINGA-387:
-------------------------------------------------------
Commit b55b046ccac33876a28861cba6badbddfae75788 in incubator-singa's branch
refs/heads/master from xuewanqi
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=b55b046 ]
SINGA-387 Modified the design of autograd backward engine and correct some
mistakes in it
Optimize the design of autograd engine:
- differentiate creator of inputs and creator of parameters, even though they
are both Dummy class.
- this can avoid unnecessary memory use, maining in store unnecessary gradients.
> Modified the design of autograd backward engine and correct some mistakes in
> it
> -------------------------------------------------------------------------------
>
> Key: SINGA-387
> URL: https://issues.apache.org/jira/browse/SINGA-387
> Project: Singa
> Issue Type: Improvement
> Reporter: Xue Wanqi
> Priority: Major
>
> Old design of autograd engine will return intermediate gradients. We should
> avoid the happen of that. For the parameters(tensors) need to store
> gradients, they or their creator should know whether the gradients are
> calculated readily. and only when gradients are prepared then they can be
> output.
> Old design of autograd engine will store some unused gradient in dictionary
> not_ready. this should be avoid as well in the new design.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
