Wang Ji created SINGA-145:
-----------------------------
Summary: New SGD based optimization Updaters: AdaDelta, Adam,
AdamMax
Key: SINGA-145
URL: https://issues.apache.org/jira/browse/SINGA-145
Project: Singa
Issue Type: New Feature
Environment: Universal
Reporter: Wang Ji
Priority: Minor
This ticket implements three Stochastic Gradient Descent (SGD) based
optimization algorithms in Updater.cc, i.e. AdaDelta, Adaptive Moment
Estimation(Adam) with its variant AdamMax .These algorithms adapt the learning
rate to the parameters, performing larger updates for infrequent and smaller
updates for frequent parameters.
For algorithm details, refer to AdaDelata (http://arxiv.org/abs/1212.5701),
Adam (http://arxiv.org/pdf/1412.6980.pdf).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
