GitHub user iyerr3 opened a pull request:
https://github.com/apache/madlib/pull/313
MLP: Simplify momentum and Nesterov updates
Momentum updates are complicated due to Nesterov requiring an initial
update before gradient calculations. There is, however, a different form of the
Nesterov update that can be cleanly performed after the regular update,
simplifying the code.
Note: the `gradientInPlace` update for zero momentum is still retained
without the velocity vectors since it avoids copying the gradient vectors.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/madlib/madlib feature/simplify_nesterov
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/madlib/pull/313.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #313
----
commit d18f69f461d1ca5ab692c15c748ab6df2bc8959e
Author: Rahul Iyer <riyer@...>
Date: 2018-08-17T08:42:53Z
MLP: Simplify momentum and Nesterov updates
----
---
