Great progress here. Thanks everyone. @StephanieYuan 1. Please rebase and resolve conflicts. 2. Please comment about tests added and coverage. 3. We had discussed in above comment, to verify this implementation with MNIST to confirm correctness per the claims in paper. Please comment.
@anirudhacharya - I would still prefer keeping SVRG optimizer in contrib and allow it to evolve. As mentioned in above comments and design doc, Other combination like Adam+SVRG could also be used but not benchmarked. I would set the default optimizer for SVRGModule to be SVRGOptimizer but not remove the option altogether and make SVRGOptimizer internal. Thoughts? [ Full content available at: https://github.com/apache/incubator-mxnet/pull/12376 ] This message was relayed via gitbox.apache.org for [email protected]
