Hi MXNet dev community, My name is Stephanie Yuan and it's great to join the MXNet dev family! I'm proposing a new design doc for implementing SVRG optimization technique in MXNet Python Module.
*Problem Description: * SVRG optimization is a technique that complements SGD, which was first proposed in the paper Accelerating Stochastic Gradient Descent using Predicative Variance Reduction <https://papers.nips.cc/paper/4937-accelerating-stochastic-gradient-descent-using-predictive-variance-reduction.pdf> in 2013. It has provable guarantees for strongly convex functions and converges much faster than SGD. An initial set of experiments using YearPredictionMSD dataset has been conducted and yields promising results, which is one of the motivations for this proposal. *Expected Deliverables:* The goal is to implement a MXNet Python Module that implements SVRG optimization technique. Detailed implementation approaches and Benchmark results can be found in the Confluence design doc <https://cwiki.apache.org/confluence/display/MXNET/SVRG+Optimization+in+MXNet+Python+Module> . Please let me know if you have any questions! Thank you very much for your time and considerations! Cheers, Stephanie Yuan
