[ https://issues.apache.org/jira/browse/SPARK-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596991#comment-14596991 ]
Aaron Staple commented on SPARK-1503: ------------------------------------- [~josephkb] and [~lewuathe] Sure, happy to coordinate. So far I have just been duplicating the the convergence tolerance check in TFOCS, the matlab package on which the accelerated gradient descent implementation is based. TFOCS also tests for convergence by checking if the relative change in the weight vector is below a specified threshold. But there are some differences from the SPARK-3382 implementation. For example in TFOCS the relative difference between new and old weight vectors is measured with respect to the new weight vector instead of the old. And if the new weight vector is smaller than the unit vector the convergence test is changed to be an absolute rather than relative difference between successive weight vectors. I am just describing the implementation here, happy to discuss further and potentially look at making changes. Here is the relevant code if you are interested (there is also a separate condition when the weight vector does not change between iterations): https://github.com/cvxr/TFOCS/blob/e34c0daeb136935d23b8df506de8b7b191f6b0a3/private/tfocs_iterate.m#L19-L24 > Implement Nesterov's accelerated first-order method > --------------------------------------------------- > > Key: SPARK-1503 > URL: https://issues.apache.org/jira/browse/SPARK-1503 > Project: Spark > Issue Type: New Feature > Components: MLlib > Reporter: Xiangrui Meng > Assignee: Aaron Staple > Attachments: linear.png, linear_l1.png, logistic.png, logistic_l2.png > > > Nesterov's accelerated first-order method is a drop-in replacement for > steepest descent but it converges much faster. We should implement this > method and compare its performance with existing algorithms, including SGD > and L-BFGS. > TFOCS (http://cvxr.com/tfocs/) is a reference implementation of Nesterov's > method and its variants on composite objectives. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org