[
https://issues.apache.org/jira/browse/MAHOUT-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13714805#comment-13714805
]
Kun Yang commented on MAHOUT-1273:
----------------------------------
PS, the MPI version(see Steven Boyd's webpage) can not do cross validation
which is quite essential. I think there is a trade-off: if you want to do
cross-validation, large number of predictors will be an issue; if you want to
handle large number of predictors, cross-validation will be an issue.
Freedman's paper about coordinate descent:
http://www.jstatsoft.org/v33/i01/paper, http://www.jstatsoft.org/v33/i01/paper
Steven Boyd's paper about iterative distributed version:
http://www.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf, their
implementation is on MPI.
Best,
-Kun
> Single Pass Algorithm for Penalized Linear Regression on MapReduce
> ------------------------------------------------------------------
>
> Key: MAHOUT-1273
> URL: https://issues.apache.org/jira/browse/MAHOUT-1273
> Project: Mahout
> Issue Type: New Feature
> Reporter: Kun Yang
> Attachments: PenalizedLinear.pdf
>
> Original Estimate: 720h
> Remaining Estimate: 720h
>
> Penalized linear regression such as Lasso, Elastic-net are widely used in
> machine learning, but there are no very efficient scalable implementations on
> MapReduce.
> The published distributed algorithms for solving this problem is either
> iterative (which is not good for MapReduce, see Steven Boyd's paper) or
> approximate (what if we need exact solutions, see Paralleled stochastic
> gradient descent); another disadvantage of these algorithms is that they can
> not do cross validation in the training phase, which requires a
> user-specified penalty parameter in advance.
> My ideas can train the model with cross validation in a single pass. They are
> based on some simple observations.
> I have implemented the primitive version of this algorithm in Alpine Data
> Labs. Advanced features such as inner-mapper combiner are employed to reduce
> the network traffic in the shuffle phase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
