Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/10639#issuecomment-173327510 @yanboliang The proposal looks okay to me. Some minor comments: 1. `WeightedLeastSquares` model contains extra content that we don't need in IRLS iterations, e.g., `diagInvAWA`. This could be a follow-up work because the API is currently private. 2. `reweightedFunction` -> `reweightFunc`. Does it need to operate on RDDs? We can simplify the API to `(Instance, Model) => (Double, Double)`. 3. `standardizeFeatures` and `standardizeLabel` only affect the regularization term. We don't need it to match `glm` in R. So let's remove them from the API. Turning them on would change the convergence of IRLS algorithm, which needs further discussion. QR is more stable than the normal equation solver for WLS. We should switch to it in the future. For general Lp regression, there is an interior-point method implemented in R's `quantreg` package.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org