[
https://issues.apache.org/jira/browse/SPARK-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395274#comment-14395274
]
Apache Spark commented on SPARK-6705:
-------------------------------------
User 'oefirouz' has created a pull request for this issue:
https://github.com/apache/spark/pull/5301
> MLLIB ML Pipeline's Logistic Regression has no intercept term
> -------------------------------------------------------------
>
> Key: SPARK-6705
> URL: https://issues.apache.org/jira/browse/SPARK-6705
> Project: Spark
> Issue Type: Improvement
> Components: ML, MLlib
> Reporter: Omede Firouz
>
> Currently, the ML Pipeline's LogisticRegression.scala file does not allow
> setting whether or not to fit an intercept term. Therefore, the pipeline
> defers to LogisticRegressionWithLBFGS which does not use an intercept term.
> This makes sense from a performance point of view because adding an intercept
> term requires memory allocation.
> However, this is undesirable statistically, since the statistical default is
> usually to include an intercept term, and one needs to have a very strong
> reason for not having an intercept term.
> Explicitly modeling the intercept by adding a column of all 1s does not
> work because LogisticRegressionWithLBFGS forces column normalization, and a
> column of all 1s has 0 variance and so dividing by 0 kills it.
> We should open up the API for the ML Pipeline to explicitly allow controlling
> whether or not to fit an intercept.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
