[
https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426033#comment-15426033
]
Qian Huang commented on SPARK-17090:
------------------------------------
Hi, seth, are you working on it? If not, I am glad to try. I will generate
couple of datasets with different sizes, features and use them to run
linear/logistic regression with various value of aggregation depth to get a
expert formula.
> Make tree aggregation level in linear/logistic regression configurable
> ----------------------------------------------------------------------
>
> Key: SPARK-17090
> URL: https://issues.apache.org/jira/browse/SPARK-17090
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Seth Hendrickson
> Priority: Minor
>
> Linear/logistic regression use treeAggregate with default aggregation depth
> for collecting coefficient gradient updates to the driver. For high
> dimensional problems, this can case OOM error on the driver. We should make
> it configurable, perhaps via an expert param, so that users can avoid this
> problem if their data has many features.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
