[
https://issues.apache.org/jira/browse/SPARK-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365227#comment-15365227
]
Seth Hendrickson commented on SPARK-16404:
------------------------------------------
cc [~dbtsai] I looked in to using the @transient tag, but this prevents the
coefficients from being serialized and broadcast to the executors at all,
resulting in a {{NullPointerException}}. I am not sure of a way around this. I
can submit a patch utilizing the same strategy as in LoR later this week.
> LeastSquaresAggregator in Linear Regression serializes unnecessary data
> -----------------------------------------------------------------------
>
> Key: SPARK-16404
> URL: https://issues.apache.org/jira/browse/SPARK-16404
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Seth Hendrickson
>
> This is basically the same issue as
> [SPARK-16008|https://issues.apache.org/jira/browse/SPARK-16008], but for
> linear regression, where {{coefficients}} and {{featuresStd}} are
> unnecessarily serialized between stages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
