[ https://issues.apache.org/jira/browse/SPARK-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14232920#comment-14232920 ]
Xiangrui Meng commented on SPARK-4722: -------------------------------------- [~Arthur][ You can use `StreamingLinearRegression.model` to get the latest model. It may be expensive and unnecessary to make predictOn return a DStream of model weights. If you want to re-use the previously trained model, you save the last model coefficients in the first run and then set initial weights in the second run. > StreamingLinearRegression should return a DStream of weights when calling > trainOn > --------------------------------------------------------------------------------- > > Key: SPARK-4722 > URL: https://issues.apache.org/jira/browse/SPARK-4722 > Project: Spark > Issue Type: Improvement > Components: MLlib, Streaming > Reporter: Arthur Andres > Priority: Minor > Labels: mllib, regression, streaming > > When training a model with a stream of new data (Spark Streaming + Spark > Mlllib), the weights (and the other part of the regression model) update at > every iterations. > At the moment the only output we can get is the prediction when calling > predictOn (class StreamingLinearRegression) > It would be a nice improvement if trainOn would return a Dstream of weights > (and any other underlying model data) so we can access it and see it evolve. > At the moment they are only outputted in the log > For example this could then be saved so when reloading the application we can > access this information without having to train the model again. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org