How to Integrate Spark mllib Streaming Training Models To Spark Structured Streaming

Spark mllib library Streaming Training models work with DStream. So is there any way to use them with spark structured streaming.

Re: MLLib + Streaming

Thanks, Guru. After reading the implementation of StreamingKMean, StreamingLinearRegressionWithSGD and StreamingLogisticRegressionWithSGD, I reached the same conclusion. But unfortunately, this distinction between true online learning and offline learning are implied in the documentation and I

Re: MLLib + Streaming

Guru:This is a really great response. Thanks for taking the time to explain all of this. Helpful for me too. -- Chris Miller On Sun, Mar 6, 2016 at 1:54 PM, Guru Medasani wrote: > Hi Lan, > > Streaming Means, Linear Regression and Logistic Regression support online > machine

MLLib + Streaming

Hi, there I hope someone can clarify this for me. It seems that some of the MLlib algorithms such as KMean, Linear Regression and Logistics Regression have a Streaming version, which can do online machine learning. But does that mean other MLLib algorithm cannot be used in Spark streaming

Re: MLlib + Streaming

Hi Fernando, There’s currently no streaming ALS in Spark. I’m exploring a streaming singular value decomposition (JIRA) based on this paper (http://www.stat.osu.edu/~dmsl/thinSVDtracking.pdf), which might be one way to think about it. There has also been some cool recent work explicitly on

Re: MLlib + Streaming

We have streaming linear regression (since v1.1) and k-means (v1.2) in MLlib. You can check the user guide: http://spark.apache.org/docs/latest/mllib-linear-methods.html#streaming-linear-regression http://spark.apache.org/docs/latest/mllib-clustering.html#streaming-clustering -Xiangrui On Tue,

Re: MLlib + Streaming

Hey Xiangrui, Is there any plan to have a streaming compatible ALS version? Or if it's currently doable, is there any example? On Tue, Dec 23, 2014 at 4:31 PM, Xiangrui Meng men...@gmail.com wrote: We have streaming linear regression (since v1.1) and k-means (v1.2) in MLlib. You can