Hi Bryan
Agreed. It is a single statement to print the centers once for *every*
streaming batch (4 secs) - remember this is in streaming mode and the
receiver has fresh data every batch. That is, as the model is trained
continuously so I expect the centroids to change with incoming streams (at
least until convergence)
But am seeing same centers always for the entire duration - ran the app for
several hours with a custom receiver.
Yes I am using the latestModel to predict using "labeled" test data. But
also like to know where my centers are
regards
Krishna
On Fri, Feb 19, 2016 at 10:18 AM, Bryan Cutler <cutl...@gmail.com> wrote:
> Could you elaborate where the issue is? You say calling
> model.latestModel.clusterCenters.foreach(println) doesn't show an updated
> model, but that is just a single statement to print the centers once..
>
> Also, is there any reason you don't predict on the test data like this?
>
> model.predictOnValues(testData.map(lp => (lp.label, lp.features))).print()
>
>
>
> On Thu, Feb 18, 2016 at 5:59 PM, ramach1776 <ram...@s1776.com> wrote:
>
>> I have streaming application wherein I train the model using a receiver
>> input
>> stream in 4 sec batches
>>
>> val stream = ssc.receiverStream(receiver) //receiver gets new data every
>> batch
>> model.trainOn(stream.map(Vectors.parse))
>> If I use
>> model.latestModel.clusterCenters.foreach(println)
>>
>> the value of cluster centers remain unchanged from the very initial value
>> it
>> got during first iteration (when the streaming app started)
>>
>> when I use the model to predict cluster assignment with a labeled input
>> the
>> assignments change over time as expected
>>
>> testData.transform {rdd =>
>> rdd.map(lp => (lp.label,
>> model.latestModel().predict(lp.features)))
>> }.print()
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/StreamingKMeans-does-not-update-cluster-centroid-locations-tp26275.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>>
>
