Thanks for your response Burak it was very helpful. I am noticing that if I run PCA before KMeans that the KMeans algorithm will actually take longer to run than if I had just run KMeans without PCA. I was hoping that by using PCA first it would actually speed up the KMeans algorithm.
I have followed the steps you've outlined but Im wondering if I need to cache/persist the RDD[Vector] rows of the RowMatrix returned after multiplying. Something like: val newData: RowMatrix = data.multiply(bcPrincipalComponents.value) val cachedRows = newData.rows.persist() KMeans.run(cachedRows) cachedRows.unpersist() It doesnt seem intuitive to me that a smaller dimensional version of my data set would take longer for KMeans... unless Im missing something? Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-run-kmeans-after-pca-tp14473p15409.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
