Join on Spark too slow.

2015-04-09 Thread Kostas Kloudas
Hello guys, I am trying to run the following dummy example for Spark, on a dataset of 250MB, using 5 machines with 10GB RAM each, but the join seems to be taking too long ( 2hrs). I am using Spark 0.8.0 but I have also tried the same example on more recent versions, with the same results. Do

Re: RMSE in MovieLensALS increases or stays stable as iterations increase.

2014-11-27 Thread Kostas Kloudas
RMSE = 1.467238593043199 I'm not sure what the difference is. I looked at your modifications and they seem very similar. Is it the data you're using? On Wed, Nov 26, 2014 at 3:34 PM, Kostas Kloudas kklou...@gmail.com wrote: For the training I am using the code in the MovieLensALS example

RMSE in MovieLensALS increases or stays stable as iterations increase.

2014-11-26 Thread Kostas Kloudas
Hi all, I am getting familiarized with Mllib and a thing I noticed is that running the MovieLensALS example on the movieLens dataset for increasing number of iterations does not decrease the rmse. The results for 0.6% training set and 0.4% test are below. For training set to 0.8%, the

Re: RMSE in MovieLensALS increases or stays stable as iterations increase.

2014-11-26 Thread Kostas Kloudas
converges - but not necessarily to decrease. On Wed, Nov 26, 2014 at 1:57 PM, Kostas Kloudas kklou...@gmail.com mailto:kklou...@gmail.com wrote: Hi all, I am getting familiarized with Mllib and a thing I noticed is that running the MovieLensALS example on the movieLens dataset

Re: RMSE in MovieLensALS increases or stays stable as iterations increase.

2014-11-26 Thread Kostas Kloudas
On Nov 26, 2014, at 2:41 PM, Sean Owen so...@cloudera.com wrote: How are you computing RMSE? and how are you training the model -- not with trainImplicit right? I wonder if you are somehow optimizing something besides RMSE. On Wed, Nov 26, 2014 at 2:36 PM, Kostas Kloudas kklou...@gmail.com wrote