Re: MLib: How to set preferences for ALS implicit feedback in Collaborative Filtering?

2015-01-20 Thread Xiangrui Meng
The assumption of implicit feedback model is that the unobserved ratings are more likely to be negative. So you may want to add some negatives for evaluation. Otherwise, the input ratings are all 1 and the test ratings are all 1 as well. The baseline predictor, which uses the average rating (that i

Re: MLib: How to set preferences for ALS implicit feedback in Collaborative Filtering?

2015-01-16 Thread Sean Owen
On Fri, Jan 16, 2015 at 9:58 AM, Zork Sail wrote: > And then train ALSL: > > val model = ALS.trainImplicit(ratings, rank, numIter) > > I get RMSE 0.9, which is a big error in case of preferences taking 0 or 1 > value: This is likely the problem. RMSE is not an appropriate evaluation metric w

MLib: How to set preferences for ALS implicit feedback in Collaborative Filtering?

2015-01-16 Thread Zork
e: 0.7302343904091481 The best model improves the baseline by -Infinity%. Which is still a big error, I guess. Also I get strange baseline improvement where baseline model is simply mean (1). -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/M

MLib: How to set preferences for ALS implicit feedback in Collaborative Filtering?

2015-01-16 Thread Julian Ricardo
ew this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLib-How-to-set-preferences-for-ALS-implicit-feedback-in-Collaborative-Filtering-tp21185.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --

MLib: How to set preferences for ALS implicit feedback in Collaborative Filtering?

2015-01-16 Thread Zork Sail
I am trying to use Spark MLib ALS with implicit feedback for collaborative filtering. Input data has only two fields `userId` and `productId`. I have **no product ratings**, just info on what products users have bought, that's all. So to train ALS I use: def trainImplicit(ratings: RDD[Rating],