Question about evaluating a Recommender System

2013-05-07 Thread Zhongduo Lin
Hi All, I am using the Mahout to build a user-based recommender system (RS). The evaluation method I am using is AverageAbsoluteDifferenceRecommenderEvaluator, which according to the "Mahout in Action" randomly sets aside some existing preference and calculate the difference between the predi

Re: Question about evaluating a Recommender System

2013-05-08 Thread Zhongduo Lin
e to Mahout, it is going to give me a really nice result because most of the preference is 1. So is there any other metric to evaluate this? Any help will be highly appreciated. Best Regards, Jimmy Zhongduo Lin (Jimmy) MASc candidate in ECE department University of Toronto On 2013-05-08

Re: Question about evaluating a Recommender System

2013-05-08 Thread Zhongduo Lin
do one of those two things. Try mean average precision for a metric that is not directly related to the prediction values. On Wed, May 8, 2013 at 2:45 PM, Zhongduo Lin wrote: Thank you for your reply. I think the evaluation process involves randomly choosing the evaluation proportion. The

Re: Question about evaluating a Recommender System

2013-05-08 Thread Zhongduo Lin
can't predict item 4 in that case. that shows the weakness of neighborhood approaches for sparse data. That's pretty much the story -- it's all working correctly. Maybe you should not use this approach. On Wed, May 8, 2013 at 4:00 PM, Zhongduo Lin wrote: Thank you for the quick res

Re: Question about evaluating a Recommender System

2013-05-08 Thread Zhongduo Lin
m so as to not favor one algorithm. I think you are just arguing that the algorithm you are using isn't good for your data -- so just don't use it. Is that not the answer? I don't know what you mean by the mapping algorithm. On Wed, May 8, 2013 at 4:17 PM, Zhongduo Lin wrote: Thank

Re: Question about evaluating a Recommender System

2013-05-08 Thread Zhongduo Lin
This accounts for why a neighborhood size of 2 always gives me the best result. Thank you! Best Regards, Jimmy Zhongduo Lin (Jimmy) MASc candidate in ECE department University of Toronto On 2013-05-08 2:40 PM, Alejandro Bellogin Kouki wrote: AFAIK, the recommender would predict a NaN, which

Re: Question about evaluating a Recommender System

2013-05-08 Thread Zhongduo Lin
I see. Thank you for your information! Any idea about evaluating the method of mapping inferred preference to a smaller scale with Mahout? Best Regards, Jimmy Zhongduo Lin (Jimmy) MASc candidate in ECE department University of Toronto On 2013-05-08 3:32 PM, Sean Owen wrote: Ah, yes that&#