Re: Evaluating boolean preference data sets

2011-07-21 Thread Marko Ciric
Also the evaluation could be done per user, and thus manually running multiple times per each user. Or simple defining a matrix with relevant items per each user.. On Jul 21, 2011 4:18 PM, "Marko Ciric" wrote: > Yes, there should exist an evaluation that allows you to pass which items > are releva

Re: Evaluating boolean preference data sets

2011-07-21 Thread Marko Ciric
Yes, there should exist an evaluation that allows you to pass which items are relevant. On the other hand, generally speaking, I am also trying to evaluate with having relevant items all chosen randomly. Maybe both implementations should exist. On 21 July 2011 15:59, Sean Owen wrote: > You mean,

Re: Evaluating boolean preference data sets

2011-07-21 Thread Sean Owen
You mean, have the user specify all items that are considered relevant? yes that could be useful. Do you have a patch in mind? Your analysis is correct, and I would not call it a bug. It's a symptom of how little information the evaluation has to work with here without ratings. It has to pick rand

Evaluating boolean preference data sets

2011-07-21 Thread Marko Ciric
Hi guys, I wonder if Mahout should have a "precision and recall" evaluator that calculates the relevant items data set without looking to the relevance threshold. This would be suitable for data sets with boolean preference nature. In addition, the relevant items can be removed from the training d