Hi all,
I'm using GenericRecommenderIRStatsEvaluator to evaluate IRStatistics of a Collaborative Filtering recommender built on top of GenericBooleanPrefDataModel. I've noticed that precision and recall are affected by the order in which itemIDs are associated with a userID in the given GenericBooleanPrefDataModel, which, I think, is not correct. I've put together a simple test case [1] based on the "ratings.csv" file from the MovieLens dataset [2] to demonstrate the problem. The test case illustrates the problem for the GenericBooleanPrefDataModel, but I think the problem may also show up for non-boolean DataModels where different items associated to a user have the same preference value. It would be helpful if someone from the Dev team can confirm that this behaviour is not correct, or explain why we have this. If you agree that this behaviour is not correct, I think there is an easy solution. Thank you in advance! Best Regards, Marco [1] https://gist.github.com/marcosbodio/0f3647429ac17fcec2663ffd60e545a7 [2] http://files.grouplens.org/datasets/movielens/ml-latest-small.zip Best Regards, Marco --- Marco Luca Sbodio, Ph.D. Research Staff Member IBM Research, Ireland IBM Ireland Limited registered in Ireland under company number 16226 Registered Office: IBM House, Shelbourne Road, Ballsbridge, Dublin 4
