Hi all,
I would like to evaluate the IR statistics of my item based recommender with
the GenericRecommenderIRStatsEvaluator. However, precision and recall are 0.0
for each user, as I can see from my logs.
Possible, that my data are too sparse?
Regards,
Mirko
My data:
936080 preferences
162291 users
109661 items
How I call the evaluator:
IRStatistics irStats = new
GenericRecommenderIRStatsEvaluator().evaluate(recBuilder, null, dm, null, 10,
Double.NaN, 0.1);
log.debug("Precision: "+irStats.getPrecision());
log.debug("Recall: "+irStats.getRecall());
My RecommenderBuilder implementation:
public class SimpleRecommenderBuilder implements RecommenderBuilder{
@Override
public Recommender buildRecommender(DataModel dm) throws TasteException
{
CachingItemSimilarity itemSim = new
CachingItemSimilarity(new LogLikelihoodSimilarity(dm), dm);
ItemBasedRecommender itemRecommender = new
GenericItemBasedRecommender(dm, itemSim);
return itemRecommender;
}
}
My logs:
[...]
2010-02-24 19:50:30,517 Processed 160000 users 2010-02-24 19:50:30,521
Processed 162291 users 2010-02-24 19:50:35,473 Retrieving new recommendations
for user ID '-85512851039269613' 2010-02-24 19:50:35,473 Recommending items
for user ID '-85512851039269613' 2010-02-24 19:50:59,995 Recommendations are:
[RecommendedItem[item:454067076717825534, value:0.005567929],
RecommendedItem[item:-2322998021761419502, value:0.005567929],
RecommendedItem[item:612773236668408
7105, value:0.005567929], RecommendedItem[item:-3981461966915310408,
value:0.005567929], RecommendedItem
[item:-5313445508550677921, value:0.005567929],
RecommendedItem[item:-1832988160817640861, value:0.00556
7929], RecommendedItem[item:2911886255972263265, value:0.005567929],
RecommendedItem[item:26217060620275
57406, value:0.005567929], RecommendedItem[item:-3536836517010980436,
value:0.005567929], RecommendedIte
m[item:-6721918598917466000, value:0.005567929]]
2010-02-24 19:50:59,995 Evaluated with user -85512851039269613 in 29782ms
2010-02-24 19:50:59,995 Precision/recall/fall-out: 0.0 / 0.0 /
9.119405351853041E-5
2010-02-24 19:51:00,061 Processed 10000 users
2010-02-24 19:51:00,074 Processed 20000 users
2010-02-24 19:51:00,088 Processed 30000 users
2010-02-24 19:51:00,101 Processed 40000 users
2010-02-24 19:51:00,118 Processed 50000 users
2010-02-24 19:51:00,131 Processed 60000 users
2010-02-24 19:51:00,145 Processed 70000 users
2010-02-24 19:51:00,159 Processed 80000 users
2010-02-24 19:51:00,180 Processed 90000 users
2010-02-24 19:51:00,195 Processed 100000 users
2010-02-24 19:51:00,209 Processed 110000 users
2010-02-24 19:51:00,224 Processed 120000 users
2010-02-24 19:51:00,239 Processed 130000 users
2010-02-24 19:51:00,254 Processed 140000 users
2010-02-24 19:51:00,284 Processed 150000 users
2010-02-24 19:51:00,298 Processed 160000 users
2010-02-24 19:51:00,302 Processed 162291 users
2010-02-24 19:51:05,254 Retrieving new recommendations for user ID
'124198514856103961'
2010-02-24 19:51:05,254 Recommending items for user ID '124198514856103961'
2010-02-24 19:52:00,228 Recommendations are:
[RecommendedItem[item:6710022688720561421, value:0.005021
555], RecommendedItem[item:-9152989315951376207, value:0.00500819],
RecommendedItem[item:-10610960939022
57025, value:0.00500819], RecommendedItem[item:-3145875224888183517,
value:0.0047857966], RecommendedIte
m[item:4764651006754212795, value:0.004762556],
RecommendedItem[item:6530066880626179760, value:0.004762
132], RecommendedItem[item:5360311107615737665, value:0.004639993],
RecommendedItem[item:162168311040648
6210, value:0.004639993], RecommendedItem[item:-6562592997555071847,
value:0.004632245], RecommendedItem
[item:8239589664710936158, value:0.0046123066]]
2010-02-24 19:52:00,228 Evaluated with user 124198514856103961 in 60233ms
2010-02-24 19:52:00,228 Precision/recall/fall-out: 0.0 / 0.0 /
9.119404271753251E-5
[...]
