Re: Using IDF in CF recommender

2013-02-06 Thread Paulo Villegas
> The affect of downweighting the popular items is very similar to > removing them from recommendations so I still suspect precision will > go down using IDF. Obviously this can pretty easily be tested, I just > wondered if anyone had already done it. > > This brings up a problem with holdout bas

Re: Using IDF in CF recommender

2013-02-06 Thread Pat Ferrel
The affect of downweighting the popular items is very similar to removing them from recommendations so I still suspect precision will go down using IDF. Obviously this can pretty easily be tested, I just wondered if anyone had already done it. This brings up a problem with holdout based precisi

Re: Using IDF in CF recommender

2013-02-06 Thread Paulo Villegas
This results in no information for universally preferred items, which is indeed what I was looking for. It looks like this should also work for other values or explicit preferences--item prices, ratings, etc.. Intuition says this will result in a lower precision related cross validation measu

Re: Using IDF in CF recommender

2013-02-06 Thread Pat Ferrel
oops, forgot the log So... idf weighted preference value = item preference value * log (number of all items/number of users with specific item pref) items 1 0 0 users 1 0 0 1 1 0 freq

Re: Using IDF in CF recommender

2013-02-05 Thread Ted Dunning
On Tue, Feb 5, 2013 at 11:29 AM, Pat Ferrel wrote: > I think you meant: "Human relatedness decays much slower than item > popularity." > Yes. Oops. > So to make sure I understand the implications of using IDF… For > boolean/implicit preferences the sum of all prefs (after weighting) for a >

Using IDF in CF recommender

2013-02-05 Thread Pat Ferrel
I think you meant: "Human relatedness decays much slower than item popularity." So to make sure I understand the implications of using IDF… For boolean/implicit preferences the sum of all prefs (after weighting) for a single item over all users will always be 1 or 0. This no matter whether the