Not quite, the evaluation considers every item in the test set to be
"good", but you would and should fix the test set size across
evaluations for this reason. You are right that there is a big
assumption there -- that everything in the test set is good. You have
to believe your test split process supports that assumption.
On Thu, Jan 24, 2013 at 6:37 PM, Zia mel <ziad.kame...@gmail.com> wrote:
> In general boolean recommender will get higher precision than using a
> recommender with preferences, since the boolean considers every item
> as good which is not true! So is there a way to make a realistic
> measure from boolean ? For example, does dividing the precison by 2
> makes sense since we get high precison using boolean?
> Thanks
>
>
>
> On Wed, Jan 23, 2013 at 3:49 PM, Ted Dunning <ted.dunn...@gmail.com> wrote:
>> LLR should not be used to indicate proximity, but rather simply as a value
>> to compare to a threshold.
>>
>> On Thu, Jan 24, 2013 at 1:45 AM, Zia mel <ziad.kame...@gmail.com> wrote:
>>
>>> OK . The TanimotoCoefficientSimilarity and LogLikelihoodSimilarity
>>> used in MIA page 54 and 55 provide a score, so it seems they were not
>>> using a Boolean recommender , something like code 1 maybe? Thanks
>>>
>>> On Tue, Jan 22, 2013 at 10:42 AM, Sean Owen <sro...@gmail.com> wrote:
>>> > Yes any metric that concerns estimated value vs real value can't be
>>> > used since all values are 1. Yes, when you use the non-boolean version
>>> > with boolean data you always get 1. When you use the boolean version
>>> > with boolean data you will get nonsense since the output of this
>>> > recommender is not an estimated rating at all.
>>> >
>>> > On Tue, Jan 22, 2013 at 4:40 PM, Zia mel <ziad.kame...@gmail.com> wrote:
>>> >> I got 0 when I used GenericUserBasedRecommender in code 2 but when
>>> >> using GenericBooleanPrefUserBasedRecommender score was not 0 . I
>>> >> repeat the test with different data and again I got some results.
>>> >> Moreover , when I use
>>> >> DataModel model = new FileDataModel(new File("ua.base"));
>>> >> in code 2, the MAE score was higher.
>>> >>
>>> >> When you say RMSE can't be used with boolean data, I assume MAE also
>>> >> can't be used?
>>> >>
>>> >> Thanks !
>>> >>
>>> >> On Tue, Jan 22, 2013 at 10:08 AM, Sean Owen <sro...@gmail.com> wrote:
>>> >>> RMSE can't
>>> >>> be used with boolean data.
>>>
