Re: Is RankingMetrics' NDCG implementation correct?

(cc'ing dev list also) I think a more general version of ranking metrics that allows arbitrary relevance scores could be useful. Ranking metrics are applicable to other settings like search or other learning-to-rank use cases, so it should be a little more generic than pure recommender settings.

Re: Is RankingMetrics' NDCG implementation correct?

Thanks for the clarification and the relevant links. I overlooked the comments explicitly saying that the relevance is binary. I understand that the label is not a relevance, but I have been, and I think many people are using the label as relevance in the implicit feedback context where the

Re: Is RankingMetrics' NDCG implementation correct?

Yes, relevance is always 1. The label is not a relevance score so don't think it's valid to use it as such. On Mon, Sep 19, 2016 at 4:42 AM, Jong Wook Kim wrote: > Hi, > > I'm trying to evaluate a recommendation model, and found that Spark and > Rival give different results,

Re: Is RankingMetrics' NDCG implementation correct?

The PR already exists for adding RankingEvaluator to ML - https://github.com/apache/spark/pull/12461. I need to revive and review it. DB, your review would be welcome too (and also on https://github.com/apache/spark/issues/12574 which has implications for the semantics of ranking metrics in the

Re: Is RankingMetrics' NDCG implementation correct?

Hi Jong, I think the definition from Kaggle is correct. I'm working on implementing ranking metrics in Spark ML now, but the timeline is unknown. Feel free to submit a PR for this in MLlib. Thanks. Sincerely, DB Tsai -- Web:

Is RankingMetrics' NDCG implementation correct?

Hi, I'm trying to evaluate a recommendation model, and found that Spark and Rival give different results, and it seems that Rival's one is what Kaggle defines :