Hi Saikat, I wrote my master thesis about evaluating recommender in real world examples:
https://source.apaxo.de/svn/semrecsys/trunk/doc/2010-Manuel-Blechschmidt-730786-EvalRecSys.pdf So what you are going to do is current research. This means that there are currently not a lot of experiences. In 2009 there was an online evaluation challenges which was part of ECML PKDD. 2009 ECML PKDD Discovery Challeng: Online Tag Recommendations. http:// www.kde.cs.uni-kassel.de/ws/dc09/online. Version: 2009, Checked: 2011-04-23 You will have to run all your recommenders in parallel to figure out which one is the best one for optimizing business goals. I founded a company which is developing the described technology. I am currently searching for a project starting in July 2012 where I can try this. So if you are interested in hiring my feel free to send me a personal message. /Manuel On 10.04.2012, at 17:41, Saikat Kanjilal wrote: > > Hi everyone,We're looking at building out some clustering and classification > algorithms using mahout and one of the things we're also looking at doing is > to build performance metrics around each of these algorithms, as we go down > the path of choosing the best model in an iterative closed feedback loop > (i.e. our business users manipulate weights for each attribute for our > feature vectors->we use these changes to regenerate an asynchronous model > using the appropriate clustering/classification algorithms and then replenish > our online component with this newly recalculated data for fresh > recommendations). So our end goal is to have a basket of algorithms and use > a set of performance metrics to pick and choose the right algorithm on the > fly. I was wondering if anyone has done this type of analysis before and if > so are there approaches that have worked well and approaches that haven't > when it comes to measuring the "quality" of each of the recommendation > algorithms. > Regards -- Manuel Blechschmidt CTO - Apaxo GmbH blechschm...@apaxo.de http://www.apaxo.de Weinbergstr. 16 14469 Potsdam Telefon +49 (0)6204 9180 593 Fax +49 (0)6204 9180 594 Mobil: +49 173/6322621 Skype: Manuel_B86 Twitter: http://twitter.com/Manuel_B Sitz der Gesellschaft: Viernheim Handelsregister HRB 87159 Ust-IdNr. DE261368874 Amtsgericht Darmstadt Geschäftsführer Friedhelm Scharhag
