Re: Own recommender
spark-itemsimilarity uses SimialrityAnalysis,cooccurrence, which does the majority of the work. Examples of how to use it and handle I/O are in the CLI driver mahout/spark/src/…/drivers/ItemSimilarityDriver.scala. All of this is available as a library. The nature of Spark makes creating map-reduce and other parallel ops almost transparent to the programmer and therefore the code is easier to use as a lib. Calling Scala from Java is a bit tricky since Scala renames some things for access by Java—things that don’t exist in Java. Calling Java from Scala is very easy. They are mixed in the Mahout codebase. On Jan 21, 2015, at 7:49 AM, Ted Dunning ted.dunn...@gmail.com wrote: Juanjo, Using the Taste components, it will be almost impossible to get really high performance. For that, using the itemsimilarity program to feed a search index is the best alternative. The scala version of the itemsimilarity program is available in Scala and could be called fairly easily as a library. The older map-reduce version is not easily used as a library. On Wed, Jan 21, 2015 at 2:20 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi Manuel, Thanks for the update. I'm using Mahout in a simple Java application myself. Following Ted's comment a few posts back, I was just concerned about the performance. Is performance the only concern when using Taste or the algorithm's implementation has also been improved in the current implementations accessible via CLI. Thanks. On Wed, Jan 21, 2015 at 10:14 AM, Manuel Blechschmidt manuel.blechschm...@gmx.de wrote: Hi Juan, On 21.01.2015, at 11:05, Juanjo Ramos jjar...@gmail.com wrote: Thanks Pat for the resources. Please correct me if I'm wrong but all Mahout's latest tools are command line tools only, is that correct? Yes, this is kind of correct. All tools are command line based. There was some development for an interactive console similar to R https://issues.apache.org/jira/browse/MAHOUT-1489 https://issues.apache.org/jira/browse/MAHOUT-1489 I was wondering if there is a library with the latest implementation that can be used in a Java or Scala project? The following project uses Mahout in a full blown simple Java EE application: https://github.com/ManuelB/facebook-recommender-demo https://github.com/ManuelB/facebook-recommender-demo Best. /Manuel -- Manuel Blechschmidt Twitter: http://twitter.com/Manuel_B
Re: Own recommender
Juanjo, Using the Taste components, it will be almost impossible to get really high performance. For that, using the itemsimilarity program to feed a search index is the best alternative. The scala version of the itemsimilarity program is available in Scala and could be called fairly easily as a library. The older map-reduce version is not easily used as a library. On Wed, Jan 21, 2015 at 2:20 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi Manuel, Thanks for the update. I'm using Mahout in a simple Java application myself. Following Ted's comment a few posts back, I was just concerned about the performance. Is performance the only concern when using Taste or the algorithm's implementation has also been improved in the current implementations accessible via CLI. Thanks. On Wed, Jan 21, 2015 at 10:14 AM, Manuel Blechschmidt manuel.blechschm...@gmx.de wrote: Hi Juan, On 21.01.2015, at 11:05, Juanjo Ramos jjar...@gmail.com wrote: Thanks Pat for the resources. Please correct me if I'm wrong but all Mahout's latest tools are command line tools only, is that correct? Yes, this is kind of correct. All tools are command line based. There was some development for an interactive console similar to R https://issues.apache.org/jira/browse/MAHOUT-1489 https://issues.apache.org/jira/browse/MAHOUT-1489 I was wondering if there is a library with the latest implementation that can be used in a Java or Scala project? The following project uses Mahout in a full blown simple Java EE application: https://github.com/ManuelB/facebook-recommender-demo https://github.com/ManuelB/facebook-recommender-demo Best. /Manuel -- Manuel Blechschmidt Twitter: http://twitter.com/Manuel_B
Re: Own recommender
Hi Juan, On 21.01.2015, at 11:05, Juanjo Ramos jjar...@gmail.com wrote: Thanks Pat for the resources. Please correct me if I'm wrong but all Mahout's latest tools are command line tools only, is that correct? Yes, this is kind of correct. All tools are command line based. There was some development for an interactive console similar to R https://issues.apache.org/jira/browse/MAHOUT-1489 https://issues.apache.org/jira/browse/MAHOUT-1489 I was wondering if there is a library with the latest implementation that can be used in a Java or Scala project? The following project uses Mahout in a full blown simple Java EE application: https://github.com/ManuelB/facebook-recommender-demo https://github.com/ManuelB/facebook-recommender-demo Best. /Manuel -- Manuel Blechschmidt Twitter: http://twitter.com/Manuel_B
Re: Own recommender
Thanks Pat for the resources. Please correct me if I'm wrong but all Mahout's latest tools are command line tools only, is that correct? I was wondering if there is a library with the latest implementation that can be used in a Java or Scala project? Best. On Mon, Jan 19, 2015 at 9:51 PM, Pat Ferrel p...@occamsmachete.com wrote: I guess I’ll put a page on the mahout site. For now some references: small free book here, which talks about the general idea: https://www.mapr.com/practical-machine-learning preso, which talks about mixing actions or other indicators: http://occamsmachete.com/ml/2014/10/07/creating-a-unified-recommender-with-mahout-and-a-search-engine/ two blog posts: http://occamsmachete.com/ml/2014/08/11/mahout-on-spark-whats-new-in-recommenders/http://occamsmachete.com/ml/2014/09/09/mahout-on-spark-whats-new-in-recommenders-part-2/ mahout docs: http://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html On Jan 19, 2015, at 3:02 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi Pat, Do you know if there is any tutorial for the Scala recommender code? Mahout's site keeps pointing here: http://mahout.apache.org/users/recommender/userbased-5-minutes.html Thanks. On Sat, Jan 17, 2015 at 4:24 PM, Pat Ferrel p...@occamsmachete.com wrote: The newest recommender code runs on the new Scala R-like DSL. It is cooccurrence based and supports only LLR. LLR is used to downsample cooccurrences comparing all pairs of items. I’ve done fairly careful offline testing of all the similarity methods of Mahout’s hadoop and in-memory recommenders and LLR was a clear winner. However if you have something new you want to try, look at the Scala SimilarityAnalysis class. For runtime efficiency it first calculates cooccurrences by performing [AA’] then calculating LLR on elements by row and downsampling in one step. You could look at some other similarity method for downsampling there. On Jan 16, 2015, at 12:44 AM, ARROYO MANCEBO David david.arr...@altran.com wrote: Any idea, Ted? :) -Mensaje original- De: Ted Dunning [mailto:ted.dunn...@gmail.com] Enviado el: jueves, 15 de enero de 2015 20:05 Para: user@mahout.apache.org Asunto: Re: Own recommender The old Taste code is not the state of the art. User-based recommenders built on that will be slow. On Thu, Jan 15, 2015 at 7:10 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi David, You implement your custom algorithm and create your own class that implements the UserSimilarity interface. When you then instantiate your User-Based recommender, just pass your custom class for the UserSimilarity parameter. Best. On Thu, Jan 15, 2015 at 1:11 PM, ARROYO MANCEBO David david.arr...@altran.com wrote: Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
Re: Own recommender
Hi Juanjo, On 21.01.2015, at 11:20, Juanjo Ramos jjar...@gmail.com wrote: Hi Manuel, Thanks for the update. I'm using Mahout in a simple Java application myself. Following Ted's comment a few posts back, I was just concerned about the performance. So if you have more than around 300.000 preferences for around 10.000 users it will take some seconds to generate recommendations for users with a lot of preferences. Normally what you do is just down sample the preferences that a certain user has to make it faster. When I understood LLR correctly than it is a smart way of doing this downsampling. Another approach would be using a model based approach: https://mahout.apache.org/users/recommender/matrix-factorization.html https://mahout.apache.org/users/recommender/matrix-factorization.html The following class contains an example: https://github.com/ManuelB/facebook-recommender-demo/blob/master/src/main/java/de/apaxo/bedcon/AnimalFoodRecommender.java https://github.com/ManuelB/facebook-recommender-demo/blob/master/src/main/java/de/apaxo/bedcon/AnimalFoodRecommender.java Is performance the only concern when using Taste or the algorithm's implementation has also been improved in the current implementations accessible via CLI. Thanks. /Manuel -- Manuel Blechschmidt Twitter: http://twitter.com/Manuel_B
Re: Own recommender
Hi Manuel, Thanks for the update. I'm using Mahout in a simple Java application myself. Following Ted's comment a few posts back, I was just concerned about the performance. Is performance the only concern when using Taste or the algorithm's implementation has also been improved in the current implementations accessible via CLI. Thanks. On Wed, Jan 21, 2015 at 10:14 AM, Manuel Blechschmidt manuel.blechschm...@gmx.de wrote: Hi Juan, On 21.01.2015, at 11:05, Juanjo Ramos jjar...@gmail.com wrote: Thanks Pat for the resources. Please correct me if I'm wrong but all Mahout's latest tools are command line tools only, is that correct? Yes, this is kind of correct. All tools are command line based. There was some development for an interactive console similar to R https://issues.apache.org/jira/browse/MAHOUT-1489 https://issues.apache.org/jira/browse/MAHOUT-1489 I was wondering if there is a library with the latest implementation that can be used in a Java or Scala project? The following project uses Mahout in a full blown simple Java EE application: https://github.com/ManuelB/facebook-recommender-demo https://github.com/ManuelB/facebook-recommender-demo Best. /Manuel -- Manuel Blechschmidt Twitter: http://twitter.com/Manuel_B
Re: Own recommender
I guess I’ll put a page on the mahout site. For now some references: small free book here, which talks about the general idea: https://www.mapr.com/practical-machine-learning preso, which talks about mixing actions or other indicators: http://occamsmachete.com/ml/2014/10/07/creating-a-unified-recommender-with-mahout-and-a-search-engine/ two blog posts: http://occamsmachete.com/ml/2014/08/11/mahout-on-spark-whats-new-in-recommenders/http://occamsmachete.com/ml/2014/09/09/mahout-on-spark-whats-new-in-recommenders-part-2/ mahout docs: http://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html On Jan 19, 2015, at 3:02 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi Pat, Do you know if there is any tutorial for the Scala recommender code? Mahout's site keeps pointing here: http://mahout.apache.org/users/recommender/userbased-5-minutes.html Thanks. On Sat, Jan 17, 2015 at 4:24 PM, Pat Ferrel p...@occamsmachete.com wrote: The newest recommender code runs on the new Scala R-like DSL. It is cooccurrence based and supports only LLR. LLR is used to downsample cooccurrences comparing all pairs of items. I’ve done fairly careful offline testing of all the similarity methods of Mahout’s hadoop and in-memory recommenders and LLR was a clear winner. However if you have something new you want to try, look at the Scala SimilarityAnalysis class. For runtime efficiency it first calculates cooccurrences by performing [AA’] then calculating LLR on elements by row and downsampling in one step. You could look at some other similarity method for downsampling there. On Jan 16, 2015, at 12:44 AM, ARROYO MANCEBO David david.arr...@altran.com wrote: Any idea, Ted? :) -Mensaje original- De: Ted Dunning [mailto:ted.dunn...@gmail.com] Enviado el: jueves, 15 de enero de 2015 20:05 Para: user@mahout.apache.org Asunto: Re: Own recommender The old Taste code is not the state of the art. User-based recommenders built on that will be slow. On Thu, Jan 15, 2015 at 7:10 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi David, You implement your custom algorithm and create your own class that implements the UserSimilarity interface. When you then instantiate your User-Based recommender, just pass your custom class for the UserSimilarity parameter. Best. On Thu, Jan 15, 2015 at 1:11 PM, ARROYO MANCEBO David david.arr...@altran.com wrote: Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
Re: Own recommender
Hi Pat, Do you know if there is any tutorial for the Scala recommender code? Mahout's site keeps pointing here: http://mahout.apache.org/users/recommender/userbased-5-minutes.html Thanks. On Sat, Jan 17, 2015 at 4:24 PM, Pat Ferrel p...@occamsmachete.com wrote: The newest recommender code runs on the new Scala R-like DSL. It is cooccurrence based and supports only LLR. LLR is used to downsample cooccurrences comparing all pairs of items. I’ve done fairly careful offline testing of all the similarity methods of Mahout’s hadoop and in-memory recommenders and LLR was a clear winner. However if you have something new you want to try, look at the Scala SimilarityAnalysis class. For runtime efficiency it first calculates cooccurrences by performing [AA’] then calculating LLR on elements by row and downsampling in one step. You could look at some other similarity method for downsampling there. On Jan 16, 2015, at 12:44 AM, ARROYO MANCEBO David david.arr...@altran.com wrote: Any idea, Ted? :) -Mensaje original- De: Ted Dunning [mailto:ted.dunn...@gmail.com] Enviado el: jueves, 15 de enero de 2015 20:05 Para: user@mahout.apache.org Asunto: Re: Own recommender The old Taste code is not the state of the art. User-based recommenders built on that will be slow. On Thu, Jan 15, 2015 at 7:10 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi David, You implement your custom algorithm and create your own class that implements the UserSimilarity interface. When you then instantiate your User-Based recommender, just pass your custom class for the UserSimilarity parameter. Best. On Thu, Jan 15, 2015 at 1:11 PM, ARROYO MANCEBO David david.arr...@altran.com wrote: Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
RE: Own recommender
Any idea, Ted? :) -Mensaje original- De: Ted Dunning [mailto:ted.dunn...@gmail.com] Enviado el: jueves, 15 de enero de 2015 20:05 Para: user@mahout.apache.org Asunto: Re: Own recommender The old Taste code is not the state of the art. User-based recommenders built on that will be slow. On Thu, Jan 15, 2015 at 7:10 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi David, You implement your custom algorithm and create your own class that implements the UserSimilarity interface. When you then instantiate your User-Based recommender, just pass your custom class for the UserSimilarity parameter. Best. On Thu, Jan 15, 2015 at 1:11 PM, ARROYO MANCEBO David david.arr...@altran.com wrote: Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
Re: Own recommender
The old Taste code is not the state of the art. User-based recommenders built on that will be slow. On Thu, Jan 15, 2015 at 7:10 AM, Juanjo Ramos jjar...@gmail.com wrote: Hi David, You implement your custom algorithm and create your own class that implements the UserSimilarity interface. When you then instantiate your User-Based recommender, just pass your custom class for the UserSimilarity parameter. Best. On Thu, Jan 15, 2015 at 1:11 PM, ARROYO MANCEBO David david.arr...@altran.com wrote: Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
Re: Own recommender
Hi David, You implement your custom algorithm and create your own class that implements the UserSimilarity interface. When you then instantiate your User-Based recommender, just pass your custom class for the UserSimilarity parameter. Best. On Thu, Jan 15, 2015 at 1:11 PM, ARROYO MANCEBO David david.arr...@altran.com wrote: Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
Own recommender
Hi folks, How I can start to build my own recommender system in apache mahout with my personal algorithm? I need a custom UserSimilarity. Maybe a subclass from UserSimilarity like PearsonCorrelationSimilarity? Thanks Regards :)
