To Dmitriy's point, I agree ggplot is def the priority, The mahout plots are at this point are really just a POC, but at some point we may be want to integrate some data transformation features into the mahout plots classes so they're really more future work.
long story short: >OK. I'll read through the examples and try to do something with some data, >then do a ggplot and/or an angular plot on it (probably ggplot). >I'll do a quick tutorial. Then I'll reopen discussion on that Zeppelin issue >about weather we want to go ahead and add another interpreter. Souds Great. Thank you. ________________________________ From: Trevor Grant <trevor.d.gr...@gmail.com> Sent: Monday, May 16, 2016 5:49:17 PM To: Dmitriy Lyubimov Cc: Andrew Palumbo; Pat Ferrel; Suneel Marthi Subject: Re: Intro - Future Mahout - Zeppelin work I just signed up for dev, should i just reply all and cc dev or start a new thread? Trevor Grant Data Scientist https://github.com/rawkintrevo [https://avatars3.githubusercontent.com/u/5852441?v=3&s=400]<https://github.com/rawkintrevo> rawkintrevo (Trevor Grant) · GitHub<https://github.com/rawkintrevo> github.com rawkintrevo has 12 repositories written in Python, Batchfile, and R. Follow their code on GitHub. http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org "Fortunate is he, who is able to know the causes of things." -Virgil On Mon, May 16, 2016 at 4:46 PM, Dmitriy Lyubimov <dlie...@gmail.com<mailto:dlie...@gmail.com>> wrote: fwiw ggplot2 is pretty darn advanced:) i am a bit skeptical smile would have something that ggplot2 would not, the other way around is much more expected by me:) anyhow if ggplot2 and matplotlib are available in Zeppelin without major limitations, it sounds like Zeppelin should be an all around very nice venue then. On Mon, May 16, 2016 at 2:42 PM, Andrew Palumbo <ap....@outlook.com<mailto:ap....@outlook.com>> wrote: yeah we should probably move this over to dev@ sorry- answering a question from a couple emails back on the thread. If possible, I think it would be great to eventually have both (native mahout/smile plots and ggplot), since in the future we're going to be adding more visualization features rather than simple scatter plots etc that may not be covered by ggplot. That's why we were thinking about using angular and the pngs. But what youre saying in your last email would be great! Thank you! ________________________________ From: Trevor Grant <trevor.d.gr...@gmail.com<mailto:trevor.d.gr...@gmail.com>> Sent: Monday, May 16, 2016 5:33:12 PM To: Andrew Palumbo Cc: Pat Ferrel; Suneel Marthi; Dmitriy Lyubimov Subject: Re: Intro - Future Mahout - Zeppelin work I somehow replied to your last email without seeing it... OK. I'll read through the examples and try to do something with some data, then do a ggplot and/or an angular plot on it (probably ggplot). I'll do a quick tutorial. Then I'll reopen discussion on that Zeppelin issue about weather we want to go ahead and add another interpreter. Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org "Fortunate is he, who is able to know the causes of things." -Virgil On Mon, May 16, 2016 at 4:26 PM, Trevor Grant <trevor.d.gr...@gmail.com<mailto:trevor.d.gr...@gmail.com>> wrote: sorry for double email but are you thinking visualization should be a library internal to mahout or should we leverage zeppelins visualization capabilities? Also, should we move this discussion to dev? tg Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org "Fortunate is he, who is able to know the causes of things." -Virgil On Mon, May 16, 2016 at 4:14 PM, Andrew Palumbo <ap....@outlook.com<mailto:ap....@outlook.com>> wrote: Sorry- to be a little more clear, Part of what we're trying to is to get the new plotting features integrated with Zeppelin. We plan on adding more advanced plotting. ________________________________ From: Andrew Palumbo <ap....@outlook.com<mailto:ap....@outlook.com>> Sent: Monday, May 16, 2016 5:04:49 PM To: Pat Ferrel; Trevor Grant Cc: Suneel Marthi; Dmitriy Lyubimov Subject: Re: Intro - Future Mahout - Zeppelin work Awesome! most of the hard work was done by Dmitriy[😊] , I've just reworked it a couple of times to keep up with spark's refactoring. I think that you will also need to include: mahout-spark_2.10-0.12.1-SNAPSHOT-dependency-reduced.jar For the new plotting features that we're working on. the plotting is still a work in progress, and the grid and surface plots are not working properly. The plots are swing based and can currently be exported as PNGs. There are a few examples on the closed PR: https://github.com/apache/mahout/pull/230 There is an example script in examples/bin/spark-shell-plot.mscala (commited to master) :https://github.com/apache/mahout/blob/master/examples/bin/spark-shell-plot.mscala Thanks! ________________________________ From: Pat Ferrel <p...@occamsmachete.com<mailto:p...@occamsmachete.com>> Sent: Monday, May 16, 2016 4:54:15 PM To: Trevor Grant Cc: Andrew Palumbo; Suneel Marthi; Dmitriy Lyubimov Subject: Re: Intro - Future Mahout - Zeppelin work This is only the beginning. Andy has been using Smile as a visualization lib since it is pretty rich in ML support. We are looking at integrating some of that with Zeppelin then adding code to feed the new visualizations in Mahout. I’m here because I’m fairly familiar with AngularJS if that’s the way to go. Smile is swing based but can output pngs, maybe other image formats—Andy? BTW Dmitriy is still very involved but has rouble getting permission to donate code. On May 16, 2016, at 1:45 PM, Trevor Grant <trevor.d.gr...@gmail.com<mailto:trevor.d.gr...@gmail.com>> wrote: Hey Andrew, thanks- you basically did all of the hard work for me! I've got the linear regression example working from: http://mahout.apache.org/users/sparkbindings/play-with-shell.html my java is sketchy at best, i tend to over import. I pulled in the following jars: org/apache/mahout/mahout-math/0.12.1-SNAPSHOT/mahout-math-0.12.1-SNAPSHOT.jar org/apache/mahout/mahout-math-scala_2.10/0.12.1-SNAPSHOT/mahout-math-scala_2.10-0.12.1-SNAPSHOT.jar org/apache/mahout/mahout-spark_2.10/0.12.1-SNAPSHOT/mahout-spark_2.10-0.12.1-SNAPSHOT.jar org/apache/mahout/mahout-spark-shell_2.10/0.12.1-SNAPSHOT/mahout-spark-shell_2.10-0.12.1-SNAPSHOT.jar I think those are all necessary... should I be pulling in more? I hate to say it (but will do so bc this isn't public) this integration is super easy from a user perspective, almost too easy- eg why not let the user add it themselves... Add the appropriate maven artifacts, restart the interpreter and run the following in a notebook: ``` import org.apache.mahout.math._ import org.apache.mahout.math.scalabindings._ import org.apache.mahout.math.drm._ import org.apache.mahout.math.scalabindings.RLikeOps._ import org.apache.mahout.math.drm.RLikeDrmOps._ import org.apache.mahout.sparkbindings._ implicit val sdc: org.apache.mahout.sparkbindings.SparkDistributedContext = sc2sdc(sc) ``` Then whatever code you want and you're off to the races... that said, adding a build profile like -PsparkMahout and creating an interpretter like %spark.mahout should be fairly straight forward. Second question, do you have an example that would be more 'visualization friendly'? I could pass the results to Angular or R just to show off how to do it. Which leads back to the question, is this even worth building a full interpreter for or just make a really nice blog post with examples on how to integrate with R...? Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org<http://trevorgrant.org/> "Fortunate is he, who is able to know the causes of things." -Virgil On Mon, May 16, 2016 at 2:09 PM, Andrew Palumbo <ap....@outlook.com<mailto:ap....@outlook.com>> wrote: Hi Trevor, welcome! It's great to have you helping out, thanks very much. I've done a good amount of work on our mahout spark shell .. so let me know if you have any questions there about what we did there.. Thanks alot! Andy -------- Original message -------- From: Suneel Marthi <smar...@apache.org<mailto:smar...@apache.org>> Date: 05/16/2016 2:44 PM (GMT-05:00) To: Trevor Grant <trevor.d.gr...@gmail.com<mailto:trevor.d.gr...@gmail.com>> Cc: Suneel Marthi <smar...@apache.org<mailto:smar...@apache.org>>, Pat Ferrel <p...@occamsmachete.com<mailto:p...@occamsmachete.com>>, Andrew Palumbo <ap....@outlook.com<mailto:ap....@outlook.com>> Subject: Re: Intro - Future Mahout - Zeppelin work Oh yes, he's around. I see him online. On Mon, May 16, 2016 at 2:42 PM, Trevor Grant <trevor.d.gr...@gmail.com<mailto:trevor.d.gr...@gmail.com>> wrote: Is Dmitriy Lyubimov still around? Looks like he created this issue for Zeppelin a while ago. (The old lost code to which you were referring?) https://issues.apache.org/jira/browse/ZEPPELIN-116 tg Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org<http://trevorgrant.org/> "Fortunate is he, who is able to know the causes of things." -Virgil On Mon, May 16, 2016 at 1:37 PM, Suneel Marthi <smar...@apache.org<mailto:smar...@apache.org>> wrote: Welcome to the party TG !! On Mon, May 16, 2016 at 2:28 PM, Trevor Grant <trevor.d.gr...@gmail.com<mailto:trevor.d.gr...@gmail.com>> wrote: Hey all, I'm excited for a chance to help out. I'm actually getting ready to download now and start playing around. I had talked about this briefly but it given a properly functioning Zeppelin interpreter for Apache Mahout, one could leverage all of the Zeppelin visualizations, anything in AngularJS, or anything in R (through clever use of Zeppelin's Resource Pools). I'll work on getting logged in to the slack channel as well. Nice to meet you all, looking forward to helping out! tg Trevor Grant Data Scientist https://github.com/rawkintrevo http://stackexchange.com/users/3002022/rawkintrevo http://trevorgrant.org<http://trevorgrant.org/> "Fortunate is he, who is able to know the causes of things." -Virgil On Sun, May 15, 2016 at 12:56 PM, Suneel Marthi <smar...@apache.org<mailto:smar...@apache.org>> wrote: FYi... Trevor was there for my talk, so he has some idea of Mahout Samsara. On Sun, May 15, 2016 at 1:51 PM, Pat Ferrel <p...@occamsmachete.com<mailto:p...@occamsmachete.com>> wrote: Hey Trevor, Good to meet you. As you probably know Mahout-Samsara is a reincarnation of the project in a new body, which is less a collection of algorithms than a roll-your-own math/algorithm tool. The major benefit is that during experimentation and later in production the code is by nature scalable on Spark and Flink. Most of the Mahout DSL is R-like and supports tensor math but we are now looking at streaming online algo support too. In any case you probably know we have a Mahout version of the Spark Shell, which has been integrated with an old version of Zeppelin (code is lost). Recently Andy has experimented with some very nice visualizations of ML data (not just analytics data). We as a project are interested in Zeppelin integration of our shell and graphics. From what I understand the graphics extension mechanism of Zeppelin is based on AngularJS, which I have some experience with. So, we’d like to start the conversation about how to proceed. We would love some help but will move ahead in any case. Pat On May 15, 2016, at 9:52 AM, Suneel Marthi <smar...@apache.org<mailto:smar...@apache.org>> wrote: Hi Trevor, Nice meeting u last week in Vancouver. Per our conversation, I wanted to introduce u to Andrew Palumbo (Mahout Chair) and Pat Ferrel (Mahout PMC). As I mentioned in my talk, we are actively looking at Zeppelin integration with Mahout (primarily for spark) and would appreciate your help (as also all things DL and ML). We definitely can use all your help as we r revamping the Mahout project and shedding its legacy MapReduce image. I sent u an invite to the Mahout slack channel, mahout.apache.org<http://mahout.apache.org/> - that's where we all hangout and not having to worry about avoiding naughty words. Looking forward to working with you Suneel
