+1 on the maintenance idea Popularity is too volatile. Nowadays technology xxx may be popular but may become obsolete within a few years
On Wed, Aug 5, 2015 at 10:54 PM, moon soo Lee <m...@apache.org> wrote: > Apologies for silence. > > Thanks for the idea about popularity. > Do you have any good way to track # of downloads per each interpreter? > Because of they're distributed all together it's not going to be easy. > > I have little bit different opinion. I believe core interpreters are the > interpreters that zeppelin community can maintain, regardless of > popularity. > For example, let's say one day someone contributed awesome interpreter, but > no one in the community does improve, fix bugs, answer the question, then i > think it is not maintainable and better not belongs to core. > > Best, > moon > > On 2015년 7월 26일 (일) at 오후 3:50 tog <guillaume.all...@gmail.com> wrote: > > > Hi moon > > > > I believe popularity can be measured by (unique) downloads and/or > starring > > by users. Probably tracking the # of downloads is easier. > > > > Cheers > > Guillaume > > > > On Sunday, July 26, 2015, moon soo Lee <m...@apache.org> wrote: > > > > > Thanks for the opinion. How do you define popularity? > > > > > > Best, > > > moon > > > On 2015년 7월 24일 (금) at 오후 2:52 tog <guillaume.all...@gmail.com > > > <javascript:;>> wrote: > > > > > > > >About the maven profile DuyHai suggested, how do we decide which > > > > >interpreter should be in core and which interpreter should not be > in? > > > > > > > > Several strategies here ;-) > > > > > > > > My initial take would be: > > > > - > > > > (1) Core: > > > > 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web' > > > > - (2) Externals but supported by Zeppelin project: spark, PySpark. > > > > - (3) All others externals at the beginning - then you could move > > > things > > > > between (2) and (3) ie deprecate plugins that are getting less > popular > > > and > > > > promote plugins that are getting more tractions > > > > > > > > On Friday, July 24, 2015, moon soo Lee <m...@apache.org > > <javascript:;>> > > > wrote: > > > > > > > > > Documenting adding external interpreter and making an sample > minimal > > > > > project sounds great!! That would help both users and developers. > > > > > > > > > > About the maven profile DuyHai suggested, how do we decide which > > > > > interpreter should be in core and which interpreter should not be > in? > > > > > > > > > > Another way of doing similar action is using -pl flag. > > > > > > > > > > # build Zeppelin with spark and flink interpreter > > > > > mvn package -pl > > > > > > > > > > > > > > > > > > > > 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web,spark,fllnk' > > > > > > > > > > # build Zeppelin except for spark and flink interpreter > > > > > mvn package -pl '!spark,!flink' > > > > > > > > > > > > > > > Thanks, > > > > > moon > > > > > > > > > > On 2015년 7월 23일 (목) at 오후 6:43 tog <guillaume.all...@gmail.com > > > <javascript:;> > > > > > <javascript:;>> wrote: > > > > > > > > > > > That would indeed be a nice addition. > > > > > > Apparently moon said it should be possible to drop a jar under > > > > > > interpreter/[name] directory ... that jar would nevertheless need > > to > > > > > extend > > > > > > interpreter class so we may want to provide a sample minimal > > project > > > > > > showing that or a tutorial (the shell interpreter is good for > that > > > > > because > > > > > > simple) > > > > > > Can work with Eran on that. > > > > > > > > > > > > On 23 July 2015 at 10:31, Corneau Damien <cornead...@gmail.com > > > <javascript:;> > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > Any way to add an interpreter without rebuilding zeppelin? > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 5:12 PM, DuyHai Doan < > > doanduy...@gmail.com > > > <javascript:;> > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > > > For decoupling the build of core Zeppelin vs third-party > > > > > interpreters, > > > > > > we > > > > > > > > can use Maven profiles: > > > > > > > > > > > > > > > > in the root pom.xml, you can define in the default profile > all > > > the > > > > > > Maven > > > > > > > > modules that should be part of the core Zeppelin build. > > > Additional > > > > > > > > interpreters can be defined in separated profiles so that > > people > > > > keen > > > > > > on > > > > > > > > using them can build them easily. Ex: > > > > > > > > > > > > > > > > <profiles> > > > > > > > > > > > > > > > > <!-- Core profile --> > > > > > > > > <profile> > > > > > > > > <id>core</id> > > > > > > > > <activation> > > > > > > > > <activeByDefault>true</activeByDefault> > > > > > > > > </activation> > > > > > > > > <modules> > > > > > > > > <module>zeppelin-interpreter</module> > > > > > > > > <module>zeppelin-zengine</module> > > > > > > > > <module>spark</module> > > > > > > > > <module>markdown</module> > > > > > > > > <module>angular</module> > > > > > > > > <module>shell</module> > > > > > > > > <module>zeppelin-web</module> > > > > > > > > <module>zeppelin-server</module> > > > > > > > > <module>zeppelin-distribution</module> > > > > > > > > </modules> > > > > > > > > </profile> > > > > > > > > > > > > > > > > <!-- Cassandra interpreter --> > > > > > > > > <profile> > > > > > > > > <id>cassandra</id> > > > > > > > > <modules> > > > > > > > > <module>cassandra</module> > > > > > > > > </modules> > > > > > > > > </profile> > > > > > > > > > > > > > > > > <!-- Lens interpreter --> > > > > > > > > <profile> > > > > > > > > <id>lens</id> > > > > > > > > <modules> > > > > > > > > <module>lens</module> > > > > > > > > </modules> > > > > > > > > </profile> > > > > > > > > </profiles> > > > > > > > > > > > > > > > > To build Zeppelin with support for Cassandra & Lens: > > > > > > > > > > > > > > > > mvn clean package -Pcore,cassandra,lens -D.... > > > > > > > > > > > > > > > > > > > > > > > > What do you think ? > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 7:38 AM, IT CTO <goi....@gmail.com > > > <javascript:;> > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > > > I volunteer to write a doc for the zeppelin documentation > > site > > > on > > > > > > step > > > > > > > by > > > > > > > > > step for adding an "external interpreter" targeted for non > > > > > developers > > > > > > > who > > > > > > > > > wants to add someone's else external interpreter. > > > > > > > > > The challenge I see is from where to get the compiled > > > interpreter > > > > > > from? > > > > > > > > > - my answer is to have a list of external interpreter on > our > > > site > > > > > > which > > > > > > > > > points to the github of the external interpreter and the > user > > > > will > > > > > > > > download > > > > > > > > > himself. - I think this should help with licensing issues > as > > > well > > > > > as > > > > > > an > > > > > > > > > external site can hold a non apache license interpreter > > (MySQL, > > > > > > Oracle) > > > > > > > > > > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 2:38 AM moon soo Lee < > > m...@apache.org > > > <javascript:;> > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Thanks for the opinions and feedback. > > > > > > > > > > > > > > > > > > > > My question was more like simply asking about merging > code > > > that > > > > > > does > > > > > > > > not > > > > > > > > > > have test, but apparently went back to recurrent subject. > > :-) > > > > > > > > > > > > > > > > > > > > Actually, plug'n'play interpreter is already possible by > > > > dropping > > > > > > all > > > > > > > > the > > > > > > > > > > necessary jars under interpreter/[name] directory and add > > > > config > > > > > to > > > > > > > > > > zeppelin-site.xml. I think nothing stops making > externally > > > > > managed > > > > > > > > > > interpreter. > > > > > > > > > > > > > > > > > > > > I think having list of all internal + external > interpreters > > > in > > > > > > > homepage > > > > > > > > > or > > > > > > > > > > wiki and let user update it would help, as a first step. > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > moon > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 9:42 PM Paul Curtis < > > > > > pcur...@maprtech.com <javascript:;> <javascript:;>> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > I agree with tog ... the core interpreters should be > > those > > > > that > > > > > > can > > > > > > > > > > > complete all the tests and provide the functionality as > > > well. > > > > > > Spark > > > > > > > > > local > > > > > > > > > > > being the best example. The Apache Drill interpreter I > > > wrote > > > > > > would > > > > > > > > > need a > > > > > > > > > > > single node Drill installation in order to provide the > > same > > > > > > testing > > > > > > > > > > > capability. I am working to provide this. > > > > > > > > > > > > > > > > > > > > > > However, I would suggest that review may be needed. As > > each > > > > > test > > > > > > > and > > > > > > > > > > > interpreter adds Zeppelin code, it also adds to the > > > > > distributable > > > > > > > as > > > > > > > > > > well. > > > > > > > > > > > Currently, the distributable is around ~500MB in size. > Is > > > the > > > > > > goal > > > > > > > to > > > > > > > > > > > provide a completely standalone Zeppelin with all the > > > > > > interpreters > > > > > > > > and > > > > > > > > > > > environments included? Or is the goal to provide a > front > > > end > > > > to > > > > > > > those > > > > > > > > > > > environments? > > > > > > > > > > > > > > > > > > > > > > Maybe that's the decision: which environments and > > > > interpreters > > > > > > are > > > > > > > > > > included > > > > > > > > > > > in a standalone distribution. > > > > > > > > > > > > > > > > > > > > > > paul > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 7:52 AM, IT CTO < > > goi....@gmail.com > > > <javascript:;> > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > I think that we can have the interpreters add in the > > > build > > > > > > system > > > > > > > > > using > > > > > > > > > > > > compilation parameters (e.g. same as the list in the > > > > > > > > zeppelin-env.xml > > > > > > > > > > > that > > > > > > > > > > > > way the basic build builds only the core interpreters > > and > > > > the > > > > > > > user > > > > > > > > > can > > > > > > > > > > > > easily interperters to the build process. > > > > > > > > > > > > With regard to a release, this can be just as adding > > the > > > > jar > > > > > > and > > > > > > > > the > > > > > > > > > > > config > > > > > > > > > > > > or just the jar with auto-discovery > > > > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 1:40 PM DuyHai Doan < > > > > > > > doanduy...@gmail.com <javascript:;> <javascript:;>> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > If we want to make the intepreters system modular > to > > > > > decouple > > > > > > > > souce > > > > > > > > > > > code, > > > > > > > > > > > > > the process of activating an interpreter should be > > > > flexible > > > > > > and > > > > > > > > > easy > > > > > > > > > > to > > > > > > > > > > > > > use. > > > > > > > > > > > > > > > > > > > > > > > > > > Asking end-users to make a custom build is not a > > viable > > > > > > > strategy. > > > > > > > > > > > > Recently > > > > > > > > > > > > > I've seen some people trying to make a custom build > > of > > > > > > Zeppelin > > > > > > > > and > > > > > > > > > > > > facing > > > > > > > > > > > > > a lot of issues (incorrect Maven version, no Bower > > > > > installed, > > > > > > > > > > incorrect > > > > > > > > > > > > > repository policies settings for Maven etc ...) > > > > > > > > > > > > > > > > > > > > > > > > > > Ideally, activating an interpreter should be as > > simple > > > as > > > > > > > > dropping > > > > > > > > > a > > > > > > > > > > > jar > > > > > > > > > > > > > into the lib directory > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 12:28 PM, < > > > > > felixcheun...@hotmail.com <javascript:;> <javascript:;> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 what tog says > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: tog > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent: Wednesday, July 22, 12:22 AM > > > > > > > > > > > > > > > > > > > > > > > > > > > > Subject: Interpreter with no test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > To: dev@zeppelin.incubator.apache.org > > <javascript:;> > > > <javascript:;> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > This seems to be a recurrent topic ;-) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It has to be decided what are the core > interpreters > > > the > > > > > > > > Zeppelin > > > > > > > > > > want > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > support - then I would believe you can have a > > > > > wiki/webpage > > > > > > > > > > dedicated > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > > > listing all interpreters that are known to be > > working > > > > > with > > > > > > > > > > Zeppelin. > > > > > > > > > > > > You > > > > > > > > > > > > > > > > > > > > > > > > > > > > may even consider having recommended plugins > and/or > > > > > ranking > > > > > > > in > > > > > > > > > > case 2 > > > > > > > > > > > > > > > > > > > > > > > > > > > > plugins are doing the same job. Grails (but it is > > > > > probably > > > > > > > not > > > > > > > > > the > > > > > > > > > > > only > > > > > > > > > > > > > > > > > > > > > > > > > > > > one) is implementing this (see > > > > > https://grails.org/plugins/ > > > > > > ) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In order to keep Zeppelin low in term of > footprint > > - > > > > the > > > > > > core > > > > > > > > > > > commiters > > > > > > > > > > > > > can > > > > > > > > > > > > > > > > > > > > > > > > > > > > even decide to support some of those plugins > (shell > > > for > > > > > > > > example) > > > > > > > > > > > > > > > > > > > > > > > > > > > > At one end of the spectrum you could imagine > > Zeppelin > > > > > > without > > > > > > > > any > > > > > > > > > > > > > > > > > > > > > > > > > > > > interpreters and at the other end Zeppelin with > all > > > > known > > > > > > > > > plugins - > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > truth is probably in the middle. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wednesday, July 22, 2015, Anthony Corbacho < > > > > > > > > > > > > > anthonycorba...@apache.org <javascript:;> > > > <javascript:;> > > > > > > > > > > > > > > > > > > > > > > > > > > > > <javascript:_e(%7B%7D,'cvml',' > > > > anthonycorba...@apache.org <javascript:;> > > > > > <javascript:;> > > > > > > > ');>> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IT CTO is right, right now, zeppelin merge > every > > > > > > > interpreter > > > > > > > > in > > > > > > > > > > the > > > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > > > > > > > > > > > > base and its mean that we (committer) have to > > take > > > > care > > > > > > of > > > > > > > > > build > > > > > > > > > > > > > falling > > > > > > > > > > > > > > or > > > > > > > > > > > > > > > > > > > > > > > > > > > > > questions for those interpreters. and > personally > > i > > > > have > > > > > > no > > > > > > > > > > > experience > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > presto, cassandra etcetc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think, It can be interesting to detach > > > interpreters > > > > > > from > > > > > > > > > > zeppelin > > > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > > > > > > > > > > > > base and create a sortof plug'n'play module > where > > > the > > > > > > users > > > > > > > > can > > > > > > > > > > > plug > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > > interpreter he wants to use. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With this approach, we can keep zeppelin code > > base > > > > > > smaller > > > > > > > > (we > > > > > > > > > > > > provide > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > > core + maybe some interpreters (spark, md, > shell) > > > as > > > > > > > > default). > > > > > > > > > > and > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > > community can build and manage other > interpreters > > > (i > > > > > > assume > > > > > > > > if > > > > > > > > > > they > > > > > > > > > > > > > build > > > > > > > > > > > > > > > > > > > > > > > > > > > > > it they have experiences and probably can > answer > > > > > > > questions). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 2:34 PM, IT CTO < > > > > > > goi....@gmail.com <javascript:;> <javascript:;> > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the question is what\who is going to > > fix > > > > > issues > > > > > > > in > > > > > > > > > > these > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Interpreters if something fails? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am guessing that if one uses these > > interpreters > > > > and > > > > > > > > > approach > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > community with questions then we might not > have > > > the > > > > > > right > > > > > > > > > > support > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > > > > > > > > > > > > > him. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 8:04 AM moon soo Lee > < > > > > > > > > > m...@apache.org <javascript:;> <javascript:;>> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There're some open pullrequests with > complete > > > > > > > interpreter > > > > > > > > > > > > > > > > > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > but no test (eg. > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/110 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > , > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/68 > > > > > > ). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Personally, I'm feeling not safe having > code > > > > > without > > > > > > > > test, > > > > > > > > > > but > > > > > > > > > > > at > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > same > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > time, keeping these great contributions > > > unmerged > > > > > > sounds > > > > > > > > not > > > > > > > > > > > cool. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to hear opinions about merging > them > > > with > > > > > > some > > > > > > > > mark > > > > > > > > > > > such > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 'beta' > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > or 'untested', and create issue for adding > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > moon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > *Paul Curtis - *Senior Product Specialist - Field > > > Enablement > > > > > Team > > > > > > > > > > > *O: *+1 203-660-0015 - *M:* +1 203-539-9705 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > > > > > > -- > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > -- > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > >
