Re: Small tips when running Zeppelin on EMR

[Alexander Bezzubov]

Thank you for sharing, Kevin!

Great tips, especially how to setup S3 storage on EMR.

--
Alex

On Wed, May 18, 2016 at 6:04 PM, Kevin (Sangwoo) Kim 
wrote:

> Hi Ahyoung,
>
> I just added #6 while writing this mail, after realized I kept the cluster
> turn on after the presentation.. (Haha)
>
> I'm attaching the slide.
> (Sorry for non-Korean readers, but most of the slide is screen-shots, I
> hope it helps!)
>
> - Kevin
>
>
>
> 2016년 5월 18일 (수) 오후 4:55, Hyung Sung Shim 님이 작성:
>
>> Thank you for sharing great information!
>>
>>
>> 2016-05-18 16:49 GMT+09:00 Ahyoung Ryu :
>>
>>> Hi Kevin,
>>>
>>> Thanks for the sharing. It's really helpful indeed not only me but also
>>> to many others.
>>> I think *6.**Don't forget to terminate cluster when you're done your
>>> job* is the most important thing :)
>>> Is there any way I can see your slide? If so, it will be really
>>> appreciate.
>>>
>>> Best regards,
>>> Ahyoung
>>>
>>> 2016년 5월 18일 (수) 오후 3:19, Kevin (Sangwoo) Kim 님이
>>> 작성:
>>>
 Hi Zeppelin users,

 I'v been presenting some demo on "Spark+Zeppelin on AWS EMR" at AWS
 Summit Seoul yesterday. I'm so sad that the slides are written in Korean so
 it's hard to share, but I'd like to share some essentials.

 1. Running Z on EMR is super easy. (EMR team did really good job. You
 can do that with only few clicks, took 8min to launch)

 2. You can launch EMR with spot instances, it will save your money.

 3. You can provide some configs when you launch EMR cluster, so you may
 want to save your notebook on S3, proper config is as follow.

 [
   {
 "Classification": "zeppelin-env",
 "Properties": {},
 "Configurations": [
   {
 "Classification": "export",
 "Properties": {
   “ZEPPELIN_NOTEBOOK_STORAGE"
  :"org.apache.zeppelin.notebook.repo.S3NotebookRepo",
   "ZEPPELIN_NOTEBOOK_S3_BUCKET": "BUCKET_NAME",
   "ZEPPELIN_NOTEBOOK_S3_USER": "SOME_USER_NAME"
 },
 "Configurations": []
   }
 ]
   }
 ]

 4. You need to set proper spark.executor.memory in Zeppelin interpreter
 setting.

 5. You can increase or decrease cluster size in cluster detail page.

 6. Don't forget to terminate cluster when you're done your job :)

 That's all!


 If you have more tips, plz add it on this mail thread. Thanks!

 - Kevin




>>

Re: How to disable google analytics

[Alexander Bezzubov]

Hi,


first, plese let me the confusion.

As PMC member of the project, I can assure you that there is NO any
tracking facilities build into Apache Zeppelin source or binary
distribution that I'm aware of. Nither enabled nor disabled.

The parameter that you are referring, as everything under the /doc/
directory, is part of the documentation website and it does have project's
Google Analytics enabled, but it's the staic website that is generated
using Jekyll.
You can lear more about how Jekyll is used in [1].

Hope it helps to clarifie the concerns.

 1. https://github.com/apache/zeppelin/blob/master/docs/README.md

--
Alex

On Thu, Jun 9, 2016, 19:26 Charles Givre  wrote:

> All,
> Just my .02 but this REALLY should be disabled by default.
> —C
>
> > On Jun 9, 2016, at 12:27, George Webster  wrote:
> >
> > Hey guys and gals,
> >
> > Google Analytics appears to be built into Zeppelin and is active. Is
> > their a way to disable this without rebuilding?
> >
> >
> https://github.com/apache/incubator-zeppelin/blob/84d25a8eabfe8efce0170e43a7b3c5a715e5104d/docs/_config.yml
> >
> > # Settings for analytics helper
> > # Set 'provider' to the analytics provider you want to use.
> > # Set 'provider' to false to turn analytics off globally.
> > #
> > analytics :
> > provider : google_universal
> > google_classic :
> > tracking_id : 'UA-45176241-2'
> > google_universal :
> > tracking_id : 'UA-45176241-5'
> > domain : 'zeppelin.apache.org'
> > getclicky :
> > site_id :
> > mixpanel :
> > token : '_MIXPANEL_TOKEN_'
> > piwik :
> > baseURL : 'myserver.tld/piwik' # Piwik installation address (without
> protocol)
> > idsite : '1' # the id of the site on Piwik
>
>

Re: Ask opinion regarding 0.6.0 release package

[Alexander Bezzubov]

It sounds as a most flexible way, let's try it for 0.6 release and see if
it addresses all user needs well.

Untill we have GUI for interpreter loading, I feel we also should try our
best to make sure netinst usage is documented well, so users on different
environments (no internet, corporate proxy, etc) all know how to use it.

On Tue, Jun 21, 2016, 15:25 mina lee  wrote:

> Moon, having netinst package for the sake of simplicity and flexibility
> totally makes sense to me.
> If there is no strong objection, I would like to follow your approach for
> 0.6.0 release.
>
> On Mon, Jun 20, 2016 at 6:23 PM, moon soo Lee  wrote:
>
>> "zeppelin-bin-min" package, I worried about lack of written policy which
>> goes in which does not.
>> Without policy, yes, we can always vote for list of interpreters. But
>> then, we'll need vote everytime we add/remove interpreter, and this
>> sounds not good.
>>
>> Even if it is true that majority of user uses 'spark',
>> other users may ask "zeppelin-bin-cassandra-min",
>> "zeppelin-bin-flink-min" and so on.
>> Once we have 'zeppelin-bin-min' package with 'spark', then there will be
>> no good excuse of not having other *min packages.
>> And we can end up with a lot of binary packages in each release. which is
>> not really optimal.
>>
>> For this reasons, I believe we'll need a written policy based on
>> community consensus to make 'zeppelin-bin-min'.
>> But I think making netinst script will be a lot easier and give more
>> flexibility to users than making written policy for 'zeppelin-bin-min'.
>>
>> Anyway, it's really great to hear volunteer some time to help.
>> Thanks  Mohit Jaggi.
>> Whether we have multiple packages or not, we'll need a lot of help on
>> improving release script [1] and verification of release candidates.
>>
>> Regarding maintainers/contributors of each interpreter(s),
>> Spark community recently removed 'maintainer' role from review process
>> [2] for some reasons.
>> DuyHai Doan, could you give little more details about how your idea
>> different from maintainers of Spark?
>>
>> Thanks,
>> moon
>>
>> [1] https://github.com/apache/zeppelin/blob/master/dev/create_release.sh
>> [2]
>> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=34835307&originalVersion=61&revisedVersion=66
>>
>>
>> On Mon, Jun 20, 2016 at 3:10 AM DuyHai Doan  wrote:
>>
>>> +1 for zeppelin-bin-min release package
>>>
>>> What I would suggest is that for a specific package of Zeppelin with XXX
>>> interpreter(s) built-in is that the maintainers/contributors of each
>>> interpreter(s) can help releasing those "custom" builds for the community.
>>> Any thought on this idea ?
>>>
>>> On Mon, Jun 20, 2016 at 10:30 AM, Partridge, Lucas (GE Aviation) <
>>> lucas.partri...@ge.com> wrote:
>>>
 I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy
 to configure it to work with a proxy for users behind a corporate firewall.

 Thanks, Lucas.



 *From:* Mohit Jaggi [mailto:mohitja...@gmail.com]
 *Sent:* 17 June 2016 18:06
 *To:* users@zeppelin.apache.org
 *Subject:* EXT: Re: Ask opinion regarding 0.6.0 release package



 sure…that is possible. one can also make a build from source and
 customize as needed. but not having to do that makes things easier. i do
 believe that for the vast majority of cases a minimal build with spark (and
 possibly other small items like shell, jdbc, python) will be quite
 valuable, imho.

 is there a lot of overhead involved in having multiple binaries
 available? i am happy to volunteer some time to help with this if needed.



 On Jun 17, 2016, at 9:45 PM, moon soo Lee  wrote:



 In case of no internet access, how about



 a. download 'zeppelin-bin-netinst' and run
 'bin/install-interpreter.sh', and then copy the package to production env.

 b. download 'zeppelin-bin-all' and copy the package to production env.



 ?



 Thanks,

 moon





 On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi 
 wrote:

 Many production environments have no internet access. A script like
  this can be useful to some but it should not replace the proposed min
 binary.

 Sent from my iPhone


 On Jun 17, 2016, at 9:20 PM, moon soo Lee  wrote:

 Hi,



 Thanks for bringing this discussion.

 it's great idea minimize binary package size.



 Can we set a policy to decide which interpreter goes to
 'zeppelin-bin-min', which is not?



 One alternative is, instead of making 'zeppelin-bin-min', we can make
 'zeppelin-bin-netinst'.

 We can provide a shell script such as, 'bin/install-interpreter.sh' and
 the script will download interpreters and their dependencies from maven
 repository and store under /interpreter dir

BigQuery Interpreter for Apache Zeppelin

[Alexander Bezzubov]

Dear All,

is anybody from the community interested\looking forward a BigQuery [1]
interpreter support in Apache Zeppelin?


 1. https://cloud.google.com/bigquery

--
Kind regards,
Alexander.

Re: Elastic Search Storage for Zeppelin Notebooks

[Alexander Bezzubov]

Hi Ashish,

let's move this discussion to dev@ mailing list (by removing users@ from CC)

As far as I understood, you are working on multiple things one being
NotebookRepo implementation backed by Elasticsearch.
That sounds great, and I would be happy to assist you with it and help by
reviewing ideas\code as soon as you are ready to share those with the
community here!

Good place to start is - to create a separate JIRA issues for each of the
changes you want to submit, attaching mocks\design idea before actual
implementation. It's a good place to have technical discussion as well.

>From there please just submit at PR from you repo\branch, following
project's contribution instructions [1].
Feel free to \cc people in PR comments, so they get notified as soon as you
think the code is ready for review (i.e my GH id is @bzz)

We are always happy to bring new contributions to the project!

 1. https://github.com/apache/zeppelin/blob/master/CONTRIBUTING.md

--
Alex

On Thu, Jun 30, 2016 at 5:18 PM, Andrea Santurbano 
wrote:

> Hi Ashish,
> if you share the github repo i would like to contribute!
> Bests
> Andrea
>
> Il giorno gio 30 giu 2016 alle ore 08:12 ashish rawat 
> ha scritto:
>
>> Hi,
>>
>> We have implemented a basic version of Elastic Search Storage for
>> Zeppelin and would like to contribute it back to the main repo. But before
>> we spend time in preparing a Pull Request, I just wanted to check
>>
>>1. Does this feature make sense for the Zeppelin project and would be
>>accepted in main repo,
>>2. Would anyone be willing to review it, so that we can have an
>>initial discussion around how ready is our code for raising a Pull 
>> Request.
>>
>> We are also working on a couple of other features, including a read-only
>> (Dashboard) mode, which would restrict updating params and output to
>> notebook. It will be very helpful if someone can help us initially in
>> making contributions to the project.
>>
>> Regards,
>> Ashish
>>
>

Re: Review on notebook

[Alexander Bezzubov]

Thank you for sharing nice example of Machine Learning and Visualization
notebook using Spark!

--
Alex

On Thu, Jun 30, 2016 at 6:12 AM, tog  wrote:

> Hi
>
> Tomorrow with a colleague, I will be presenting Apache Zeppelin to the
> Toulouse Data Science.
> The demo will be based on the following notebook:
> https://www.zeppelinhub.com/viewer/notebooks/bm90ZTovL3RvZy9Mb2NhbFplcHBlbGluLzUxODMvbm90ZS5qc29u
>
> It is inspired by some work from Carol Mc Donald (mapR) - we added some
> stuff & a couple of visualization. Feel free to have a look
>
> All comments and improvements are welcome :-)
>
> Thanks to Bala & Sunil for reviewing the code and finding some issues!
>
> Cheers
> Guillaume
>

Re: Adjusting Font / Size

[Alexander Bezzubov]

That's a good question!

So far you have an option to make browser font size bigger manually, or to
customize .css or .js to setup Ace editor by either '#editor' id or

editor.setOptions({
  fontFamily: "tahoma",
  fontSize: "10pt"
});

Right now you can not change it through the web app UI though.

--
Alex

On Wed, Jul 20, 2016, 08:45 Shayne Hodge  wrote:

> I've looked through the docs, but can't find any option to change the font
> / size being used for coding. Is this possible at this point in time?
>
> Thanks,
>
> Shayne Hodge
> sho...@snaplogic.com
>
>

Re: Spark/Zeppelin/Cassandra talk

[Alexander Bezzubov]

Thanks for sharing the feedback, Duy Hai Doan!

It could be great to have ZeppelinServer as well as all interpreters
exposing their metrics with something like http://metrics.dropwizard.io and
we could make default notebook after them periodically and visualize the
results :)

--
Alex


On Fri, Jul 22, 2016, 11:50 moon soo Lee  wrote:

> Cool.
>
> Exposing metrics through REST api and/or JMX sounds interesting!
> And thanks for sharing slide and how the event was as well as feedbacks.
>
> moon
>
> On Fri, Jul 22, 2016 at 1:10 AM DuyHai Doan  wrote:
>
>> Here are the slides for the talk last night at Spark London meetup :
>> http://www.slideshare.net/doanduyhai/spark-zeppelincassandra-at-synchrotron-64253321
>>
>> There were > 80 people last night and a lot of question about Zeppelin.
>>
>> One recurrent remark about Zeppelin is that when running the server on a
>> long run (several weeks) users are facing instability e.g. sometimes the
>> server just hangs so they need to restart it.
>>
>> Possible improvements could be:
>>
>> 1) Expose REST API to return metrics about the state of ZeppelinServer
>> and of each Interpreter
>> 2) Expose JMX metrics about the state of ZeppelinServer and of each
>> Interpreter. Although I prefer REST API, JMX is the de-facto standard for
>> monitoring and there are a lot of existing monitoring tool that are
>> designed for JMX
>>
>> When I have some bandwidth, I can push some PRs for those monitoring stuff
>>
>>
>>
>>
>>
>> On Wed, Jul 6, 2016 at 4:49 PM, moon soo Lee  wrote:
>>
>>> Thanks for sharing great event.
>>>
>>>
>>> On Wed, Jul 6, 2016 at 7:20 AM DuyHai Doan  wrote:
>>>
 Hello Zeppelin fans

  If you're living in the London area, I'll be giving a talk about using
 Spark/Cassandra and Zeppelin to store, aggregate and visualize particle
 accelerator metrics:

 http://www.meetup.com/fr-FR/Spark-London/events/232355493/

 Also if you know any event related to Zeppelin, please ping me so I can
 add it to the community page here:
 https://cwiki.apache.org/confluence/display/ZEPPELIN/Community

 Regards

 Duy Hai DOAN

>>>
>>

Re: Release time frame of 0.6.1?

[Alexander Bezzubov]

Hi,

thanks for asking! We all looking forward and working hard on the next
release of Apache Zeppelin.

You can track the ongoing progress on 0.6.1 release under
https://issues.apache.org/jira/browse/ZEPPELIN-1177

--
Alex

On Thu, Jul 28, 2016, 12:19 Pedro Rodriguez  wrote:

> On similar regards, when is spark 2.0.0 expected to be supported? I know
> it was just released, so this is more to get an idea of when to expect it.
>
> On Wed, Jul 27, 2016 at 8:34 PM, Haopu Wang  wrote:
>
>> Hi, do you know when will Zeppelin 0.6.1 be released? Thanks!
>>
>>
>>
>> Regards,
>>
>> Haopu
>>
>>
>>
>
>
>
> --
> Pedro Rodriguez
> PhD Student in Distributed Machine Learning | CU Boulder
> UC Berkeley AMPLab Alumni
>
> ski.rodrig...@gmail.com | pedrorodriguez.io | 909-353-4423
> Github: github.com/EntilZha | LinkedIn:
> https://www.linkedin.com/in/pedrorodriguezscience
>
>

Re: Drill JDBC Interpreter: Connection Refused

[Alexander Bezzubov]

Hi Kevin,

this is very strange indeed.

Could you please try to delete ./logs/* restart Zeppelin and try again, and
then post 2 full log files, of both zeppelin server and drill interpreter
I.e on gist or pastebin and share a link here?

This should help to debug the situation.

--
Alex


On Fri, Jul 29, 2016, 01:17 Kevin Verhoeven 
wrote:

> Thanks Jongyoul,
>
>
>
> I tried your suggestion and used %drill with:
>
>
>
> drill.driver
>
> drill.url
>
> drill.user
>
> drill.password
>
>
>
> However, the query still failed with a “Connection refused” error.
>
>
>
> My drill.url is: jdbc:drill:zk=localhost:2181/drill/drillbits1
>
>
>
> ZooKeeper and Drill both work, I’ve run many queries through Drill’s UI.
> Only Zeppelin throws an error.
>
>
>
> Do you have any other suggestions?
>
>
>
> Thanks,
>
>
>
> Kevin
>
>
>
> The error I found in the log:
>
>
>
> ERROR [2016-07-28 09:13:37,639] ({Thread-97}
> RemoteScheduler.java[getStatus]:256) - Can't get status information
>
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:96)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:254)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:212)
>
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>
> ... 8 more
>
> Caused by: java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:589)
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
>
> ... 9 more
>
>
>
> *From:* Jongyoul Lee [mailto:jongy...@gmail.com]
> *Sent:* Thursday, July 28, 2016 3:02 AM
>
>
> *To:* users@zeppelin.apache.org
> *Subject:* Re: Drill JDBC Interpreter: Connection Refused
>
>
>
> Hello Kevin,
>
>
>
> I've got to know what the problem is.
>
>
>
> If you are running query with "%drill ...", you can set
>
>
>
> drill.url
>
> drill.user
>
> drill.password
>
>
>
> If you set default.*, you should use "%jdbc ..."
>
>
>
> It's a little bit confused, and it will be improved by 0.7.0.
>
>
>
> Hope this help,
>
> Jongyoul
>
>
>
> On Thu, Jul 28, 2016 at 7:56 AM, Kevin Verhoeven <
> kevin.verhoe...@ds-iq.com> wrote:
>
> I have the same problem. I am running Zeppelin version 0.6.0 and Drill
> 1.5. When I attempt a Drill query from Zeppelin I receive a "Connection
> refused" error. The query works successfully in Drill. My Interpreter
> configurations:
>
>
>
> default.url: jdbc:drill:zk=my*ServerName*:8121/drill/drillbits1
>
> default.user:
>
> default.password:
>
>
>
> Dependency: org.apache.drill.exec:drill-jdbc:1.5.0
>
>
>
> The query I run is very simple, but the Zeppelin paragraph runs for a few
> seconds and then returns the connection refused error. I pasted the full
> error below.
>
>
>
> Any recommendations?
>
>
>
> Thanks!
>
>
>
> Kevin
>
>
>
> Zeppelin Paragraph:
>
>
>
> %drill
>
> SELECT * FROM asdf LIMIT 100
>
>
>
> Zeppelin Error:
>
>
>
> java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>
> at
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.

Re: Zeppelin JSON files - Table not found

[Alexander Bezzubov]

Hi Joaquin,

Zeppelin itself does not have any restrictions on the format of the input
data, everything depends on the data processing system and interpreter you
choose to use. looking at you example - as you use Apache Spark [1] would
be the place to look for details.

Hope this helps!

1.
http://spark.apache.org/docs/1.6.1/sql-programming-guide.html#json-datasets

--
Alex

On Fri, Jul 29, 2016, 01:00 Joaquin Alzola 
wrote:

>
>
> >ERROR:
> >org.apache.spark.sql.AnalysisException: Table not found: facebook; line
> 2 pos 5
>
>
>
> Changing the zeppelin.spark.useHiveContext to false solved the issue.
>
>
>
> What format does Zeppelin need the JSON to be?
>
> Playing aroung everything on the same line only picks up the last , while
> in different line it give corrupt JSON.
> This email is confidential and may be subject to privilege. If you are not
> the intended recipient, please do not copy or disclose its content but
> contact the sender immediately upon receipt.
>

Re: Multiuser support of Zeppelin.

[Alexander Bezzubov]

Hi Egor,

let me share two approaches that we used to archive Apache Zeppelin working
in muliteanant environment with Apache Spark:

- run a separate container with Zeppelin per-user, from the small cluster
of Docker machines (so a single machine runs just 2-3 containers \w
SparkContexts)
This works well, supports Spark standalone cluster but requires central
external auth and small "resource manager" to allocate the containers to
the Docker cluster + a reverse proxy as a single point of entry for the
user.
We have implemented all this in one binary under openaource project called
Z-Manager Muliteanancy, you can get more details here [1]. It is beta and
we didnt have capacity to support it recently.

- run single Zeppelin with Auth enabled + Livy interpreter + Spark in YARN
server mode

This is more generic solution, but it requires particular cluster
configuration. Here YARN is used as a resource manager to handle multiple
Spark contexts/drivers processes on the same cluster as tasks themselves.
Afaik Mesos cluster might be used instead by I do not have first hand
experience about it. You can read more about it here [2].

Hope this helps!

--

Alex

1. https://github.com/NFLabs/z-manager/blob/master/multitenancy/README.md
2. http://zeppelin.apache.org/docs/0.6.0/interpreter/livy.html

On Sat, Aug 6, 2016, 06:12 vincent gromakowski <
vincent.gromakow...@gmail.com> wrote:

> One zeppelin per user in mesos container on datanode type server is fine
> for me. An Ansible script configure each instance with user specifities and
> launch it in Marathon. A service discovery (basic shell script) update an
> apache server with basic auth and route each user to his instance. Mesos
> also run a SMACK stack on which zeppelin rely.
>
> Le 5 août 2016 11:01 PM, "Egor Pahomov"  a écrit :
>
> I need to build a chart for 10 days for all countries(200) for several
> products by some dimensions. I would need at least 4-6 gb per zeppelin for
> it.
>
> 2016-08-05 12:31 GMT-07:00 Mohit Jaggi :
>
>> put your big results somewhere else not in Z’s memory?
>>
>> On Aug 5, 2016, at 12:26 PM, Egor Pahomov  wrote:
>>
>> - Use spark driver in “cluster mode” where driver runs on a worker
>>> instead of the node running Z
>>
>>
>> Even without driver Z is heavy process. You need a lot of RAM to keep big
>> results from job. And most of all - zeppelin 0.5.6 does not support cluster
>> mode and I'm not ready to move to 0.6.
>>
>> 2016-08-05 12:03 GMT-07:00 Mohit Jaggi :
>>
>>> Egor,
>>> Running a scale out system like Spark with multiple users is always
>>> tricky. Operating systems are designed to let multiple users share a single
>>> machine. But for “big data” a single user requires the use of several
>>> machines which is the exact opposite. Having said that I would suggest the
>>> following:
>>>
>>> - Use spark driver in “cluster mode” where driver runs on a worker
>>> instead of the node running Z
>>> - Set appropriate limits/sizes in spark master configuration
>>> - run separate instances of Z per user, but then you will have a tough
>>> time collaborating and sharing notebooks…maybe they can be stored in a
>>> shared space and all Z instances can read them but I am afraid that shared
>>> access might clobber the files. Z developers can tell us if that is true
>>>
>>> Another alternative is virtualization using containers but I think that
>>> will not be easy either.
>>>
>>> Mohit
>>> Founder,
>>> Data Orchard LLC
>>> www.dataorchardllc.com
>>>
>>>
>>> On Aug 5, 2016, at 11:45 AM, Egor Pahomov 
>>> wrote:
>>>
>>> Hi,  I'd like to discuss best practices for using zeppelin in the
>>> multi-user environment. There are several naive approaches, I've tried for
>>> at least couple month each and not a single one worked:
>>>
>>> *All users on one zeppelin.*
>>>
>>>- One spark context - people really break sc and when they are all
>>>in the same boat a single person can stop many from working.
>>>- No resource management support. One person can allocate all
>>>resources for a long time
>>>- The number of notebooks is enormous - it's hard to find anything
>>>in it.
>>>- No security separation - everyone sees everything. I do not care
>>>about security, but I care about fool prove. And people can accidently
>>>delete notebooks of each other.
>>>
>>> *Every user has his own Zeppelin on one machine*
>>>
>>>- Every zeppelin instance eats memory for zeppelin itself. It's not
>>>enough memory at some point.
>>>- Every spark driver(I use yarn client mode) eats memory. Same issue.
>>>- Single point of failure
>>>- Cores might be not enough
>>>- I can not prove it, but even if memory and cores enough, Zeppelin
>>>experience problems when it's >10 zeppelin instances on one machine. Do 
>>> not
>>>know for which reason, maybe it's spark driver issues.
>>>
>>> Our current approach:
>>> *Every department has it's own VM, it's own zeppelin in it.*
>>>
>>>- I'm not Devops I do not have 

Re: Stickers

[Alexander Bezzubov]

Looks awesome, thank you for putting it together!

--
Alex

On Sat, Aug 13, 2016, 05:19 Trevor Grant  wrote:

> That's awesome,  thanks so much!!
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>
>
> On Fri, Aug 12, 2016 at 4:16 AM, Corneau Damien 
> wrote:
>
>> Seems it takes some time to be accepted by Stickermule.
>> However I made this PR in case anybody would like to make their own:
>> https://github.com/apache/zeppelin/pull/1325
>>
>> On Thu, Aug 11, 2016 at 9:45 AM, Corneau Damien 
>> wrote:
>>
>>> Thanks Trevor,
>>> I will take care of that today then.
>>> Once it's done , I'll share the link back to the mailing list and add it
>>> also on the project website
>>>
>>> On Aug 11, 2016 2:38 AM, "Trevor Grant" 
>>> wrote:
>>>
 On Mahout we just set up a sticker mule so anyone could get one
 whenever they wanted...
 https://www.stickermule.com/marketplace/13179-apache-mahout

 I think the spirit of Apache merchandising is to prevent for-profit
 entities from using our logos with out authorization.

 I forget the name of the guy who is in charge of merch, but I'm sure I
 can find him on the community mailing list.  Does anyone have any
 interest/objections to doing this?

 https://www.stickermule.com/marketplace/sell

 Basically just upload the image and set the markup to zero.

 I can take point on this or Corneau or someone else in the community- I
 just want to rep some zeppelin ;)

 lmk

 tg



 Trevor Grant
 Data Scientist
 https://github.com/rawkintrevo
 http://stackexchange.com/users/3002022/rawkintrevo
 http://trevorgrant.org

 *"Fortunate is he, who is able to know the causes of things."  -Virgil*


 On Wed, Aug 10, 2016 at 11:01 AM, Corneau Damien 
 wrote:

> We don't have any official sticker to order from.
> And I'm not sure about the rules for printing your own stickers.
> Although I found this:
> http://www.apache.org/foundation/marks/merchandise.html
>
> If you need it, I can send you the logo .svg
> (I really need to host it on the website)
>
>
> On Thu, Aug 11, 2016 at 12:03 AM, Trevor Grant <
> trevor.d.gr...@gmail.com> wrote:
>
>> Hey,
>>
>> Does anyone have any Zeppelin stickers or is there somewhere we can
>> order them (aka sticker mule).
>>
>> Thanks
>>
>> tg
>>
>> Trevor Grant
>> Data Scientist
>> https://github.com/rawkintrevo
>> http://stackexchange.com/users/3002022/rawkintrevo
>> http://trevorgrant.org
>>
>> *"Fortunate is he, who is able to know the causes of things."
>>  -Virgil*
>>
>>
>

>>
>

Re: Access resource pool from python interpreter

[Alexander Bezzubov]

Yep, it's not supported yet so please file a JIRA issue and I will be happy
to help adding it!

--
Alex

On Sat, Aug 13, 2016, 01:16 moon soo Lee  wrote:

> %python does not expose the api to access resource pool, yet. I think
> it's great to have.
>
> PyZeppelinContext [1] can be the place where z.put(), z.get() python api
> can be exposed.
>
> The api implementation will need to call JVM PythonInterpreter instance to
> access the ResourcePool. Example of accessing resource pool can be found in
> ZeppelinContext [2].
>
> Please feel free to file a JIRA issue!
>
> Thanks,
> moon
>
> [1]
> https://github.com/apache/zeppelin/blob/master/python/src/main/resources/bootstrap.py#L110
> [2]
> https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/ZeppelinContext.java#L705
>
>
>
> On Fri, Aug 12, 2016 at 8:42 AM Trevor Grant 
> wrote:
>
>> In pyspark we can access the resource pool with something like
>> z.get("myVar")
>>
>> In %python, that doesn't seem to work.
>>
>> Does anyone know a good hack for making that happen or should I open a
>> JIRA ticket?
>>
>> tg
>>
>>
>> Trevor Grant
>> Data Scientist
>> https://github.com/rawkintrevo
>> http://stackexchange.com/users/3002022/rawkintrevo
>> http://trevorgrant.org
>>
>> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>>
>>

Re: show() function not found by the python interpreter after modifying the source code

[Alexander Bezzubov]

Hi Alexander,

thank you for contributing Matplotlib integration improvements!

You are right, it's odd and this is not your text editor's fault - our
current Python interpreter implementation is quite early stages and these
are artefacts of the way it currently works. Sometimes it does not accept
valid Python files [1].

I'm working on improving it right now [2], by changing the underneath
mechanism of communication with Python process, which should address this
issues, as well as few others [3], like windows support, better error
detection, etc.

I hope to submit a PR next week, meanwhile the best way to debug your
changes is to enable logging [4] by adding a line to log4j.properties "
log4j.logger.org.apache.zeppelin.python.PythonInterpreter=DEBUG" (i.e by
creating it in ./python/src/test/resources/ if you want it to work only in
unit-tests, or just changing code to `log.warn` to always log it) in order
to be able to see what exactly actual Python repl process is not happy
about.

Hope this helps!

 1. https://issues.apache.org/jira/browse/ZEPPELIN-1133
 2. https://issues.apache.org/jira/browse/ZEPPELIN-1325
 3. https://issues.apache.org/jira/browse/ZEPPELIN-1133?jql=
project%20%3D%20ZEPPELIN%20AND%20component%20%3D%20python-interpreter%20AND%
20resolution%20%3D%20Unresolved
 4. https://github.com/apache/zeppelin/blob/master/python/src/main/java/org/
apache/zeppelin/python/PythonInterpreter.java#L221

--
Alex

On Sat, Aug 13, 2016, 01:04 Goodman, Alexander (398K)  wrote:

> An update: It seems like the problem might lie in my editor. After a bit
> of hacking into the log fiIes, I found that there were several
> IndentationErrors being reported, starting at the PyZeppelinContext
> constructor, resulting in a seemingly empty class once bootstrap.py is
> processed.  It's still odd because again I can import bootstrap.py in my
> local python installation and use z.show() without IndentationErrors.
>
> On Fri, Aug 12, 2016 at 7:19 AM, Goodman, Alexander (398K) <
> alexander.good...@jpl.nasa.gov> wrote:
>
>> Hi,
>>
>> I am attempting to improve the matplotlib plotting capabilities in the
>> python interpreter (see the ZEPPELIN-1318 JIRA issue) and have ran into
>> some issues. Specifically, after saving my changes to bootstrap.py and
>> rebuilding, I get the following error message after running the unit tests:
>>
>> AttributeError: 'PyZeppelinContext' object has no attribute 'show' 
>> expected: but was:
>>
>>   PythonInterpreterPandasSqlTest.sqlOverTestDataPrintsTable:129 Traceback 
>> (most recent call last):
>>
>>   File "", line 1, in 
>>
>> The same error message also appeared when I attempted to run the simple
>> matplotlib example notebook. This seems to happen if I make any changes to
>> the show_matplotlib() method in bootstrap.py, including one line changes
>> that don't actually do anything. I also tested the module for syntax errors
>> in my local python installation and everything seems to check out, so I
>> can't understand where the problem lies. I am fairly certain my changes are
>> sound (see [1]). I tested this for a zeppelin build without any additional
>> options (mvn clean package -DskipTests) both with the modified and
>> unmodified source code, the latter having no issues.
>>
>> [1] - https://github.com/agoodm/zeppelin/commit/10c06a9ceb7be940
>> 85ea39ebaad8beab7e0ff024
>>
>> Any help would be appreciated.
>>
>> Thanks,
>> Alex
>>
>> --
>> Alex Goodman
>> Data Scientist I
>> Science Data Modeling and Computing (398K)
>> Jet Propulsion Laboratory
>> California Institute of Technology
>> Tel: +1-818-354-6012
>>
>
>
>
> --
> Alex Goodman
> Data Scientist I
> Science Data Modeling and Computing (398K)
> Jet Propulsion Laboratory
> California Institute of Technology
> Tel: +1-818-354-6012
>

Re: show() function not found by the python interpreter after modifying the source code

[Alexander Bezzubov]

Hi Alex,
answer inline

On Sat, Aug 13, 2016 at 1:15 PM, Goodman, Alexander (398K)  wrote:

> Thank you guys for the responses.
>
> I wonder how much the editor has to do with it since I was able to get my
> changes to work (ie the ones in my PR) after manually adding them in a
> vanilla editor (I was previously using Atom). However I did later have
> issues similar to those mentioned in ZEPPELIN-1133 when importing some
> notebooks that I had previously used with the pyspark interpreter in
> zeppelin 0.5 and replacing each cell with the python interpreter. If you
> are interested, I would be happy to share them with you.
>

It would be very appreciated indeed if you could followup on the issues
that you had \w some reproducible examples i.e in comments in ZEPPELIN-1133

In general I also think we should continue to try taking matplotlib
> integration much further, hopefully to the point where its inline plotting
> features in zeppelin are comparable to those found in Jupyter.
>

I totally agree. Could you elaborate a bit on what "inline plotting
features found in Jupyter" do you mean here?


> The latter obviously has the advantage of having more direct python and
> javascript integration, but nonetheless this is a subject I would
> personally be very interested in exploring in more detail.
>

I'll be happy to help here, let's join forces. What kind of
python\javascript integration do you think we should have in Apache
Zeppelin? May be we should start a new thread discussing it in more details.


> Thanks,
> Alex
>
> On Fri, Aug 12, 2016 at 8:53 PM, Alexander Bezzubov 
> wrote:
>
>> Hi Alexander,
>>
>> thank you for contributing Matplotlib integration improvements!
>>
>> You are right, it's odd and this is not your text editor's fault - our
>> current Python interpreter implementation is quite early stages and these
>> are artefacts of the way it currently works. Sometimes it does not accept
>> valid Python files [1].
>>
>> I'm working on improving it right now [2], by changing the underneath
>> mechanism of communication with Python process, which should address this
>> issues, as well as few others [3], like windows support, better error
>> detection, etc.
>>
>> I hope to submit a PR next week, meanwhile the best way to debug your
>> changes is to enable logging [4] by adding a line to log4j.properties "
>> log4j.logger.org.apache.zeppelin.python.PythonInterpreter=DEBUG" (i.e by
>> creating it in ./python/src/test/resources/ if you want it to work only
>> in unit-tests, or just changing code to `log.warn` to always log it) in
>> order to be able to see what exactly actual Python repl process is not
>> happy about.
>>
>> Hope this helps!
>>
>>  1. https://issues.apache.org/jira/browse/ZEPPELIN-1133
>>  2. https://issues.apache.org/jira/browse/ZEPPELIN-1325
>>  3. https://issues.apache.org/jira/browse/ZEPPELIN-1133?jql=
>> project%20%3D%20ZEPPELIN%20AND%20component%20%3D%20python-in
>> terpreter%20AND%20resolution%20%3D%20Unresolved
>>  4. https://github.com/apache/zeppelin/blob/master/python/src
>> /main/java/org/apache/zeppelin/python/PythonInterpreter.java#L221
>>
>> --
>> Alex
>>
>> On Sat, Aug 13, 2016, 01:04 Goodman, Alexander (398K) > jpl.nasa.gov> wrote:
>>
>>> An update: It seems like the problem might lie in my editor. After a bit
>>> of hacking into the log fiIes, I found that there were several
>>> IndentationErrors being reported, starting at the PyZeppelinContext
>>> constructor, resulting in a seemingly empty class once bootstrap.py is
>>> processed.  It's still odd because again I can import bootstrap.py in my
>>> local python installation and use z.show() without IndentationErrors.
>>>
>>> On Fri, Aug 12, 2016 at 7:19 AM, Goodman, Alexander (398K) <
>>> alexander.good...@jpl.nasa.gov> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am attempting to improve the matplotlib plotting capabilities in the
>>>> python interpreter (see the ZEPPELIN-1318 JIRA issue) and have ran into
>>>> some issues. Specifically, after saving my changes to bootstrap.py and
>>>> rebuilding, I get the following error message after running the unit tests:
>>>>
>>>> AttributeError: 'PyZeppelinContext' object has no attribute 'show' 
>>>> expected: but was:
>>>>
>>>>   PythonInterpreterPandasSqlTest.sqlOverTestDataPrintsTable:129 Traceback 
>>>> (most recent call last):
>>>>
>>>>   File "", l

Re: show() function not found by the python interpreter after modifying the source code

[Alexander Bezzubov]

Thank you for detailed explanation, Alex! I see what you mean now and that
would be splendid to have such level of integration.

I think a very good start will be - to create an umbrella JIRA issue, put
the detailed description there, and then post the link to the dev@z.i.a.o
mailing list, moving further discussion there.

I have some experience with Python, but not familiar with internals of
matplotlib yet. Would be happy to help on the JVM\Javascript side of things.

--
Alex

On Sun, Aug 14, 2016 at 1:07 PM, Goodman, Alexander (398K) <
alexander.good...@jpl.nasa.gov> wrote:

> Hi Alex,
>
> On Sat, Aug 13, 2016 at 7:53 PM, Alexander Bezzubov 
> wrote:
>
> >
>
> > Hi Alex,
> > answer inline
> >
> > On Sat, Aug 13, 2016 at 1:15 PM, Goodman, Alexander (398K) <
> alexander.good...@jpl.nasa.gov> wrote:
>
> >>
>
> >> Thank you guys for the responses.
> >>
> >> I wonder how much the editor has to do with it since I was able to get
> my changes to work (ie the ones in my PR) after manually adding them in a
> vanilla editor (I was previously using Atom). However I did later have
> issues similar to those mentioned in ZEPPELIN-1133 when importing some
> notebooks that I had previously used with the pyspark interpreter in
> zeppelin 0.5 and replacing each cell with the python interpreter. If you
> are interested, I would be happy to share them with you.
> >
> >
> > It would be very appreciated indeed if you could followup on the issues
> that you had \w some reproducible examples i.e in comments in ZEPPELIN-1133
>
> Will do. I'll post the notebook in question in the JIRA thread a bit later
> and how to reproduce the errors I was getting.
>
>
> >
> >
>
> >> In general I also think we should continue to try taking matplotlib
> integration much further, hopefully to the point where its inline plotting
> features in zeppelin are comparable to those found in Jupyter.
>
>
> >
> >
> > I totally agree. Could you elaborate a bit on what "inline plotting
> features found in Jupyter" do you mean here?
>
> Essentially at we should continue to make matplotlib plotting feel more
> seamless. That means we should at the minimum try to accomplish what we
> have now using plt.show() instead of z.show(plt) (plt being
> matplotlib.pyplot of course). This will require implementing a custom
> matplotlib backend which defines the show() function specific for a
> zeppelin notebook. Jupyter / ipython accomplishes this through a custom
> "inline" matplotlib backend. It could also help pave the way for using
> matplotlib's interactive plotting features, which will require some
> javascript integration. matplotlib ships a backend that does this called
> "nbagg" which accomplishes this for Jupyter.
>
>
> >
> >
>
>
> >>
> >> The latter obviously has the advantage of having more direct python and
> javascript integration, but nonetheless this is a subject I would
> personally be very interested in exploring in more detail.
> >
> >
> > I'll be happy to help here, let's join forces. What kind of
> python\javascript integration do you think we should have in Apache
> Zeppelin? May be we should start a new thread discussing it in more details.
>
> Sure thing. Should I make a thread about this on the dev mailing list or
> make a new JIRA issue? I think I have already described the gist of it
> above, so for now I am thinking we should prioritize the potential
> development of a new matplotlib backend for zeppelin that shows static
> images. I am mainly a python specialist so I would be happy to collaborate.
>
>
>
> Thanks,
> Alex
> >> On Fri, Aug 12, 2016 at 8:53 PM, Alexander Bezzubov 
> wrote:
>
> >>>
>
> >>> Hi Alexander,
> >>>
> >>> thank you for contributing Matplotlib integration improvements!
> >>>
> >>> You are right, it's odd and this is not your text editor's fault - our
> current Python interpreter implementation is quite early stages and these
> are artefacts of the way it currently works. Sometimes it does not accept
> valid Python files [1].
> >>>
> >>> I'm working on improving it right now [2], by changing the underneath
> mechanism of communication with Python process, which should address this
> issues, as well as few others [3], like windows support, better error
> detection, etc.
> >>>
> >>> I hope to submit a PR next week, meanwhile the best way to debug your
> changes is to enable logging [4] by adding a line to log4j.properties "
> log4j.logge

Re: installing and running zeppelin on mac OS X

[Alexander Bezzubov]

Thank you for posting back the solution, Scott!

--
Alex

On Sun, Aug 14, 2016 at 3:53 PM, Scott C. Cote  wrote:

> solved the problem.
>
> traced the culprit to the .npm folder - for some reason, root owned the
> folders under it.  changed the ownership to my user and group and all the
> problems went away ….
>
> SCott
> Scott C. Cote
> scottcc...@gmail.com
> 972.672.6484
>
>
>
> On Aug 14, 2016, at 12:35 AM, Scott C. Cote  wrote:
>
> Hello,
>
> I’m trying to make Zeppelin via git/maven on my macbook air and the
> install fails with maven complaining at the “web application” project….
>
> [INFO] Copying zeppelin-alluxio-0.7.0-SNAPSHOT.jar to /Users/scottccote/
> DownloadInstalled/apache-zeppelin-0.6/zeppelin/alluxio/
> target/../../interpreter/alluxio/zeppelin-alluxio-0.7.0-SNAPSHOT.jar
> [INFO]
> [INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @
> zeppelin-alluxio ---
> [INFO]
>
> [INFO] 
> 
> [INFO] Building Zeppelin: web Application 0.7.0-SNAPSHOT
> [INFO] 
> 
> [INFO]
> [INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) @ zeppelin-web
> ---
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.13:check (checkstyle-fail-build) @
> zeppelin-web ---
> [INFO]
> [INFO]
> [INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @
> zeppelin-web ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 17 resources
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce) @ zeppelin-web
> ---
> [INFO]
> [INFO] --- frontend-maven-plugin:0.0.25:install-node-and-npm (install
> node and npm) @ zeppelin-web ---
> [INFO] Node v0.12.13 is already installed.
> [INFO] Found NPM version 2.15.0
> [INFO]
> [INFO] --- frontend-maven-plugin:0.0.25:npm (npm install) @ zeppelin-web
> ---
> [INFO] Running 'npm install --color=false' in /Users/scottccote/
> DownloadInstalled/apache-zeppelin-0.6/zeppelin/zeppelin-web
> [ERROR] npm ERR! Darwin 15.6.0
> [ERROR] npm ERR! argv "/Users/scottccote/DownloadInstalled/apache-
> zeppelin-0.6/zeppelin/zeppelin-web/node/node" "/Users/scottccote/
> DownloadInstalled/apache-zeppelin-0.6/zeppelin/zeppelin-web/node/node_modules/npm/bin/npm-cli.js"
> "install" "--color=false"
> [ERROR] npm ERR! node v0.12.13
> [ERROR] npm ERR! npm  v2.15.0
> [ERROR] npm ERR! path /Users/scottccote/.npm/karma-jasmine/0.1.6
> [ERROR] npm ERR! code EACCES
> [ERROR] npm ERR! errno -13
> [ERROR]
> [ERROR] npm ERR! Error: EACCES, mkdir '/Users/scottccote/.npm/karma-
> jasmine/0.1.6'
> [ERROR] npm ERR! at Error (native)
> [ERROR] npm ERR!  { [Error: EACCES, mkdir '/Users/scottccote/.npm/karma-
> jasmine/0.1.6']
> [ERROR] npm ERR!   errno: -13,
> [ERROR] npm ERR!   code: 'EACCES',
> [ERROR] npm ERR!   path: '/Users/scottccote/.npm/karma-jasmine/0.1.6',
> [ERROR] npm ERR!   parent: 'zeppelin-web' }
> [ERROR] npm ERR!
> [ERROR] npm ERR! Please try running this command again as
> root/Administrator.
> [ERROR]
> [ERROR] npm ERR! Please include the following file with any support
> request:
> [ERROR] npm ERR! /Users/scottccote/DownloadInstalled/apache-
> zeppelin-0.6/zeppelin/zeppelin-web/npm-debug.log
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Zeppelin ... SUCCESS [
> 2.919 s]
> [INFO] Zeppelin: Interpreter .. SUCCESS [
> 9.682 s]
> [INFO] Zeppelin: Zengine .. SUCCESS [
> 5.980 s]
> [INFO] Zeppelin: Display system apis .. SUCCESS [
> 13.025 s]
> [INFO] Zeppelin: Spark dependencies ... SUCCESS [
> 28.592 s]
> [INFO] Zeppelin: Spark  SUCCESS [
> 24.361 s]
> [INFO] Zeppelin: Markdown interpreter . SUCCESS [
> 0.380 s]
> [INFO] Zeppelin: Angular interpreter .. SUCCESS [
> 0.284 s]
> [INFO] Zeppelin: Shell interpreter  SUCCESS [
> 0.522 s]
> [INFO] Zeppelin: Livy interpreter . SUCCESS [
> 0.911 s]
> [INFO] Zeppelin: HBase interpreter  SUCCESS [
> 4.984 s]
> [INFO] Zeppelin: PostgreSQL interpreter ... SUCCESS [
> 0.574 s]
> [INFO] Zeppelin: JDBC interpreter . SUCCESS [
> 1.863 s]
> [INFO] Zeppelin: File System Interpreters . SUCCESS [
> 1.380 s]
> [INFO] Zeppelin: Flink  SUCCESS [
> 7.251 s]
> [INFO] Zeppelin: Apache Ignite interpreter  SUCCESS [
> 1.209 s]
> [INFO] Zeppelin: Kylin interpreter  SUCCESS [
> 0.615 s]
> [INFO] Zeppelin: Python interpreter ... SUCCESS [
> 0.588 s]
> [INFO] Zeppelin: Lens interpreter . SUCCESS [
> 3.467 s]
> [INFO] Zeppelin: Apache Cassandra 

Re: [ANNOUNCE] Apache Zeppelin 0.6.1 released

[Alexander Bezzubov]

Congratulations, everyone!

And thank you Mina for putting it all together!

https://twitter.com/ApacheZeppelin/status/765459508315303936

--
Alex

On Tue, Aug 16, 2016 at 4:57 PM, Mina Lee  wrote:

> The Apache Zeppelin community is pleased to announce the availability of
> the 0.6.1 release.
>
> Zeppelin is a collaborative data analytics and visualization tool for
> distributed, general-purpose data processing system such as Apache Spark,
> Apache Flink, etc.
>
> The community put significant effort into improving Apache Zeppelin since
> the last release, focusing on supporting Scala 2.11 and Spark 2.0.
> 22 contributors provided 80+ patches for new features, improvements
> and bug fixes. More than 60+ issues have been resolved.
>
> We encourage download the latest release from
> http://zeppelin.apache.org/download.html
>
> Release note is available at
> http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html
>
> We welcome your help and feedback. For more information on the project and
> how to get involved, visit our website at http://zeppelin.apache.org/
>
> Thanks to all users and contributors who have helped to improve Apache
> Zeppelin.
>
> Regards,
> The Apache Zeppelin community
>

Re: build issue, do i need root?

[Alexander Bezzubov]

Hi Mohit,

no, you do not need to be root in order to build Apache Zeppelin.

This looks like zeppelin-web module failed to build web application. This
might happen due to different reasons i.e one common is networking issues
with bower or npm access. Usually maven build logs list path to npm logs,
inside ./zeppelin-web/ where you can find more details about your
particular failure.

Please let me know if that helps!

--
Alex

On Sat, Aug 20, 2016 at 4:19 AM, Mohit Jaggi  wrote:

> Hi All,
> I cloned zeppelin from master and tried to build using the following
> commands:
>
> ./dev/change_scala_version.sh 2.11
> mvn clean package -Pspark-2.0 -Phadoop-2.6  -Ppyspark -Psparkr
> -Pscala-2.11 -DskipTests
>
> I get the following failure. Do I need root permission to build zeppelin?
>
>
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 20:55 min
> [INFO] Finished at: 2016-08-19T12:17:25-07:00
> [INFO] Final Memory: 167M/628M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> com.github.eirslett:frontend-maven-plugin:0.0.25:bower
> (bower install) on project zeppelin-web: Failed to run task: 'bower
> --allow-root install' failed. (error code 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
> MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR]   mvn  -rf :zeppelin-web
>
> Mohit Jaggi
> Founder,
> Data Orchard LLC
> www.dataorchardllc.com
>
>
>
>
>

Re: Can't download zeppelin 0.6.1 binary

[Alexander Bezzubov]

Hi,

hotfix for the website has just went live - download page should work again!

Thank you York for a notice and Damien for prompt responce and fixing the
issue.

--
Alex

On Wed, Sep 7, 2016 at 2:04 PM, Corneau Damien  wrote:

> Hi York,
>
> Thanks for reporting it, seems we had a small problem when compiling the
> website last time.
> We are working on a Hotfix right now, shouldn't take long before it's back
> up again.
> We will post in this thread once its done.
>
>
> On Wed, Sep 7, 2016 at 8:50 AM, York Huang 
> wrote:
>
>> Hi,
>>
>> I go to the link http://zeppelin.apache.org/download.html.
>> But I can't download zeppelin binary from there.
>>
>> Does anyone know why?
>>
>> Thanks,
>>
>> York
>>
>
>

Re: Using R package with hidden credentials

[Alexander Bezzubov]

Hi Kevin,

approach #2 sounds very reasonable and should work indeed.

Ahyoung, how do you think you could you take a quick look at the cases with
SparkR that Kevin described and see if that is documented\expected outcome?

--
Alex

On Fri, Sep 23, 2016 at 9:17 AM, Kevin Niemann 
wrote:

> I'm using the Spark R interpreter with e.g. the mongolite package which
> requires authentication to access MongoDB.
>
> require(mongolite)
> url <- "mongodb://username:passw...@some.server.com/DB"
> m <- mongo(collection = "events", db = "myDB", url = url, verbose = FALSE)
>
> I'm looking at different approaches to hide the username/password
> credentials from the user:
>
> 1. Hide editor from readers. Only allow Writers to see the code. Downside
> is the editor must remember to hide editor if making changes (potential for
> human error) and no one else can see the actual code to produce the report.
>
> 2. Use a form to require input username/password at runtime. I can't
> actually get .z.input("username") working in %spark.r paragraph. Angular
> forms didn't work for me either, not sure how to assign an input or angular
> binding to an R variable.
>
> 3. Use Data source authorization/credential API. Seems pretty much out of
> the question since Spark isn't passing the creds to the R function.
>
> Anything I missed?
>
> Lastly (somewhat unrelated), returning a data frame should show the
> display system, but it's not. e.g.
>
> %spark.r
> data(mtcars)
> mtcars
> Mazda RX4 21.0 6 160.0 110 3.90 2.620 16.46 0 1 4 4
>
>
> Thanks,
>

Re: Two different errors while executing Spark SQL queries against cached temp tables

[Alexander Bezzubov]

Hi Florian,

sorry for slow response, I guess the main reason for not much feedback here
is that its hard to reproduce the error you describe, as it does not happen
reliably even on your local environment.

java.lang.NoSuchMethodException: org.apache.spark.io.LZ4CompressionCodec

This can be a sign of Hadoop FS codec miss-configuration.

Could you share a bit more details on Zeppelin/Spark/Hadoop configuration
that you use?

What is your SPARK_HOME ? zeppelin-env.sh ? Do you use external Spark
cluster? Is it Spark standalone or yarn-client cluster configuration? You
have shared Spark interpreter logs, but just in case, is there anything
strange in Zeppelin server .log/.out ?

Details like this would enable more people to chime in and help.

--

Alex

On Wed, Nov 16, 2016, 12:31 Florian Schulz  wrote:

Hi,

can anyone help me with this? It is very anoying, because I get this error
very often (on my local maschine and also on a second vm). I use Zeppelin
0.6.2 with Spark 2.0 and Scala 2.11.


Best regards
Florian

*Gesendet:* Montag, 14. November 2016 um 20:45 Uhr
*Von:* "Florian Schulz" 
*An:* users@zeppelin.apache.org
*Betreff:* Two different errors while executing Spark SQL queries against
cached temp tables
Hi everyone,

I have some trouble while executing some Spark SQL queries against some
cached temp tables. I query different temp tables and while doing
aggregates etc., I often get these errors back:

java.lang.NoSuchMethodException:
org.apache.spark.io.LZ4CompressionCodec.(org.apache.spark.SparkConf)
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getConstructor(Class.java:1825)
at
org.apache.spark.io.CompressionCodec$.createCodec(CompressionCodec.scala:72)
at
org.apache.spark.io.CompressionCodec$.createCodec(CompressionCodec.scala:66)
at org.apache.spark.sql.execution.SparkPlan.org
$apache$spark$sql$execution$SparkPlan$$decodeUnsafeRows(SparkPlan.scala:265)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$executeTake$1.apply(SparkPlan.scala:351)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$executeTake$1.apply(SparkPlan.scala:350)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at
org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:350)
at
org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:39)
at
org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$execute$1$1.apply(Dataset.scala:2183)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:57)
at org.apache.spark.sql.Dataset.withNewExecutionId(Dataset.scala:2532)
at org.apache.spark.sql.Dataset.org
$apache$spark$sql$Dataset$$execute$1(Dataset.scala:2182)
at org.apache.spark.sql.Dataset.org
$apache$spark$sql$Dataset$$collect(Dataset.scala:2189)
at
org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:1925)
at
org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:1924)
at org.apache.spark.sql.Dataset.withTypedCallback(Dataset.scala:2562)
at org.apache.spark.sql.Dataset.head(Dataset.scala:1924)
at org.apache.spark.sql.Dataset.take(Dataset.scala:2139)
at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.zeppelin.spark.ZeppelinContext.showDF(ZeppelinContext.java:216)
at
org.apache.zeppelin.spark.SparkSqlInterpreter.interpret(SparkSqlInterpreter.java:129)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
  org.codehaus.janino.JaninoRuntimeException: Class
'org.apache.spark.sql.catalyst.expressions.codegen.GeneratedClass' was
loaded through a different loader
at
org.codehaus.janino.SimpleCompiler$1.getDelegate(SimpleCompiler.java:337)
at org.codehaus.janino.SimpleCompiler$1.accept(SimpleCompiler.java:291)
at org.codehaus.janino.UnitCo

Re: Unable to connect with Spark Interpreter

[Alexander Bezzubov]

Hi Jan,

this is rather generic error saying that ZeppelinServer somehow could not
connect to the interpreter proces on your machine.

Could you please share more from logs/* in particular, .out and .log of the
Zeppelin server AND zepplein-interpreter-spark*.log - usually this is
enough to identify the reason.

Two more questions:
- does this happen on every paragraph run? if you try to click Run multiple
times in a row
- does it still happen if you re-starting Spark interpreter manually from
GUI? ("Anonymous"->Interpreters->Spark->restart)

--
Alex

On Wed, Nov 16, 2016, 12:37 Jan Botorek  wrote:

> Hello,
>
> I am not able to run any Spark code in the Zeppelin. I tried compiled
> versions of Zeppelin as well as to compile the source code on my own based
> on the https://github.com/apache/zeppelin steps.
>
> My configuration is Scala in 2.11 version and spark 2.0.1. Also, I tried
> different versions of Zeppelin available at github (master, 0.6, 0.5.6).
>
>
>
> The result is always the same. The Zeppelin starts but when any code is
> run (e.g. “2 + 1”, “sc.version”), the subsequent exception is thrown.
>
>
>
> java.net.ConnectException: Connection refused: connect at
> java.net.DualStackPlainSocketImpl.connect0(Native Method) at
> java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) at
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
> java.net.Socket.connect(Socket.java:589) at
> org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:176) at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> Based on googling and my assumptions, there is something wrong with the
> spark interpreter in relation to the Zeppelin.
>
> I also tried to connect the Spark interpreter to Spark running externally
> (in interpreter settings of Zeppelin), but it didn’t work.
>
>
>
> Do you have any ideas about what could possibly be wrong?
>
> Thank you for any help – any ideas and insights would be appreciated.
>
>
>
> Best regards,
>
> Jan Botorek
>
>
>

Re: Jetty hangs, and Zeppelin hangs thereafter

[Alexander Bezzubov]

Hi,

thank you for the feedback!
CC dev@z.a.o list as its relevant for project developers.

Do you experience same issue with the latest 0.6.2 release? There were
number of bugfixes sinse 0.6.0

New release 0.7.0 will be done from the latest master, so if that is
possible, please try it as well ansd let the community know about results.

If the issue persists, the fatest way would be to submit a PR with the
potential fix and other committers will be happy to help/review/merge it to
the projetc's codebase.

--
Alex

On Fri, Nov 18, 2016, 21:36 Zhe Sun  wrote:

> Dear Zeppeliners,
>
> *1. For Zeppelin 0.6.0, I found it's very likely to hang at
> the synchronized (noteSocketMap), for example:*
> "Thread-133" #189 prio=5 os_prio=0 tid=0x7efc14001000 nid=0x7d3f
> waiting for monitor entry [0x7efc56cef000]
> *   java.lang.Thread.State: BLOCKED (on object monitor)*
> * at
> org.apache.zeppelin.socket.NotebookServer.broadcast(NotebookServer.java:301)*
> * - waiting to lock <0x7efe1d340c90> (a java.util.HashMap)*
> at
> org.apache.zeppelin.socket.NotebookServer.broadcastNote(NotebookServer.java:389)
> at
> org.apache.zeppelin.socket.NotebookServer$ParagraphListenerImpl.afterStatusChange(NotebookServer.java:1148)
> at org.apache.zeppelin.scheduler.Job.setStatus(Job.java:150)
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.afterStatusChange(RemoteScheduler.java:408)
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:272)
> - locked <0x7f08e9f9fb50> (a
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller)
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:211)
>
> *2. Then I found the root cause is that
> Jetty org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block hangs,
> therefore Zeppelin hangs:*
> "Thread-132" #187 prio=5 os_prio=0 tid=0x7efc7001e800 nid=0x7d3a
> waiting on condition [0x7efcc0bd6000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x7f08e9ca9de8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> *at
> org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)*
> at
> org.eclipse.jetty.websocket.common.BlockingWriteCallback$WriteBlocker.block(BlockingWriteCallback.java:83)
> at
> org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.blockingWrite(WebSocketRemoteEndpoint.java:107)
> at
> org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.sendString(WebSocketRemoteEndpoint.java:387)
> *at org.apache.zeppelin.socket.NotebookSocket.send(NotebookSocket.java:69)*
> at
> org.apache.zeppelin.socket.NotebookServer.broadcast(NotebookServer.java:308)
> - locked <0x7efe1d340c90> (a java.util.HashMap)
> at
> org.apache.zeppelin.socket.NotebookServer.access$000(NotebookServer.java:62)
> at
> org.apache.zeppelin.socket.NotebookServer$ParagraphListenerImpl.onProgressUpdate(NotebookServer.java:1121)
> at
> org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgressPoller.java:51)
>
> *3. Finally, I switch to the latest Jetty: *
> 9.3.14.v20161028
> The hang has not occurred again so far.
>
> Please consider to upgrade Jetty in next release, thank you.
>
> Best,
> Zell
>

Re: Is it possible to run Java code on Zeppelin Notebook?

[Alexander Bezzubov]

Good question :)

Actually, there is a not very well known yet "hack" (I talked about it a
bit on ApacheCon this year) - to run a pure Java paragraph in Apache
Zeppelin - you can just use `%beam` interpreter!

Beam interpreter uses Beam Java API, so you can leverage it i.e to run WEKA
machine learning library (pure Java) in Zeppelin as below

```
%beam

import java.io.BufferedReader;
import java.io.FileReader;

import weka.classifiers.Classifier;
import weka.classifiers.lazy.IBk;
import weka.core.Instance;
import weka.core.Instances;
import weka.core.Instances;

public class KNN {

public static void main(String[] args) throws Exception {
BufferedReader datafile = new BufferedReader(new
FileReader("/home/ubuntu/ads-weka.txt"));

Instances data = new Instances(datafile);
data.setClassIndex(data.numAttributes() - 1);

//do not use first and second
Instance first = data.instance(2);
Instance second = data.instance(3);
data.delete(2);
data.delete(3);

Classifier ibk = new IBk();
ibk.buildClassifier(data);

double class1 = ibk.classifyInstance(first);
double class2 = ibk.classifyInstance(second);

System.out.println("first: " + class1 + "\nsecond: " + class2);
}
}
```

It will compile the code the classes and execute it.

You just need to remember to build Zeppelin with `-Pbeam` (may be can be
added to the release convenience binary as well?)

--
Alex

On Sun, Nov 20, 2016 at 11:42 AM, DuyHai Doan  wrote:

> Or wait for Java9, there will be a REPL for Java
>
> On Sun, Nov 20, 2016 at 11:40 AM, Felix Cheung 
> wrote:
>
>> I think you will need to convert Java code into Scala syntax? But Scala
>> can call into Java libraries and so on.
>>
>> I don't think we have an interpreter for Java since it does not come with
>> a REPL until Java 9?
>>
>>
>> --
>> *From:* Abhisar Mohapatra 
>> *Sent:* Thursday, November 17, 2016 9:23:24 AM
>> *To:* users@zeppelin.apache.org; Muhammad Rezaul Karim
>> *Subject:* Re: Is it possible to run Java code on Zeppelin Notebook?
>>
>> Yes it will. I guess there are some implementations too
>>
>> On Thu, Nov 17, 2016 at 10:41 PM, Muhammad Rezaul Karim <
>> reza_cse...@yahoo.com> wrote:
>>
>>> Hi All,
>>>
>>> I am a new user of Zeppelin and got to know that Apache Zeppelin is
>>> using Spark as the backend interpreter.
>>>
>>> Till date, I have run some codes written in Scala on the Zeppelin
>>> notebook. However, I am pretty familiar with writing Spark application
>>> using Java.
>>> Now my question: is it possible to run Java code on Zeppelin Notebook?
>>>
>>>
>>>
>>> Thanks and Regards,
>>> -
>>> *Md. Rezaul Karim*
>>> PhD Researcher, Insight Centre for Data Analytics
>>> National University of Ireland Galway
>>> *E-mail:* rezaul.ka...@insight-centre.org
>>> 
>>> *Web*: www.insight-centre.org
>>> *Phone:* +353892311519
>>>
>>
>>
>> _
>> The information contained in this communication is intended solely for
>> the use of the individual or entity to whom it is addressed and others
>> authorized to receive it. It may contain confidential or legally privileged
>> information. If you are not the intended recipient you are hereby notified
>> that any disclosure, copying, distribution or taking any action in reliance
>> on the contents of this information is strictly prohibited and may be
>> unlawful. If you have received this communication in error, please notify
>> us immediately by responding to this email and then delete it from your
>> system. The firm is neither liable for the proper and complete transmission
>> of the information contained in this communication nor for any delay in its
>> receipt.
>>
>
>

Re: Adjust Height of Data Visualization

[Alexander Bezzubov]

Hi,

it makes sence - did you try draghing a paragraph lower right conner to
adjust the output height?

--
Alex

On Sun, Nov 20, 2016, 16:44 s r  wrote:

> Hi, Sorry if I'm missing something obvious
> I've started using Zeppelin for data exploration and would like to
> have a bigger (higher) visualization widget to display more rows.
> Any ideas?
>

Re: Configuring table format/type detection

[Alexander Bezzubov]

Hi Everett,

this is a very good question actually.

Right now there is not, but it sounds as a great feature, so may be worth
filing a JIRA issue.

There was a discussion when this feature was contributed [1], [2] and there
also is some work on having ability to manually override text\number column
type from GUI, but AFAIK that was not finished yet.

 1. https://github.com/apache/zeppelin/pull/1371
 2. https://github.com/apache/zeppelin/pull/1363
 3. https://issues.apache.org/jira/browse/ZEPPELIN-1371

--
Alex

On Thu, Nov 24, 2016 at 10:19 AM, Everett Anderson  wrote:

> Hi,
>
> I've been using Zeppelin with Spark SQL recently.
>
> One thing I've noticed that can be confusing is that Zeppelin attempts to
> detect the type of column and format it. For example, for columns that
> appear to have mostly numbers, it will put in commas.
>
> Is there a way to configure it globally or per-interpreter type to disable
> this?
>
>

Re: Session Clustering in Zeppelin

[Alexander Bezzubov]

Hi Peter,

we at NFLabs did something like what you describe last year for a few
on-prem customers and even open sourced our reverse proxy solution for
that, calling it "Z-Manager multitenancy" [1]. We also talked about it at
ApachCon EU 2015 in Budapest [2].

It was done before Shiro auth implementation in Z, so it supposed to have
auth basic and ZeppelinHub support only. Its not polished or an actively
maintened project right now as company's shifted focus from on-prem to
cloud solutions.

Check it out and let us know what you think!

As z-mamager is not an Apache project, please feel free to take further
questions (in case of any) to a private discussion and reach anybody
@nflabs like me or Moon or anyone else in person.

Hope this helps and looking forward other solutions people use to solve
this problem these days!


 1. https://github.com/NFLabs/z-manager/blob/master/README.md#multi-tenancy
 2.
http://events.linuxfoundation.org/sites/events/files/slides/Zeppelin-AppacheCon_final.pdf

--
Alex

On Fri, Dec 2, 2016, 05:04 Peter McCaffrey  wrote:

Hey Everyone,
 I've been using Zeppelin for about 7 months now and I'm a huge fan.
We've been using zeppelin in docker for a while but now we want to scale
this out to span several docker containers behind our reverse proxy with
session clustering between them. From the Shiro docs it looks like setting
this up is certainly possible as far as Shiro is concerned but I wanted to
know if anyone has done this sort of thing with Zeppelin and whether
Zeppelin can support this?
Sincerely,
-Peter

Re: Interpreter zombie processes

[Alexander Bezzubov]

Have similar experience, although hard to say what's the reason as all
process supposed to killed, as Moon pointed out.
Also noticed that with `mvn tests`, after almost every run, there are 1-2
zombie RemoteInterpreter processes hanging around.

--
Alex

On Fri, Dec 16, 2016 at 11:24 AM, Jeff Zhang  wrote:

>
> I believe I see this before too.
>
>
> Luciano Resende 于2016年12月16日周五 上午10:17写道：
>
>> I have also seen similar issues even using zeppelin-ddeamon but didn't
>> have much time to investigate the issue when it was happening to me.
>>
>> On Thu, Dec 15, 2016 at 12:15 PM Ruslan Dautkhanov 
>> wrote:
>>
>> Moon,
>>
>> > ZeppelinServer try to terminate interpreter process when shutting down
>> [1].
>>
>> Unfortunatally, this does not happen all the time.
>> I have seen zombie spark interpreter processes many times.
>> As a double conirmation - I see spark yarn application was still running.
>>
>> > Also bin/zeppelin-deamon.sh script clean up if some processes are left
>> [2].
>>
>> We were not using that script. We were just running like zeppelin.sh
>> --config ..
>> Will try to switch to zeppelin-deamon.sh and see if it makes a
>> difference.
>>
>> Thanks!
>>
>>
>>
>>
>> --
>> Ruslan Dautkhanov
>>
>>
>>
>> On Thu, Dec 15, 2016 at 12:46 PM, moon  wrote:
>>
>> Hi,
>>
>>
>>
>>
>>
>> ZeppelinServer try to terminate interpreter process when shutting down
>> [1].
>>
>>
>> Also bin/zeppelin-deamon.sh script clean up if some processes are left
>> [2].
>>
>>
>>
>>
>>
>> If some processes are remained after shutting down Zeppelin, that's not an
>>
>>
>> expected result.
>>
>>
>>
>>
>>
>> Thanks,
>>
>>
>> moon
>>
>>
>>
>>
>>
>> [1]
>>
>>
>> https://github.com/apache/zeppelin/blob/master/zeppelin-
>> server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java#L159
>>
>>
>> [2]
>>
>>
>> https://github.com/apache/zeppelin/blob/master/bin/
>> zeppelin-daemon.sh#L218
>>
>>
>>
>>
>>
>> On Thu, Dec 15, 2016 at 11:25 AM Ruslan Dautkhanov 
>>
>>
>> wrote:
>>
>>
>>
>>
>>
>> > How to make Interpreter processes terminate if Zeppelin server exits?
>>
>>
>> >
>>
>>
>> > When we restart Zeppelin server (main process), in many cases,
>> Interpreter
>>
>>
>> > process keeps running, essential becoming a zombie processes.
>>
>>
>> >
>>
>>
>> > In case of Spark interpreter, it also holds SparkContext - consuming
>>
>>
>> > server-side
>>
>>
>> > resources too.
>>
>>
>> >
>>
>>
>> > How do we configure Zeppelin to kill all interpreters before its main
>>
>>
>> > process terminates?
>>
>>
>> > Or is this a bug?
>>
>>
>> >
>>
>>
>> > We're running ~2 weeks old snapshot of 0.7.0 Zeppelin.
>>
>>
>> >
>>
>>
>> >
>>
>>
>> > Thank you,
>>
>>
>> > Ruslan Dautkhanov
>>
>>
>> >
>>
>>
>>
>> --
>> Sent from my Mobile device
>>
>

Re: Build error on windows

[Alexander Bezzubov]

Hi Jeff,


looks like something failed during `grunt build`.
Could you also show the content of
`C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log`?
It might have some details to give a hint on the issue.

--
Alex

On Sun, Dec 18, 2016 at 10:46 PM, Jeff Zhang  wrote:

> I try to build zeppelin on windows, but hit the following error. Does
> anyone know how to address this ? Thanks
>
>
> [ERROR] npm ERR! Windows_NT 6.1.7601
> [ERROR] npm ERR! argv
> "C:\\Users\\jzhang\\zeppelin\\zeppelin-web\\node\\node.exe"
> "C:\\Users\\jzhang\\zeppelin\\zeppelin
> -web\\node\\node_modules\\npm\\bin\\npm-cli.js" "run" "build"
> [ERROR] npm ERR! node v6.9.1
> [ERROR] npm ERR! npm  v3.10.8
> [ERROR] npm ERR! code ELIFECYCLE
> [ERROR] npm ERR! zeppelin-web@0.0.0 build: `grunt build`
> [ERROR] npm ERR! Exit status 3
> [ERROR] npm ERR!
> [ERROR] npm ERR! Failed at the zeppelin-web@0.0.0 build script 'grunt
> build'.
> [ERROR] npm ERR! Make sure you have the latest version of node.js and npm
> installed.
> [ERROR] npm ERR! If you do, this is most likely a problem with the
> zeppelin-web package,
> [ERROR] npm ERR! not with npm itself.
> [ERROR] npm ERR! Tell the author that this fails on your system:
> [ERROR] npm ERR! grunt build
> [ERROR] npm ERR! You can get information on how to open an issue for this
> project with:
> [ERROR] npm ERR! npm bugs zeppelin-web
> [ERROR] npm ERR! Or if that isn't available, you can get their info via:
> [ERROR] npm ERR! npm owner ls zeppelin-web
> [ERROR] npm ERR! There is likely additional logging output above.
> [ERROR]
> [ERROR] npm ERR! Please include the following file with any support
> request:
> [ERROR] npm ERR! C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log
> [INFO]
> 
>

Re: Re: Build error on windows

[Alexander Bezzubov]

Thanks for verification and a quick fix, it has been merged to master.

--
Alex

On Mon, Dec 19, 2016 at 3:39 PM, Jeff Zhang  wrote:

> Thanks, This PR resolve my issue.
>
>
>
> zjp_j...@163.com 于2016年12月19日周一 上午10:21写道：
>
> > I think the following issue can help u.
> > - JIRA : https://issues.apache.org/jira/browse/ZEPPELIN-1239
> > - PR : https://github.com/apache/zeppelin/pull/1731
> >
> > --
> > zjp_j...@163.com
> >
> >
> > *From:* Jeff Zhang 
> > *Date:* 2016-12-19 09:27
> > *To:* users ; dev 
> > *Subject:* Re: Build error on windows
> >
> > Attach npm-debug.log
> >
> >
> > Alexander Bezzubov 于2016年12月18日周日 下午10:34写道：
> >
> > Hi Jeff,
> >
> >
> > looks like something failed during `grunt build`.
> > Could you also show the content of `C:\Users\jzhang\zeppelin\
> zeppelin-web\npm-debug.log`?
> > It might have some details to give a hint on the issue.
> >
> > --
> > Alex
> >
> > On Sun, Dec 18, 2016 at 10:46 PM, Jeff Zhang  wrote:
> >
> > I try to build zeppelin on windows, but hit the following error. Does
> > anyone know how to address this ? Thanks
> >
> >
> > [ERROR] npm ERR! Windows_NT 6.1.7601
> > [ERROR] npm ERR! argv
> > "C:\\Users\\jzhang\\zeppelin\\zeppelin-web\\node\\node.exe"
> > "C:\\Users\\jzhang\\zeppelin\\zeppelin
> > -web\\node\\node_modules\\npm\\bin\\npm-cli.js" "run" "build"
> > [ERROR] npm ERR! node v6.9.1
> > [ERROR] npm ERR! npm  v3.10.8
> > [ERROR] npm ERR! code ELIFECYCLE
> > [ERROR] npm ERR! zeppelin-web@0.0.0 build: `grunt build`
> > [ERROR] npm ERR! Exit status 3
> > [ERROR] npm ERR!
> > [ERROR] npm ERR! Failed at the zeppelin-web@0.0.0 build script 'grunt
> > build'.
> > [ERROR] npm ERR! Make sure you have the latest version of node.js and npm
> > installed.
> > [ERROR] npm ERR! If you do, this is most likely a problem with the
> > zeppelin-web package,
> > [ERROR] npm ERR! not with npm itself.
> > [ERROR] npm ERR! Tell the author that this fails on your system:
> > [ERROR] npm ERR! grunt build
> > [ERROR] npm ERR! You can get information on how to open an issue for this
> > project with:
> > [ERROR] npm ERR! npm bugs zeppelin-web
> > [ERROR] npm ERR! Or if that isn't available, you can get their info via:
> > [ERROR] npm ERR! npm owner ls zeppelin-web
> > [ERROR] npm ERR! There is likely additional logging output above.
> > [ERROR]
> > [ERROR] npm ERR! Please include the following file with any support
> > request:
> > [ERROR] npm ERR! C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log
> > [INFO]
> > 
> >
> >
> >
>

Re: 'File size limit Exceeded' when importing notes - even for small files

[Alexander Bezzubov]

Hi,

this deflinitly looks like a regredsion/bug,
Ruslan, would you mind creating a JIRA issue?

Paul, thanks for sharing notebook size reduction pro-tip!

--
Alex

On Wed, Jan 18, 2017, 10:04 Paul Brenner  wrote:

> Just a tip that when I ran into this problem I found that using the “clear
> output” button and then exporting my notebook made it easy to get below the
> size limit. Not very helpful if you need ALL the output, but maybe you can
> selectively clear output from some paragraphs?
>
>  
>  Paul Brenner 
>  
>  
> 
> 
> DATA SCIENTIST
> *(217) 390-3033 *
>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> [image:
> PlaceIQ:Location Data Accuracy]
> 
>
> On Tue, Jan 17, 2017 at 4:55 PM Ruslan Dautkhanov  > wrote:
>
> From the screenshot "JSON file size cannot exceed MB".
> Notice there is no number between "exceed" and "MB".
> Not sure if we're missing a setting or an environment variable to define
> the limit?
> It now prevents us from importing any notebooks.
>
>
>
> --
> Ruslan Dautkhanov
>
> On Tue, Jan 17, 2017 at 11:54 AM, Ruslan Dautkhanov 
> wrote:
>
> 'File size limit Exceeded' when importing notes - even for small files
>
> This happens even for tiny files - a few Kb.
>
> Is this a known issue?
>
> Running Zeppelin 0.7.0 from a few weeks old snapshot.
>
> See attached screenshot.
>
>
> --
> Ruslan Dautkhanov
>
>
>
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

[Alexander Bezzubov]

Hi Jeff,

thank you for the notice, it is a good catch!

But the issue that you mention does not have FixFor version assigned - if
you want it to part of the release and even conside it as important enough
to block the relese vote, could you please update that?

This would simplify drastically the work of release manager.

Also, as you know, the release branch has been unnounced a while a go, but
for this issue as far as I can tell, there was no any indication that it
was intended to be part of it, so seeing -1 is a bit surprising. Do you see
what I mean?

Release Manager can not read minds, as well as volonteer infinite ammount
of time preparing a new Relese Candidate after each issue beeing solved.

Cutting a release is a lot of work for Release Manger, so going further
lets try to simplify life of the brave volonteer helping with release
management! :)

Hope it does make sense! Please, let us know what you think.

--
Alex

On Sat, Jan 21, 2017, 21:55 Jeff Zhang  wrote:

> -1,  Can we include ZEPPELIN-1985
>  ? This issue cause
> only user1 can see the pig tutorial note which is a very bad user
> experience.
>
>
>
> Prabhjyot Singh 于2017年1月21日周六 下午7:25写道：
>
> > +1
> >
> > On Jan 21, 2017 4:52 PM, "Jun Kim"  wrote:
> >
> > +1
> > 2017년 1월 21일 (토) 오후 8:21, CloverHearts 님이 작성:
> >
> > +1
> >
> >
> >
> > The Zeppelin 0.7 release includes some amazing features.
> >
> > In particular, a major bug fix for the remote interpreter.
> >
> > I am actively in favor of this release.
> >
> >
> >
> >
> >
> > *출발**: *Mina Lee 
> > *회신* *대상**:* 
> > *날짜**: *2017년 1월 21일 토요일 오전 3:17
> > *받는* *사람**: *, 
> > *주제**: *[VOTE] Release Apache Zeppelin 0.7.0 (RC1)
> >
> >
> >
> > Hi folks,
> >
> > I propose the following RC to be released for the Apache Zeppelin 0.7.0
> > release.
> >
> > The commit id is bcbe0855175f55eddd10b918303806eb5e5db091 which is
> > corresponds to the tag v0.7.0-rc1:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=bcbe0855175f55eddd10b918303806eb5e5db091
> >
> > The release archives (tgz), signature, and checksums are here
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc1/
> >
> > The release candidate consists of the following source distribution
> > archive
> >
> > zeppelin-0.7.0.tgz
> >
> > In addition, the following supplementary binary distributions are
> provided
> > for user convenience at the same location
> >
> > zeppelin-0.7.0-bin-all.tgz
> > zeppelin-0.7.0-bin-netinst.tgz
> >
> >
> > The maven artifacts are here
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1022
> >
> > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >
> > Release notes available at
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544&projectId=12316221
> >
> > Vote will be open for next 72 hours (close at 3:30 am 24/Jan PST).
> >
> > [ ] +1 approve
> > [ ] 0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > --
> > Taejun Kim
> >
> > Data Mining Lab.
> > School of Electrical and Computer Engineering
> > University of Seoul
> >
> >
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC4)

[Alexander Bezzubov]

+1,

and thank you for an awesome work Mina!
Your persistence in making RCs and incorporating feedback is admirable.

Verified:
 - checksums, signatures + keys for sources and bin-all
 - bin-all can run all Spark Zeppelin Tutorial in local mode
 - sources do compile, but only without tests.
   Build \w tests fails on zeppelin-zengine for me

1) Failed tests:
  NotebookTest.testSchedulePoolUsage:397 expected: but
was:

2) frontend build on Linux also failed mysteriously executing yarn command,
most probably due to local env configuration.


--
Alex

On Thu, Feb 2, 2017 at 5:28 PM, Jeff Zhang  wrote:

> +1
>
> Ahyoung Ryu 于2017年2月2日周四 下午9:36写道：
>
> +1
>
> On Thu, Feb 2, 2017 at 10:07 PM, Jun Kim  wrote:
>
> +1
> 2017년 2월 2일 (목) 오후 9:49, Sora Lee 님이 작성:
>
> +1
>
> On Thu, Feb 2, 2017 at 9:40 PM Khalid Huseynov  wrote:
>
> > +1
> >
> > On Thu, Feb 2, 2017 at 9:21 PM, DuyHai Doan 
> wrote:
> >
> > +1
> >
> > On Thu, Feb 2, 2017 at 9:56 AM, Mina Lee  wrote:
> >
> > > I propose the following RC to be released for the Apache Zeppelin 0.7.0
> > > release.
> > >
> > > The commit id is df007f2284a09caa7c8b35f8b59d5f1993fe8b64 which is
> > > corresponds to the tag v0.7.0-rc4:
> > > *
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=sho
> rtlog;h=refs/tags/v0.7.0-rc4
> > > <
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=sho
> rtlog;h=refs/tags/v0.7.0-rc4
> > >*
> > >
> > > The release archives (tgz), signature, and checksums are here
> > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc4/
> > >
> > > The release candidate consists of the following source distribution
> > > archive
> > > zeppelin-0.7.0.tgz
> > >
> > > In addition, the following supplementary binary distributions are
> > provided
> > > for user convenience at the same location
> > > zeppelin-0.7.0-bin-all.tgz
> > > zeppelin-0.7.0-bin-netinst.tgz
> > >
> > > The maven artifacts are here
> > >
> > https://repository.apache.org/content/repositories/orgapache
> zeppelin-1027
> > >
> > > You can find the KEYS file here:
> > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > >
> > > Release notes available at
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > version=12336544&projectId=12316221
> > >
> > > Vote will be open for next 72 hours (close at 01:00 5/Feb PST).
> > >
> > > [ ] +1 approve
> > > [ ] 0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> >
> >
> >
>
> --
> Taejun Kim
>
> Data Mining Lab.
> School of Electrical and Computer Engineering
> University of Seoul
>
>
>

Re: Zeppelin Notebook Storage in Google Cloud Storage

[Alexander Bezzubov]

Hi VInay,

AFAIK there is no such notebooks storage implementation yet. You can see
the source code for available storages under [0].

Although it should be not hard to add, implementing NotebookRepo methods
[1] by delegating to appropriate calls in GCS API [2] and may be adding
appropriate credentials to configuration. One example of using such library
and an auth is BigQuery interpreter [3].


 0.
https://github.com/apache/zeppelin/tree/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo
 1.
https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java
 2.
https://cloud.google.com/storage/docs/reference/libraries#client-libraries-install-java
 3. https://github.com/apache/zeppelin/tree/master/bigquery
--
Alex

On Tue, Apr 18, 2017 at 3:49 PM, Vinay Shukla  wrote:

> Does Zeppelin have functionality to store Notebooks in GCS via VFS?
>
> http://zeppelin.apache.org/docs/0.7.1/storage/storage.html Does not
> mention GCS?
>
> Thanks
>

Re: Zeppelin with Oauth2

[Alexander Bezzubov]

Hi Minwoo,

great question, AFAIK, there is no way right now.

But please, feel free to create and Issue and post the link here (or
starting a thread in Dev mailing list [0] could be good idea) and then, as
soon as you have PoC with the code working, submit a PR with the code and
some documentation on how to use\enable it!

I assume you are talking about user authentication, so you might want to
use existing Apache Shiro [1] facilities inside Zeppelin.

Good luck!

 0. http://zeppelin.apache.org/community.html#mailing-list
 1. https://github.com/apache/zeppelin/blob/master/docs/
security/shiroauthentication.md


--
Alex

On Thu, May 4, 2017 at 9:54 AM, Kang Minwoo  wrote:

> Hello
>
> Can I use Oauth2 in Zeppelin? (or any Idea?)
> If there is no way, Can I contribute that?
>
> Best regards,
> Minwoo Kang
>

Re: [DISCUSS] Reducing default interpreters while building and releasing Zeppelin

[Alexander Bezzubov]

Hey guys,

great effort! I think people in few other communities will be very happy
with it i.e [1] and [2].

Is there an issue that tracks current status or something like that? Does
anyone have concrete plans to work on it in this/next release?

Sorry if I have missed that out. And please, keep up a good work!


--

Alex


1. https://issues.apache.org/jira/browse/BIGTOP-2269
2.
https://github.com/kubernetes/kubernetes/tree/master/examples/spark#known-issues-with-zeppelin

On Mon, Jun 5, 2017, 06:54 Jongyoul Lee  wrote:

> Felix,
> Yes, I said a bit confused. I want to release Zeppelin with some - not-all
> - interpreters, but deploy all interpreters into maven to install them if
> users want to use them.
>
> Moon,
> I think it's the best to fit the list as same as homepage by default as it
> makes users confused less. But if we want to add more interpreters, I think
> mailing questions and related issues are one of the proper criteria.
>
> Jeff,
> Agreed. We already had a menu but it just shows how to use
> install-interpreter.sh.
>
> On Mon, Jun 5, 2017 at 9:36 AM, Jeff Zhang  wrote:
>
>>
>> If possible, we need to integrate the install script in zeppelin UI. As I
>> would expect many users would ask why some interpreter is missing and how
>> to install them.
>>
>>
>>
>> moon soo Lee 于2017年6月5日周一 上午2:06写道：
>>
>>> Following is last discussion related to release package size.
>>>
>>>
>>> https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
>>>
>>> at this time, we have discussed about having bin-all (every
>>> interpreters), bin-min (selected interpreters), bin-netinst (no
>>> interpreters) package but didn't conclude the criteria and how we make a
>>> decision.
>>>
>>> Jongyoul, do you have any idea about criteria?
>>>
>>> Thanks,
>>> moon
>>>
>>> On Sun, Jun 4, 2017 at 10:47 AM Felix Cheung 
>>> wrote:
>>>
 Sure - I think it will be important to discuss what criteria to use to
 decide what is included vs what will be released separately.

 _
 From: Jongyoul Lee 

>>> Sent: Sunday, June 4, 2017 9:47 AM
 Subject: Re: [DISCUSS] Reducing default interpreters while building and
 releasing Zeppelin
 To: dev 

>>> Cc: 

>>>


 It means we release with some interpreters and deploy all interpreters
 into
 maven separately. We already had a install-interpreter script inside
 it. If
 someone wants to install specific interpreter not included in default
 release package, they can use that script to install specific one.

 On Sun, Jun 4, 2017 at 9:11 AM, Felix Cheung >>> >
 wrote:

 > Are we proposing some interpreters to be built and released
 separately?
 >
 > Is this going to be separate packaging? Or separate release pipeline?
 >
 >
 > _
 > From: Jongyoul Lee mailto:jongy...@gmail.com
 >>
 > Sent: Friday, June 2, 2017 11:04 PM
 > Subject: [DISCUSS] Reducing default interpreters while building and
 > releasing Zeppelin

>>> > To: dev mailto:d...@zeppelin.apache.org
 >>, <

>>>
 > users@zeppelin.apache.org>
 >
 >
 > Hi dev and users,
 >

>>> > Recently, zeppelin.apache.org is being

>>>
 > changed for increasing user experiences and convenience. I like this
 kind
 > of changes. I, however, saw some arguments that which interpreters we
 will
 > locate in the first page. I'd like to expand its argument to the
 package we
 > release.
 >
 > Current zeppelin packages exceed 700MB with default option because
 > Zeppelin tried to include all interpreters by default. It was good at
 the
 > early age but, nowadays, Zeppelin community suffer from the size
 because
 > ASF infra allows the package size under 500MB. So I'd like to reduce
 the
 > package size by reducing default packages.
 >
 > In case of rebuilding homepage, community proposed some criteria
 including
 > mailing list and # of question in stackoverflow. I think we can adapt
 same
 > criteria into release version of Zeppelin.
 >
 > To handle this kind of issue, I think consensus of community is the
 most
 > important factor. If someone wants to have an idea to deal with it,
 please
 > feel free to talk about it.
 >
 > Thanks,
 > Jongyoul Lee
 >
 > --
 > 이종열, Jongyoul Lee, 李宗烈
 > http://madeng.net
 >
 >
 >


 --
 이종열, Jongyoul Lee, 李宗烈
 http://madeng.net



>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Zeppelin can let users write Java code?

[Alexander Bezzubov]

It's not very well known, but Apache Zeppelin already has one Java
interpreter.
If you look at Apache Beam one [1]  - underneath [2] it has quite generic
facilities to interpreter java code, so you can use it for anything, not
just Java API of Beam.

But Felix's suggestion on using JShell in Java9 as soon as it's released
definitely sounds as a more mature solution.


 1. http://zeppelin.apache.org/docs/0.7.2/interpreter/beam.html
 2. https://github.com/apache/zeppelin/tree/master/beam

--
Alex

On Sat, Jun 24, 2017 at 9:26 PM, Felix Cheung 
wrote:

> There has been discussions on leveraging a Java REPL project to create
> such Java interpreter.
>
> Also Java 9 has JShell which seems to target a release in the next few
> months.
>
> --
> *From:* Yi Lyu 
> *Sent:* Saturday, June 24, 2017 9:17:32 AM
> *To:* users
> *Subject:* Zeppelin can let users write Java code?
>
> hi Team
> In our company, there are a lot of Java engineers, they are not good
> at Scala, may I ask, Zeppelin can let users write Java code, and then run
> them? Currently, I have tried Scala, Python, and can run it.
> and, Is there a plan to develop Java Interpreter ?
> --
>  Yi Lyu
>
>