Re: Multiuser support of Zeppelin.

[Jeff Zhang]

Hi guys,

I create an umbrella ticket for multiple user support for zeppelin as I can
see more and more requirement of this from community, feel free to add sub
task that I miss.

https://issues.apache.org/jira/browse/ZEPPELIN-1337



On Tue, Aug 9, 2016 at 9:20 AM, Alexander Bezzubov  wrote:

> Hi Egor,
>
> let me share two approaches that we used to archive Apache Zeppelin
> working in muliteanant environment with Apache Spark:
>
> - run a separate container with Zeppelin per-user, from the small cluster
> of Docker machines (so a single machine runs just 2-3 containers \w
> SparkContexts)
> This works well, supports Spark standalone cluster but requires central
> external auth and small "resource manager" to allocate the containers to
> the Docker cluster + a reverse proxy as a single point of entry for the
> user.
> We have implemented all this in one binary under openaource project called
> Z-Manager Muliteanancy, you can get more details here [1]. It is beta and
> we didnt have capacity to support it recently.
>
> - run single Zeppelin with Auth enabled + Livy interpreter + Spark in YARN
> server mode
>
> This is more generic solution, but it requires particular cluster
> configuration. Here YARN is used as a resource manager to handle multiple
> Spark contexts/drivers processes on the same cluster as tasks themselves.
> Afaik Mesos cluster might be used instead by I do not have first hand
> experience about it. You can read more about it here [2].
>
> Hope this helps!
>
> --
>
> Alex
>
> 1. https://github.com/NFLabs/z-manager/blob/master/multitenancy/README.md
> 2. http://zeppelin.apache.org/docs/0.6.0/interpreter/livy.html
>
> On Sat, Aug 6, 2016, 06:12 vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
>> One zeppelin per user in mesos container on datanode type server is fine
>> for me. An Ansible script configure each instance with user specifities and
>> launch it in Marathon. A service discovery (basic shell script) update an
>> apache server with basic auth and route each user to his instance. Mesos
>> also run a SMACK stack on which zeppelin rely.
>>
>> Le 5 août 2016 11:01 PM, "Egor Pahomov"  a
>> écrit :
>>
>> I need to build a chart for 10 days for all countries(200) for several
>> products by some dimensions. I would need at least 4-6 gb per zeppelin for
>> it.
>>
>> 2016-08-05 12:31 GMT-07:00 Mohit Jaggi :
>>
>>> put your big results somewhere else not in Z’s memory?
>>>
>>> On Aug 5, 2016, at 12:26 PM, Egor Pahomov 
>>> wrote:
>>>
>>> - Use spark driver in “cluster mode” where driver runs on a worker
 instead of the node running Z
>>>
>>>
>>> Even without driver Z is heavy process. You need a lot of RAM to keep
>>> big results from job. And most of all - zeppelin 0.5.6 does not support
>>> cluster mode and I'm not ready to move to 0.6.
>>>
>>> 2016-08-05 12:03 GMT-07:00 Mohit Jaggi :
>>>
 Egor,
 Running a scale out system like Spark with multiple users is always
 tricky. Operating systems are designed to let multiple users share a single
 machine. But for “big data” a single user requires the use of several
 machines which is the exact opposite. Having said that I would suggest the
 following:

 - Use spark driver in “cluster mode” where driver runs on a worker
 instead of the node running Z
 - Set appropriate limits/sizes in spark master configuration
 - run separate instances of Z per user, but then you will have a tough
 time collaborating and sharing notebooks…maybe they can be stored in a
 shared space and all Z instances can read them but I am afraid that shared
 access might clobber the files. Z developers can tell us if that is true

 Another alternative is virtualization using containers but I think that
 will not be easy either.

 Mohit
 Founder,
 Data Orchard LLC
 www.dataorchardllc.com


 On Aug 5, 2016, at 11:45 AM, Egor Pahomov 
 wrote:

 Hi,  I'd like to discuss best practices for using zeppelin in the
 multi-user environment. There are several naive approaches, I've tried for
 at least couple month each and not a single one worked:

 *All users on one zeppelin.*

- One spark context - people really break sc and when they are all
in the same boat a single person can stop many from working.
- No resource management support. One person can allocate all
resources for a long time
- The number of notebooks is enormous - it's hard to find anything
in it.
- No security separation - everyone sees everything. I do not care
about security, but I care about fool prove. And people can accidently
delete notebooks of each other.

 *Every user has his own Zeppelin on one machine*

- Every zeppelin instance eats memory 

PrintWriter not found

[Daniel Valdivia]

Hi,

I just upgraded my spark and zeppelin together to their respective latest 
versions (2.0 and 0.6.1) however my existing notebooks are running into problem 
that I'm not sure how to debug, if I manually append the full namespace this 
works



for example;

class Example {
   def test() {
   new PrintWriter("/tmp/test.lol") { write("test"); close }
   }
}

yields 

:15: error: not found: type PrintWriter
new PrintWriter("/tmp/test.lol") { write("test"); close }
^
:15: error: not found: value write
new PrintWriter("/tmp/test.lol") { write("test"); close }
^
:15: error: not found: value close
new PrintWriter("/tmp/test.lol") { write("test"); close }
^

This used to work fine on the previous release of zeppeling

any advice?

Thanks

Re: Issue loading dependency with SPARK_SUBMIT_OPTIONS w/ 0.6.1

[Mina Lee]

I also could reproduce it with Spark 2.0.0, but not with Spark 1.6.1.
If you want to use Zeppelin with Spark 2.0, one alternative you can try is
using [1] "dependencies" in GUI interpreter menu.

[1] http://zeppelin.apache.org/docs/0.6.1/manual/dependencymanagement.html

On Wed, Aug 17, 2016 at 1:46 AM Jeff Zhang  wrote:

> I can reproduce it in 0.6.1 & master branch, please file a ticket for
> that.
>
> On Wed, Aug 17, 2016 at 4:09 AM, Michael Sells  wrote:
>
>> Testing out 0.6.1 with Spark 2.0 and discovered the way we load
>> dependencies doesn't seem to be working with the new update.
>>
>> We pass new dependencies in via a SPARK_SUBMIT_OPTIONS environment
>> variable pass the following flags:
>> --packages com.databricks:spark-avro_2.11:3.0.0
>>
>> Now when I try to import it with:
>> import com.databricks.spark.avro._
>>
>> I get:
>> :25: error: object databricks is not a member of package com
>> import com.databricks.spark.avro._
>>
>> I checked the logs are there is no error retrieving the package. So it
>> seems to be something with the classpath.
>>
>> This works in 0.6.0. Any idea if something changed or if we're doing
>> something wrong? I tried this with a few internal packages as well and it
>> doesn't work with those either.
>>
>> Thanks,
>> Mike
>>
>>
>>
>>
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Issue loading dependency with SPARK_SUBMIT_OPTIONS w/ 0.6.1

[Jeff Zhang]

I can reproduce it in 0.6.1 & master branch, please file a ticket for that.

On Wed, Aug 17, 2016 at 4:09 AM, Michael Sells  wrote:

> Testing out 0.6.1 with Spark 2.0 and discovered the way we load
> dependencies doesn't seem to be working with the new update.
>
> We pass new dependencies in via a SPARK_SUBMIT_OPTIONS environment
> variable pass the following flags:
> --packages com.databricks:spark-avro_2.11:3.0.0
>
> Now when I try to import it with:
> import com.databricks.spark.avro._
>
> I get:
> :25: error: object databricks is not a member of package com
> import com.databricks.spark.avro._
>
> I checked the logs are there is no error retrieving the package. So it
> seems to be something with the classpath.
>
> This works in 0.6.0. Any idea if something changed or if we're doing
> something wrong? I tried this with a few internal packages as well and it
> doesn't work with those either.
>
> Thanks,
> Mike
>
>
>
>
>
>


-- 
Best Regards

Jeff Zhang

Re: Possible to use Zeppelin Shell Interpreter to start a spark-submit job?

[Jeff Zhang]

I tried the latest zeppelin, it can tell me the exact exception when I
didn't use spark-submit correctly.  What zeppelin version do you use ?
Maybe you can try the latest version

[image: Inline image 1]

On Wed, Aug 17, 2016 at 2:30 AM, Peter DeNicola <
peter.denicola.f...@statefarm.com> wrote:

> The spark shell interpreter logs don’t seem to give me much more info. It
> says it cannot run the job and the exception given is
>
>
>
> org.apache.commons.exec.ExecuteException: Process exited with an error: 1
> (Exit value: 1)
>
> at org.apache.commons.exec.DefaultExecutor.executeInternal(
> DefaultExecutor.java:404)
>
> at org.apache.commons.exec.DefaultExecutor.execute(
> DefaultExecutor.java:166)
>
> at org.apache.commons.exec.DefaultExecutor.execute(
> DefaultExecutor.java:153)
>
> at org.apache.zeppelin.shell.ShellInterpreter.interpret(
> ShellInterpreter.java:74)
>
> at org.apache.zeppelin.interpreter.ClassloaderInterpreter.
> interpret(ClassloaderInterpreter.java:57)
>
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(
> LazyOpenInterpreter.java:93)
>
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:300)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:169)
>
> at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(
> FIFOScheduler.java:134)
>
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> *From: *Jeff Zhang 
> *Reply-To: *"users@zeppelin.apache.org" 
> *Date: *Tuesday, August 16, 2016 at 9:13 AM
> *To: *"users@zeppelin.apache.org" 
> *Subject: *Re: Possible to use Zeppelin Shell Interpreter to start a
> spark-submit job?
>
>
>
> Please check the shell interpreter log.
>
>
>
> On Tue, Aug 16, 2016 at 10:11 PM, Peter DeNicola  statefarm.com> wrote:
>
> I’m trying to run a spark-submit job using a jar in HDFS from Zeppelin,
> but all I’m getting back is Process exited with an error: 1 (Exit value:
> 1)
>
>
>
> Which I’m not sure what this means. If anyone knows anything about this—or
> if it is even possible—I could use some help.
>
>
>
>
>
> --
>
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Issue loading dependency with SPARK_SUBMIT_OPTIONS w/ 0.6.1

[Michael Sells]

Testing out 0.6.1 with Spark 2.0 and discovered the way we load
dependencies doesn't seem to be working with the new update.

We pass new dependencies in via a SPARK_SUBMIT_OPTIONS environment variable
pass the following flags:
--packages com.databricks:spark-avro_2.11:3.0.0

Now when I try to import it with:
import com.databricks.spark.avro._

I get:
:25: error: object databricks is not a member of package com
import com.databricks.spark.avro._

I checked the logs are there is no error retrieving the package. So it
seems to be something with the classpath.

This works in 0.6.0. Any idea if something changed or if we're doing
something wrong? I tried this with a few internal packages as well and it
doesn't work with those either.

Thanks,
Mike

Re: Possible to use Zeppelin Shell Interpreter to start a spark-submit job?

[Peter DeNicola]

The spark shell interpreter logs don’t seem to give me much more info. It says 
it cannot run the job and the exception given is

org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit 
value: 1)
at 
org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at 
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at 
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
at 
org.apache.zeppelin.shell.ShellInterpreter.interpret(ShellInterpreter.java:74)
at 
org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
at 
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at 
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
at org.apache.zeppelin.scheduler.Job.run(Job.java:169)
at 
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

From: Jeff Zhang 
Reply-To: "users@zeppelin.apache.org" 
Date: Tuesday, August 16, 2016 at 9:13 AM
To: "users@zeppelin.apache.org" 
Subject: Re: Possible to use Zeppelin Shell Interpreter to start a spark-submit 
job?

Please check the shell interpreter log.

On Tue, Aug 16, 2016 at 10:11 PM, Peter DeNicola 
> 
wrote:
I’m trying to run a spark-submit job using a jar in HDFS from Zeppelin, but all 
I’m getting back is Process exited with an error: 1 (Exit value: 1)

Which I’m not sure what this means. If anyone knows anything about this—or if 
it is even possible—I could use some help.



--
Best Regards

Jeff Zhang

Re: [ANNOUNCE] Apache Zeppelin 0.6.1 released

[Mohit Jaggi]

awesome!

Mohit Jaggi
Founder,
Data Orchard LLC
www.dataorchardllc.com




> On Aug 16, 2016, at 6:56 AM, Ahyoung Ryu  wrote:
> 
> Congratulations, the Apache Zeppelin team & all community members! 
> 
> >
> Congratulations !!!
> 
> Great turn around with required spark 2.0 support and fix for SparkR with 
> Spark2.0.
> 
> Regards,
> Sourav
> 
> On Tue, Aug 16, 2016 at 2:02 AM, Hyung Sung Shim  > wrote:
> Congratulations for 0.6.1 release!
> Thank you for everyone especially mina.
> 
> 2016-08-16 16:57 GMT+09:00 Mina Lee >:
> The Apache Zeppelin community is pleased to announce the availability of
> the 0.6.1 release.
> 
> Zeppelin is a collaborative data analytics and visualization tool for
> distributed, general-purpose data processing system such as Apache Spark,
> Apache Flink, etc.
> 
> The community put significant effort into improving Apache Zeppelin since
> the last release, focusing on supporting Scala 2.11 and Spark 2.0.
> 22 contributors provided 80+ patches for new features, improvements
> and bug fixes. More than 60+ issues have been resolved.
> 
> We encourage download the latest release from
> http://zeppelin.apache.org/download.html 
> 
> 
> Release note is available at
> http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html 
> 
> 
> We welcome your help and feedback. For more information on the project and
> how to get involved, visit our website at http://zeppelin.apache.org/ 
> 
> 
> Thanks to all users and contributors who have helped to improve Apache
> Zeppelin.
> 
> Regards,
> The Apache Zeppelin community
> 
> 

Possible to use Zeppelin Shell Interpreter to start a spark-submit job?

[Peter DeNicola]

I’m trying to run a spark-submit job using a jar in HDFS from Zeppelin, but all 
I’m getting back is Process exited with an error: 1 (Exit value: 1)

Which I’m not sure what this means. If anyone knows anything about this—or if 
it is even possible—I could use some help.

Re: [ANNOUNCE] Apache Zeppelin 0.6.1 released

[Sourav Mazumder]

Congratulations !!!

Great turn around with required spark 2.0 support and fix for SparkR with
Spark2.0.

Regards,
Sourav

On Tue, Aug 16, 2016 at 2:02 AM, Hyung Sung Shim  wrote:

> Congratulations for 0.6.1 release!
> Thank you for everyone especially mina.
>
> 2016-08-16 16:57 GMT+09:00 Mina Lee :
>
>> The Apache Zeppelin community is pleased to announce the availability of
>> the 0.6.1 release.
>>
>> Zeppelin is a collaborative data analytics and visualization tool for
>> distributed, general-purpose data processing system such as Apache Spark,
>> Apache Flink, etc.
>>
>> The community put significant effort into improving Apache Zeppelin since
>> the last release, focusing on supporting Scala 2.11 and Spark 2.0.
>> 22 contributors provided 80+ patches for new features, improvements
>> and bug fixes. More than 60+ issues have been resolved.
>>
>> We encourage download the latest release from
>> http://zeppelin.apache.org/download.html
>>
>> Release note is available at
>> http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html
>>
>> We welcome your help and feedback. For more information on the project and
>> how to get involved, visit our website at http://zeppelin.apache.org/
>>
>> Thanks to all users and contributors who have helped to improve Apache
>> Zeppelin.
>>
>> Regards,
>> The Apache Zeppelin community
>>
>
>

Re: z.show() not working

[Jose Rivera-Rubio]

No errors in the interpreter log stored in ./logs but still no HTML table
output after doing z.show() Any other locations where I should check for
logs?

Thanks!

PS: I'm using commit 37696ea8bd6a5c3b910f7d24141530f91f80391c of the master
branch

*Jose Rivera*
p.s. We're Hiring !

*Data Engineer Lead  **Intern Avenue *
Watch Intern Avenue on: BBC Dragons’ Den

e: jose.riv...@internavenue.com  |
www.internavenue.com | Great Interns Online

On 12 August 2016 at 02:49, Jeff Zhang  wrote:

> I tried 0.6.1, it works for me. Could you check the interpreter log and
> yarn app log ?
>
> On Fri, Aug 12, 2016 at 7:20 AM, Jose Rivera-Rubio <
> jose.riv...@internavenue.com> wrote:
>
>> I built from source after pulling the latest master. I also checked out
>> the RC1 tag of v0.6.1, compiled and I could see the same behaviour: no
>> output from z.show().
>>
>> Thanks
>>
>> *Jose Rivera*
>> p.s. We're Hiring !
>>
>> *Data Engineer Lead  **Intern Avenue *
>> Watch Intern Avenue on: BBC Dragons’ Den
>> 
>> e: jose.riv...@internavenue.com  |
>> www.internavenue.com | Great Interns Online
>>
>> On 11 August 2016 at 14:05, Jeff Zhang  wrote:
>>
>>> Which version of zeppelin are you using ?
>>>
>>> On Thu, Aug 11, 2016 at 8:36 PM, Jose Rivera-Rubio <
>>> jose.riv...@internavenue.com> wrote:
>>>
 Hi,

 I saw this  JIRA
 issue but I'm facing the problem with Spark 1.6.0 and 1.6.1

 When doing z.show(dataframe) I just get the REPL output of the columns [id:
 bigint, first_name: string, last_name: string, email: string]

 But not the HTML displayed data. If I do dataframe.show() I can see the
 data so it's not an issue of the data or the Scala code.

 Any hints? Thanks!

 *Jose Rivera*
 p.s. We're Hiring !

 *Data Engineer Lead  **Intern Avenue *
 Watch Intern Avenue on: BBC Dragons’ Den
 
 e: jose.riv...@internavenue.com  |
 www.internavenue.com | Great Interns Online

>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Can I run different versions of spark interpreter in one zeppelin build ?

[Jeff Zhang]

Sorry, my bad, export SPARK_HOME doesn't work. I should try this earlier,
otherwise it can be merged into 0.6.1

Created ticket https://issues.apache.org/jira/browse/ZEPPELIN-1334



On Thu, Aug 4, 2016 at 12:29 PM, Jeff Zhang  wrote:

> SparkSession is already available in spark2 support.
>
> On Thu, Aug 4, 2016 at 12:16 PM, Vinay Shukla 
> wrote:
>
>> There are a lot of new APIs with Spark 2 and more importantly the new
>> entry point is SparkSession (available as spark variable).
>>
>> I think Z built against Spark 2 should leverage spark variable by default
>> instead of using SC.
>>
>> Please ignore this if the current Spark2 support already includes this
>> feature.
>>
>> Thanks,
>> Vinay
>>
>> On Wed, Aug 3, 2016 at 9:14 PM, Jeff Zhang  wrote:
>>
>>> Thanks moon, if that works, it seem not necessary to have different
>>> profiles for different spark version. We can always build it with the
>>> latest spark version.
>>>
>>> On Thu, Aug 4, 2016 at 11:26 AM, moon soo Lee  wrote:
>>>
 Hi,

 Could you try remove 'SPARK_HOME' from conf/zeppelin-env.sh and add
 'SPARK_HOME' property (in different version of spark directory) in each
 individual spark interpreter setting on GUI? This should work.

 Best,
 moon

 On Wed, Aug 3, 2016 at 6:42 PM Jeff Zhang  wrote:

> I build zeppelin with spark-2.0 profile enabled, and it seems I can
> also run spark 1.6 interpreter. But I am not sure whether it is officially
> supported to run different versions of spark interpreter in one zeppelin
> build ? My guess maybe it is not, otherwise we don't need profiles for
> different spark version. If it is not, then I think support multiple
> versions of spark interpreter in one zeppelin build might be useful,
> otherwise user need to setup multiple zeppelin servers for different spark
> versions.
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

[ANNOUNCE] Apache Zeppelin 0.6.1 released

[Hyung Sung Shim]

Congratulations for 0.6.1 release!
Thank you for everyone especially mina.

2016-08-16 16:57 GMT+09:00 Mina Lee >:

> The Apache Zeppelin community is pleased to announce the availability of
> the 0.6.1 release.
>
> Zeppelin is a collaborative data analytics and visualization tool for
> distributed, general-purpose data processing system such as Apache Spark,
> Apache Flink, etc.
>
> The community put significant effort into improving Apache Zeppelin since
> the last release, focusing on supporting Scala 2.11 and Spark 2.0.
> 22 contributors provided 80+ patches for new features, improvements
> and bug fixes. More than 60+ issues have been resolved.
>
> We encourage download the latest release from
> http://zeppelin.apache.org/download.html
>
> Release note is available at
> http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html
>
> We welcome your help and feedback. For more information on the project and
> how to get involved, visit our website at http://zeppelin.apache.org/
>
> Thanks to all users and contributors who have helped to improve Apache
> Zeppelin.
>
> Regards,
> The Apache Zeppelin community
>

Re: [ANNOUNCE] Apache Zeppelin 0.6.1 released

[tog]

Congrats to the team!

On Tue, 16 Aug 2016 at 10:12 Alexander Bezzubov  wrote:

> Congratulations, everyone!
>
> And thank you Mina for putting it all together!
>
> https://twitter.com/ApacheZeppelin/status/765459508315303936
>
> --
> Alex
>
> On Tue, Aug 16, 2016 at 4:57 PM, Mina Lee  wrote:
>
>> The Apache Zeppelin community is pleased to announce the availability of
>> the 0.6.1 release.
>>
>> Zeppelin is a collaborative data analytics and visualization tool for
>> distributed, general-purpose data processing system such as Apache Spark,
>> Apache Flink, etc.
>>
>> The community put significant effort into improving Apache Zeppelin since
>> the last release, focusing on supporting Scala 2.11 and Spark 2.0.
>> 22 contributors provided 80+ patches for new features, improvements
>> and bug fixes. More than 60+ issues have been resolved.
>>
>> We encourage download the latest release from
>> http://zeppelin.apache.org/download.html
>>
>> Release note is available at
>> http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html
>>
>> We welcome your help and feedback. For more information on the project and
>> how to get involved, visit our website at http://zeppelin.apache.org/
>>
>> Thanks to all users and contributors who have helped to improve Apache
>> Zeppelin.
>>
>> Regards,
>> The Apache Zeppelin community
>>
>

Re: [ANNOUNCE] Apache Zeppelin 0.6.1 released

[Alexander Bezzubov]

Congratulations, everyone!

And thank you Mina for putting it all together!

https://twitter.com/ApacheZeppelin/status/765459508315303936

--
Alex

On Tue, Aug 16, 2016 at 4:57 PM, Mina Lee  wrote:

> The Apache Zeppelin community is pleased to announce the availability of
> the 0.6.1 release.
>
> Zeppelin is a collaborative data analytics and visualization tool for
> distributed, general-purpose data processing system such as Apache Spark,
> Apache Flink, etc.
>
> The community put significant effort into improving Apache Zeppelin since
> the last release, focusing on supporting Scala 2.11 and Spark 2.0.
> 22 contributors provided 80+ patches for new features, improvements
> and bug fixes. More than 60+ issues have been resolved.
>
> We encourage download the latest release from
> http://zeppelin.apache.org/download.html
>
> Release note is available at
> http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html
>
> We welcome your help and feedback. For more information on the project and
> how to get involved, visit our website at http://zeppelin.apache.org/
>
> Thanks to all users and contributors who have helped to improve Apache
> Zeppelin.
>
> Regards,
> The Apache Zeppelin community
>

RE: How to restore the notebook from S3?

[Jonathan Gough]

Is it supposed to be ZEPPELIN_NOTEBOOK_USER instead of 
ZEPPELIN_NOTEBOOK_S3_USER?

That’s what I have in my config:

[
  {
"configurations": [
  {
"classification": "export",
"properties": {
  "ZEPPELIN_NOTEBOOK_S3_BUCKET": "mybucket",
  "ZEPPELIN_NOTEBOOK_STORAGE": 
"org.apache.zeppelin.notebook.repo.S3NotebookRepo",
  "ZEPPELIN_NOTEBOOK_USER": "user"
}
  }
],
"classification": "zeppelin-env",
"properties": {

}
]

Jonathan

From: Shuya Tsukamoto [mailto:shuya.tsukam...@gmail.com]
Sent: 16 August 2016 08:08
To: users@zeppelin.apache.org
Subject: How to restore the notebook from S3?

Hi!

I am using zeppelin 0.5.6 with Amazon EMR (emr-4.7.2).

I was tried section "Notebook Storage in S3".
I use the configuration such as the following because my case is EMR.

https://blogs.aws.amazon.com/bigdata/post/Tx3K805CZ8WFBRP/Analyze-Realtime-Data-from-Amazon-Kinesis-Streams-Using-Zeppelin-and-Spark-Strea

```
[
   {
  "Classification": "zeppelin-env",
  "Properties":{
  },
  "Configurations":[
 {
"Classification": "export",
"Properties":{
   "AWS_ACCESS_KEY_ID": "mykeyid",
   "AWS_SECRET_ACCESS_KEY": "myaccesskey",
   "ZEPPELIN_NOTEBOOK_S3_USER": "hadoop",
   "ZEPPELIN_NOTEBOOK_S3_BUCKET": "mybucket",
   "ZEPPELIN_NOTEBOOK_STORAGE": 
"org.apache.zeppelin.notebook.repo.S3NotebookRepo"
},
"Configurations": []
 }
  ]
   }
]
```


I was able to save the notebook to S3. But I couldn't restore the notebook from 
S3.
I want to restore the notebook automatically from S3 when I re-create the EMR 
cluster.

Could you please help me?
Thank you.

Shuya Tsukamoto

[ANNOUNCE] Apache Zeppelin 0.6.1 released

[Mina Lee]

The Apache Zeppelin community is pleased to announce the availability of
the 0.6.1 release.

Zeppelin is a collaborative data analytics and visualization tool for
distributed, general-purpose data processing system such as Apache Spark,
Apache Flink, etc.

The community put significant effort into improving Apache Zeppelin since
the last release, focusing on supporting Scala 2.11 and Spark 2.0.
22 contributors provided 80+ patches for new features, improvements
and bug fixes. More than 60+ issues have been resolved.

We encourage download the latest release from
http://zeppelin.apache.org/download.html

Release note is available at
http://zeppelin.apache.org/releases/zeppelin-release-0.6.1.html

We welcome your help and feedback. For more information on the project and
how to get involved, visit our website at http://zeppelin.apache.org/

Thanks to all users and contributors who have helped to improve Apache
Zeppelin.

Regards,
The Apache Zeppelin community

Any reason why PostgreSQL is not merged into jdbc interpreter

[Jeff Zhang]

Is there anything specific for postgreSQL so that zeppelin didn't merge it
into the general jdbc interpreter ?


-- 
Best Regards

Jeff Zhang

Re: How to restore the notebook from S3?

[Kevin Kim (Sangwoo)]

Hi,


In my case, it just restored notebooks, when I re-created EMR clusters.

You may look into log files to find out what's going wrong..

- Kevin


2016년 8월 16일 (화) 오후 4:07, Shuya Tsukamoto 님이 작성:

> Hi!
>
> I am using zeppelin 0.5.6 with Amazon EMR (emr-4.7.2).
>
> I was tried section "Notebook Storage in S3".
> I use the configuration such as the following because my case is EMR.
>
>
> https://blogs.aws.amazon.com/bigdata/post/Tx3K805CZ8WFBRP/Analyze-Realtime-Data-from-Amazon-Kinesis-Streams-Using-Zeppelin-and-Spark-Strea
>
> ```
> [
>{
>   "Classification": "zeppelin-env",
>   "Properties":{
>   },
>   "Configurations":[
>  {
> "Classification": "export",
> "Properties":{
>"AWS_ACCESS_KEY_ID": "mykeyid",
>"AWS_SECRET_ACCESS_KEY": "myaccesskey",
>"ZEPPELIN_NOTEBOOK_S3_USER": "hadoop",
>"ZEPPELIN_NOTEBOOK_S3_BUCKET": "mybucket",
>"ZEPPELIN_NOTEBOOK_STORAGE":
> "org.apache.zeppelin.notebook.repo.S3NotebookRepo"
> },
> "Configurations": []
>  }
>   ]
>}
> ]
> ```
>
>
> I was able to save the notebook to S3. But I couldn't restore the notebook
> from S3.
> I want to restore the notebook automatically from S3 when I re-create the
> EMR cluster.
>
> Could you please help me?
> Thank you.
>
> Shuya Tsukamoto
>

How to restore the notebook from S3?

[Shuya Tsukamoto]

Hi!

I am using zeppelin 0.5.6 with Amazon EMR (emr-4.7.2).

I was tried section "Notebook Storage in S3".
I use the configuration such as the following because my case is EMR.

https://blogs.aws.amazon.com/bigdata/post/Tx3K805CZ8WFBRP/
Analyze-Realtime-Data-from-Amazon-Kinesis-Streams-Using-
Zeppelin-and-Spark-Strea

```
[
   {
  "Classification": "zeppelin-env",
  "Properties":{
  },
  "Configurations":[
 {
"Classification": "export",
"Properties":{
   "AWS_ACCESS_KEY_ID": "mykeyid",
   "AWS_SECRET_ACCESS_KEY": "myaccesskey",
   "ZEPPELIN_NOTEBOOK_S3_USER": "hadoop",
   "ZEPPELIN_NOTEBOOK_S3_BUCKET": "mybucket",
   "ZEPPELIN_NOTEBOOK_STORAGE": "org.apache.zeppelin.notebook.
repo.S3NotebookRepo"
},
"Configurations": []
 }
  ]
   }
]
```


I was able to save the notebook to S3. But I couldn't restore the notebook
from S3.
I want to restore the notebook automatically from S3 when I re-create the
EMR cluster.

Could you please help me?
Thank you.

Shuya Tsukamoto