Re: Equivalent of Jupyter %run

2020-01-11 Thread Dave Boyd 
Jeff:
   Thanks.

I tried the following:

%ipython
%run /Dave_Folder/ElasticUtils

I get the following error:

java.util.concurrent.RejectedExecutionException: Task 
io.grpc.internal.SerializingExecutor@7068c569 rejected from 
java.util.concurrent.ThreadPoolExecutor@789e11b[Terminated, pool size = 0, 
active threads = 0, queued tasks = 0, completed tasks = 45] at 
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
 at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) 
at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) 
at io.grpc.internal.SerializingExecutor.schedule(SerializingExecutor.java:93) 
at io.grpc.internal.SerializingExecutor.execute(SerializingExecutor.java:86) at 
io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closed(ClientCallImpl.java:588)
 at io.grpc.internal.FailingClientStream.start(FailingClientStream.java:54) at 
io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:273) at 
io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:398)
 at 
io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:673)
 at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:308) at 
io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:280) at 
io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:265) at 
io.grpc.stub.ClientCalls.asyncServerStreamingCall(ClientCalls.java:73) at 
org.apache.zeppelin.python.proto.IPythonGrpc$IPythonStub.execute(IPythonGrpc.java:240)
 at 
org.apache.zeppelin.python.IPythonClient.stream_execute(IPythonClient.java:89) 
at 
org.apache.zeppelin.python.IPythonInterpreter.interpret(IPythonInterpreter.java:350)
 at 
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:103)
 at 
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:632)
 at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at 
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748)

I am running 0.8.2 in a docker container.


On 1/10/2020 6:31 PM, Jeff Zhang wrote:
You can do it via ipython interpreter which support all the of jupyter magics

http://zeppelin.apache.org/docs/0.8.2/interpreter/python.html#ipython-support


Partridge, Lucas (GE Aviation) 
mailto:lucas.partri...@ge.com>> 于2020年1月10日周五 下午5:13写道：
I've hardly used Jupyter so can't comment on an equivalent for %run.

But for Zeppelin you can put your python files on the local file system of your 
Spark driver node, or more commonly in HDFS, and then use sc.addPyFile() [1] to 
make each file available in the SparkContext.  Then you can import your python 
packages as normal.  The slightly annoying thing is that if you change your 
code you'll need to restart your Spark application to pick up the changes as 
there's no reliable way to reimport the updated modules in a running 
application.  But you could put your importing of common files in a shared 
notebook so everyone can run it easily.

Once you're happy with your code and it's fairly stable then you can package it 
with a setup.py and install the packages on all the nodes of your cluster like 
any other python package. Then you can skip the sc.addPyFile() step.

DataBricks have a great facility for allowing users to upload their own Python 
packages/libraries. It would be great if Zeppelin provided this feature as well 
(although maybe they do now as I'm on an older version...).

Lucas.

[1] 
https://spark.apache.org/docs/latest/api/python/pyspark.html?highlight=addpyfile#pyspark.SparkContext.addPyFile

-Original Message-
From: Dave Boyd mailto:db...@incadencecorp.com>>
Sent: 09 January 2020 17:44
To: users@zeppelin.apache.org
Subject: EXT: Equivalent of Jupyter %run

I have googled this but don't see a solution.

We are working on a project where we want to have some common python functions 
shared between notes.

In Jupyter we would just do a %run.  Is there an equivelent in Zeppelin?
Is there a way to store files as .py files that zeppelin can find for import to 
work.

Looking to see how folks may have solved this need.

--
= mailto:db...@incadencecorp.com 
 David W. Boyd VP,  Data Solutions
10432 Balls Ford,

Re: [ANNOUNCE] Welcome Alex Ott as New Committer

2020-01-11 Thread Prabhjyot Singh 
Hello Alex,

Hearty Greetings and a very warm welcome to you.

On Sat, Jan 11, 2020, 2:16 PM Xun Liu  wrote:

> Hi, Alex Ott.
>
> Congratulations on becoming a committer of Apache Zeppelin !!
> :-)
>
> On Sat, Jan 11, 2020 at 4:38 PM Jongyoul Lee  wrote:
>
>> That's really great!!
>>
>> I appreciate his activities against Apache Zeppelin and I'm really happy
>> as
>> I have already known how difficult to contribute an opensource
>> continuously.
>>
>> Please keep helping make Apache Zeppelin healthy.
>>
>> Regards,
>> JL
>>
>> 2020년 1월 7일 (화) 오후 6:48, Jeff Zhang 님이 작성:
>>
>> > The Project Management Committee (PMC) for Apache Zeppelin
>> > has invited Alex Ott to become a committer and we are very pleased
>> > to announce that he has accepted.
>> >
>> > We greatly appreciate all of Alex's hard work and generous
>> contributions to
>> > the project. We look forward to continued involvement in the project.
>> >
>> > Congratulations & Welcome aboard Alex !
>> >
>> >
>> > --
>> > Best Regards
>> >
>> > Jeff Zhang
>> >
>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>

Re: [ANNOUNCE] Welcome Alex Ott as New Committer

2020-01-11 Thread Xun Liu 
Hi, Alex Ott.

Congratulations on becoming a committer of Apache Zeppelin !!
:-)

On Sat, Jan 11, 2020 at 4:38 PM Jongyoul Lee  wrote:

> That's really great!!
>
> I appreciate his activities against Apache Zeppelin and I'm really happy as
> I have already known how difficult to contribute an opensource
> continuously.
>
> Please keep helping make Apache Zeppelin healthy.
>
> Regards,
> JL
>
> 2020년 1월 7일 (화) 오후 6:48, Jeff Zhang 님이 작성:
>
> > The Project Management Committee (PMC) for Apache Zeppelin
> > has invited Alex Ott to become a committer and we are very pleased
> > to announce that he has accepted.
> >
> > We greatly appreciate all of Alex's hard work and generous contributions
> to
> > the project. We look forward to continued involvement in the project.
> >
> > Congratulations & Welcome aboard Alex !
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: [ANNOUNCE] Welcome Alex Ott as New Committer

2020-01-11 Thread Jongyoul Lee 
That's really great!!

I appreciate his activities against Apache Zeppelin and I'm really happy as
I have already known how difficult to contribute an opensource continuously.

Please keep helping make Apache Zeppelin healthy.

Regards,
JL

2020년 1월 7일 (화) 오후 6:48, Jeff Zhang 님이 작성:

> The Project Management Committee (PMC) for Apache Zeppelin
> has invited Alex Ott to become a committer and we are very pleased
> to announce that he has accepted.
>
> We greatly appreciate all of Alex's hard work and generous contributions to
> the project. We look forward to continued involvement in the project.
>
> Congratulations & Welcome aboard Alex !
>
>
> --
> Best Regards
>
> Jeff Zhang
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net