Ok,
This is what I can see from the logs
DEBUG [2019-11-01 16:40:06,806] ({pool-6-thread-2}
Interpreter.java[getProperty]:229) - key: zeppelin.python, value:
/home/mansop/anaconda2/bin/python
INFO [2019-11-01 16:40:06,806] ({pool-6-thread-2}
IPythonInterpreter.java[open]:135) - Python Exec:
/home/mansop/anaconda2/bin/python
INFO [2019-11-01 16:40:07,395] ({pool-6-thread-2}
IPythonInterpreter.java[checkIPythonPrerequisite]:195) - IPython prerequisite
is meet
DEBUG [2019-11-01 16:40:07,397] ({pool-6-thread-2}
Interpreter.java[getProperty]:229) - key: zeppelin.ipython.launch.timeout,
value: 30000
INFO [2019-11-01 16:40:07,399] ({pool-6-thread-2}
IPythonInterpreter.java[open]:146) - Launching IPython Kernel at port: 45806
INFO [2019-11-01 16:40:07,399] ({pool-6-thread-2}
IPythonInterpreter.java[open]:147) - Launching JVM Gateway at port: 35123
DEBUG [2019-11-01 16:40:07,401] ({pool-6-thread-2}
Interpreter.java[getProperty]:229) - key: zeppelin.ipython.grpc.message_size,
value: 33554432
DEBUG [2019-11-01 16:40:07,631] ({pool-6-thread-2}
Interpreter.java[getProperty]:229) - key: zeppelin.py4j.useAuth, value: true
INFO [2019-11-01 16:40:07,645] ({pool-6-thread-2}
IPythonInterpreter.java[setupIPythonEnv]:315) -
PYTHONPATH:/d1/hadoop/yarn/local/usercache/mansop/appcache/application_1572410115474_0033/container_e16_1572410115474_0033_01_000001/../interpreter/python/py4j-0.9.2/src:/usr/hdp/current/spark2-client/python/lib/py4j-0.10.7-src.zip:/usr/hdp/current/spark2-client/python/::/d1/hadoop/yarn/local/usercache/mansop/appcache/application_1572410115474_0033/container_e16_1572410115474_0033_01_000001/pyspark.zip:/d1/hadoop/yarn/local/usercache/mansop/appcache/application_1572410115474_0033/container_e16_1572410115474_0033_01_000001/py4j-0.10.7-src.zip
WARN [2019-11-01 16:40:08,037] ({Exec Default Executor}
IPythonInterpreter.java[onProcessFailed]:394) - Exception happens in Python
Process
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit
value: 1)
at
org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at
org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
at
org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
at java.lang.Thread.run(Thread.java:745)
INFO [2019-11-01 16:40:08,084] ({pool-6-thread-2}
IPythonInterpreter.java[launchIPythonKernel]:293) - Wait for IPython Kernel to
be started
INFO [2019-11-01 16:40:08,185] ({pool-6-thread-2}
IPythonInterpreter.java[launchIPythonKernel]:293) - Wait for IPython Kernel to
be started
...
INFO [2019-11-01 16:40:37,604] ({pool-6-thread-2}
IPythonInterpreter.java[launchIPythonKernel]:293) - Wait for IPython Kernel to
be started
INFO [2019-11-01 16:40:37,705] ({pool-6-thread-2}
IPythonInterpreter.java[launchIPythonKernel]:293) - Wait for IPython Kernel to
be started
WARN [2019-11-01 16:40:37,705] ({pool-6-thread-2}
PySparkInterpreter.java[open]:134) - Fail to open IPySparkInterpreter
java.lang.RuntimeException: Fail to open IPythonInterpreter
at
org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:157)
at
org.apache.zeppelin.spark.IPySparkInterpreter.open(IPySparkInterpreter.java:66)
at
org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:129)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:617)
at org.apache.zeppelin.scheduler.Job.run(Job.java:188)
at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Fail to launch IPython Kernel in 30 seconds
at
org.apache.zeppelin.python.IPythonInterpreter.launchIPythonKernel(IPythonInterpreter.java:297)
at
org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:154)
... 13 more
INFO [2019-11-01 16:40:37,706] ({pool-6-thread-2}
PySparkInterpreter.java[open]:140) - IPython is not available, use the native
PySparkInterpreter
DEBUG [2019-11-01 16:40:37,711] ({pool-6-thread-2}
Interpreter.java[getProperty]:222) - key: zeppelin.interpreter.localRepo,
value: /usr/hdp/current/zeppelin-server/local-repo/mansop
So I am confused because it says that ipython prerequisites are meet but still
fails to start iphython interpreter
So what is involved in the process to start ipython interpreter from zeppelin
point of view?
Manuel
From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: Wednesday, October 30, 2019 5:10 PM
To: users
Subject: Re: can't plot
It might be due other reason, you can set the interpreter log level to be DEBUG
to get more info.
Add following into log4j.properties
log4j.logger.org.apache.zeppelin.interpreter=DEBUG
Manuel Sopena Ballesteros
<manuel...@garvan.org.au<mailto:manuel...@garvan.org.au>> 于2019年10月30日周三
下午1:51写道：
Ok,
One more question, I am getting an error when I force ipython
%mansop.ipyspark
print("Hello world!")
java.io.IOException: Fail to launch IPython Kernel in 30 seconds at
org.apache.zeppelin.python.IPythonInterpreter.launchIPythonKernel(IPythonInterpreter.java:297)
at
org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:154)
at
org.apache.zeppelin.spark.IPySparkInterpreter.open(IPySparkInterpreter.java:66)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:617)
at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
https://zeppelin.apache.org/docs/0.8.0/interpreter/python.html#ipython-support
both grpcio and jupyter are installed
any idea?
Manuel
From: Jeff Zhang [mailto:zjf...@gmail.com<mailto:zjf...@gmail.com>]
Sent: Wednesday, October 30, 2019 12:53 PM
To: users
Subject: Re: can't plot
Based on the error message, you are still using python instead of ipython. It
is hard to tell what's wrong.
One suggestion is to try 0.8.2 which is the latest release.
Manuel Sopena Ballesteros
<manuel...@garvan.org.au<mailto:manuel...@garvan.org.au>> 于2019年10月30日周三
上午9:47写道：
Didn’t like %matplotlib inline
Traceback (most recent call last):
File
"/d1/hadoop/yarn/local/usercache/mansop/appcache/application_1570749574365_0083/container_e15_1570749574365_0083_01_000001/tmp/zeppelin_pyspark-2736590645623350055.py",
line 364, in <module>
code = compile('\n'.join(stmts), '<stdin>', 'exec', ast.PyCF_ONLY_AST, 1)
File "<stdin>", line 1
%matplotlib inline
^
SyntaxError: invalid syntax
Manuel
From: Jeff Zhang [mailto:zjf...@gmail.com<mailto:zjf...@gmail.com>]
Sent: Wednesday, October 30, 2019 12:43 PM
To: users
Subject: Re: can't plot
Try this
%pyspark
%matplotlib inline
import matplotlib.pyplot as plt
plt.figure()
plt.plot([1, 2, 3])
Manuel Sopena Ballesteros
<manuel...@garvan.org.au<mailto:manuel...@garvan.org.au>> 于2019年10月30日周三
上午9:39写道：
Another example:
%pyspark
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
z.show(plt)
plt.close()
<module 'matplotlib.pyplot' from
'/home/mansop/anaconda2/lib/python2.7/site-packages/matplotlib/pyplot.pyc'>
According to documentation
https://zeppelin.apache.org/docs/0.8.0/interpreter/python.html#matplotlib-integration
Am I right assuming that I can use z.show in %pyspark?
Thank you
Manuel
From: Manuel Sopena Ballesteros
[mailto:manuel...@garvan.org.au<mailto:manuel...@garvan.org.au>]
Sent: Wednesday, October 30, 2019 12:12 PM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: can't plot
Dear Zeppelin user community,
I am running Zeppelin 0.8.0 and I am not able to print a plot using pyspark
interpreter:
This is my notebook:
%pyspark
import matplotlib.pyplot as plt
plt.figure()
plt.plot([1, 2, 3])
And this is the output:
[<matplotlib.lines.Line2D object at 0x7fd34321c450>]
Any idea?
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/copyright information. If you are not the intended
recipient, you should not read, use, disclose, copy or distribute this
communication. If you have received this message in error please notify us at
once by return email and then delete both messages. We accept no liability for
the distribution of viruses or similar in electronic communications. This
notice should not be removed.
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/copyright information. If you are not the intended
recipient, you should not read, use, disclose, copy or distribute this
communication. If you have received this message in error please notify us at
once by return email and then delete both messages. We accept no liability for
the distribution of viruses or similar in electronic communications. This
notice should not be removed.
--
Best Regards
Jeff Zhang
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/copyright information. If you are not the intended
recipient, you should not read, use, disclose, copy or distribute this
communication. If you have received this message in error please notify us at
once by return email and then delete both messages. We accept no liability for
the distribution of viruses or similar in electronic communications. This
notice should not be removed.
--
Best Regards
Jeff Zhang
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/copyright information. If you are not the intended
recipient, you should not read, use, disclose, copy or distribute this
communication. If you have received this message in error please notify us at
once by return email and then delete both messages. We accept no liability for
the distribution of viruses or similar in electronic communications. This
notice should not be removed.
--
Best Regards
Jeff Zhang
NOTICE
Please consider the environment before printing this email. This message and
any attachments are intended for the addressee named and may contain legally
privileged/confidential/copyright information. If you are not the intended
recipient, you should not read, use, disclose, copy or distribute this
communication. If you have received this message in error please notify us at
once by return email and then delete both messages. We accept no liability for
the distribution of viruses or similar in electronic communications. This
notice should not be removed.
