Combinations: SPARK_HOME set in 'O'->point to external spark , no '-' ->head to internal spark lib
spark interpreter setting 〇 〇 - - 〇 - - zeppelin -env.sh - - - - 〇 〇 〇 environment parameters - 〇 〇 - 〇 〇 - Work Fine:〇 NG:× 〇 〇 〇 〇 × × × The consequences are zeppelin-env.sh has a top priority. And if you want to use external Spark on Mac, you should not set SPARK_HOME in zeppelin-env.sh. Set it as environment parameters or in spark interpreter. Cheers, Ono Keiji 2018-03-06 12:23 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: > > If you set SPARK_HOME in spark interpreter setting, then please remove > SPARK_HOME in spark-env.sh, otherwise you do will hit weird issue. > > > > > 小野圭二 <onoke...@gmail.com>于2018年3月6日周二 上午10:56写道： > >> Hi Jeff, >> >> I did set SPARK_HOME to use external spark at first. And that was fine >> work on CentOS, but not on Mac. >> Do you mean try it again? >> >> 2018-03-06 11:12 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >> >>> >>> Seems like it is not good chemistry between Spark and Zeppelin on >>> Mac. From the reason that the consequence of removing the environment >>> parameters, Zeppelin headed to own spark lib. >>> >>> Could you specify SPARK_HOME in your spark interpreter to use external >>> spark instead of zeppelin's own spark lib ? >>> >>> >>> >>> >>> 小野圭二 <onoke...@gmail.com>于2018年3月6日周二 上午10:06写道： >>> >>>> I wonder why, but it seems work fine, and would like to share how to >>>> fix it. >>>> >>>> First of all, this issue happened just on MAC. >>>> [THE ENV WAS] >>>> macOS High Sierra v 10.13.3 >>>> cpu core i5 >>>> mem 8GB >>>> Zeppelin 0.8.0 snapshot >>>> Spark 2.2.0 >>>> [ISSUE] >>>> see this previous mails >>>> [SOLUTION] >>>> ref. http://apache-zeppelin-users-incubating-mailing-list. >>>> 75479.x6.nabble.com/Zeppelin-0-6-2-Error-Interpreter- >>>> process-failed-td4743.html >>>> According to the above ref, i removed 'SPARK_HOME', >>>> 'SPARK_SUBMIT_OPTIONS' from my environment parameters and >>>> conf/zeppelin-env.sh(set it as same like CentOS). Then restart zeppelin, >>>> that's all. >>>> [TO DO] >>>> Seems like it is not good chemistry between Spark and Zeppelin on >>>> Mac. From the reason that the consequence of removing the environment >>>> parameters, Zeppelin headed to own spark lib. >>>> Maybe should write this in the document as caution. >>>> >>>> >>>> Cheers, >>>> >>>> Ono Keiji >>>> >>>> >>>> >>>> >>>> 2018-03-06 9:15 GMT+09:00 小野圭二 <onoke...@gmail.com>: >>>> >>>>> Nothing changed. I guess this property does not effect cause of >>>>> failing to call the interpreter. >>>>> i challenged it as log4j.......zeppelin=DEBUG, but no way. >>>>> >>>>> 2018-03-05 18:24 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >>>>> >>>>>> >>>>>> Try to add the following to log4j.properties for more log >>>>>> >>>>>> log4j.logger.org.apache.zeppelin.interpreter=DEBUG >>>>>> >>>>>> >>>>>> >>>>>> 小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午5:13写道： >>>>>> >>>>>>> Actually, there was not created the interpreter log file as said >>>>>>> 'Cannot run interpreter' in the log. And it was run on local machine as >>>>>>> MAC, i mean stand alone. i stopped every firewall to make it easy. And >>>>>>> again,this happened just on Mac. On CentOS, it works fine. >>>>>>> That makes me confusion. :-b >>>>>>> >>>>>>> 2018-03-05 14:06 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >>>>>>> >>>>>>>> >>>>>>>> Could you check the interpreter log ? The issue of not be able to >>>>>>>> connecting to thrift port could happen due to network issue, like >>>>>>>> firewall >>>>>>>> or you use VPN, etc. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午12:32写道： >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I show the all log on this error.There are 2 type error. The 1 is >>>>>>>>> at the first run. The 2 is at the second run. Seems like could not >>>>>>>>> established to Thrift callback port 51348 that was set at the first >>>>>>>>> run. >>>>>>>>> First of all, does it need to establish the callback port of >>>>>>>>> Thrift server? >>>>>>>>> The notebook script was just >>>>>>>>> %pyspark >>>>>>>>> print 'aaa' >>>>>>>>> Any advice appreciate. >>>>>>>>> >>>>>>>>> 1. >>>>>>>>> INFO [2018-03-05 13:11:13,914] ({qtp1651855867-13} >>>>>>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U INFO >>>>>>>>> [2018-03-05 13:11:13,934] ({pool-2-thread-2} >>>>>>>>> SchedulerFactory.java[jobStarted]:109) >>>>>>>>> - Job 20180302-135047_99102973 started by scheduler >>>>>>>>> org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter-shared_session INFO >>>>>>>>> [2018-03-05 13:11:13,936] ({pool-2-thread-2} >>>>>>>>> Paragraph.java[jobRun]:397) - >>>>>>>>> run paragraph 20180302-135047_99102973 using pyspark >>>>>>>>> RemoteInterpreter_org.apache.zeppelin.spark. >>>>>>>>> PySparkInterpreter_shared_session INFO [2018-03-05 13:11:13,936] >>>>>>>>> ({pool-2-thread-2} ManagedInterpreterGroup.java[ >>>>>>>>> getOrCreateInterpreterProcess]:57) - Create InterperterProcess >>>>>>>>> for InterpreterGroup: 2CV1983WX:shared_process WARN [2018-03-05 >>>>>>>>> 13:11:14,010] ({pool-2-thread-2} >>>>>>>>> InterpreterSetting.java[setupPropertiesForSparkR]:796) >>>>>>>>> - sparkr.zip is not found, SparkR may not work. INFO [2018-03-05 >>>>>>>>> 13:11:14,017] ({pool-2-thread-2} >>>>>>>>> RemoteInterpreterManagedProcess.java[start]:123) >>>>>>>>> - Choose port 51347 for RemoteInterpreterProcess INFO [2018-03-05 >>>>>>>>> 13:11:14,023] ({pool-2-thread-2} >>>>>>>>> RemoteInterpreterManagedProcess.java[start]:130) >>>>>>>>> - Thrift server for callback will start. Port: 51348 INFO [2018-03-05 >>>>>>>>> 13:11:14,049] ({pool-2-thread-2} >>>>>>>>> RemoteInterpreterManagedProcess.java[start]:205) >>>>>>>>> - Run interpreter process >>>>>>>>> [/Users/onokeiji/zeppelin/zeppelin/bin/interpreter.sh, >>>>>>>>> -d, /Users/onokeiji/zeppelin/zeppelin/interpreter/spark, -c, >>>>>>>>> 172.16.20.10, -p, 51348, -l, >>>>>>>>> /Users/onokeiji/zeppelin/zeppelin/local-repo/2CV1983WX, >>>>>>>>> -g, spark] INFO [2018-03-05 13:11:14,238] ({Exec Default Executor} >>>>>>>>> RemoteInterpreterManagedProcess.java[onProcessComplete]:258) - >>>>>>>>> Interpreter process exited 0 ERROR [2018-03-05 13:12:14,058] >>>>>>>>> ({pool-2-thread-2} Job.java[run]:184) - Job failed >>>>>>>>> org.apache.zeppelin. >>>>>>>>> interpreter.InterpreterException: Cannot run interpreter at >>>>>>>>> org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterManagedProcess.start( >>>>>>>>> RemoteInterpreterManagedProcess.java:220) at org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter. >>>>>>>>> getOrCreateInterpreterProcess(RemoteInterpreter.java:101) at >>>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_ >>>>>>>>> create(RemoteInterpreter.java:153) at org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265) at >>>>>>>>> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) >>>>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182) at >>>>>>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) >>>>>>>>> at >>>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at >>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at >>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>>>>>> at java.lang.Thread.run(Thread.java:748) ERROR [2018-03-05 >>>>>>>>> 13:12:14,075] ({pool-2-thread-2} >>>>>>>>> NotebookServer.java[afterStatusChange]:2219) >>>>>>>>> - Error org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> Cannot run interpreter at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterManagedProcess.start( >>>>>>>>> RemoteInterpreterManagedProcess.java:220) at org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter. >>>>>>>>> getOrCreateInterpreterProcess(RemoteInterpreter.java:101) at >>>>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_ >>>>>>>>> create(RemoteInterpreter.java:153) at org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265) at >>>>>>>>> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) >>>>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182) at >>>>>>>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307) >>>>>>>>> at >>>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at >>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at >>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>>>>>>> at java.lang.Thread.run(Thread.java:748) WARN [2018-03-05 >>>>>>>>> 13:12:14,075] ({pool-2-thread-2} >>>>>>>>> NotebookServer.java[afterStatusChange]:2227) >>>>>>>>> - Job 20180302-135047_99102973 is finished, status: ERROR, exception: >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: Cannot run >>>>>>>>> interpreter, result: Cannot run interpreter INFO [2018-03-05 >>>>>>>>> 13:12:14,103] >>>>>>>>> ({pool-2-thread-2} VFSNotebookRepo.java[save]:221) - Saving >>>>>>>>> note:2D6S6TB3U INFO [2018-03-05 13:12:14,109] ({pool-2-thread-2} >>>>>>>>> SchedulerFactory.java[jobFinished]:115) - Job >>>>>>>>> 20180302-135047_99102973 finished by scheduler org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter-shared_session >>>>>>>>> >>>>>>>>> 2. >>>>>>>>> INFO [2018-03-05 13:15:32,736] ({qtp1651855867-13} >>>>>>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U >>>>>>>>> INFO [2018-03-05 13:15:32,751] ({pool-2-thread-2} >>>>>>>>> SchedulerFactory.java[jobStarted]:109) - Job >>>>>>>>> 20180302-135047_99102973 started by scheduler org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter-shared_session >>>>>>>>> INFO [2018-03-05 13:15:32,753] ({pool-2-thread-2} >>>>>>>>> Paragraph.java[jobRun]:397) - run paragraph 20180302-135047_99102973 >>>>>>>>> using >>>>>>>>> pyspark RemoteInterpreter_org.apache.zeppelin.spark. >>>>>>>>> PySparkInterpreter_shared_session >>>>>>>>> ERROR [2018-03-05 13:15:32,882] ({pool-2-thread-2} >>>>>>>>> Job.java[run]:184) - Job failed >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterProcess.callRemoteFunction( >>>>>>>>> RemoteInterpreterProcess.java:152) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreter.internal_create(RemoteInterpreter.java:154) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open( >>>>>>>>> RemoteInterpreter.java:126) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265) >>>>>>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph. >>>>>>>>> java:423) >>>>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182) >>>>>>>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run( >>>>>>>>> RemoteScheduler.java:307) >>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter. >>>>>>>>> call(Executors.java:511) >>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor. >>>>>>>>> java:180) >>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker( >>>>>>>>> ThreadPoolExecutor.java:1149) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>>>>>>>> ThreadPoolExecutor.java:624) >>>>>>>>> at java.lang.Thread.run(Thread.java:748) >>>>>>>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create( >>>>>>>>> ClientFactory.java:53) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create( >>>>>>>>> ClientFactory.java:37) >>>>>>>>> at org.apache.commons.pool2.BasePooledObjectFactory.makeObject( >>>>>>>>> BasePooledObjectFactory.java:60) >>>>>>>>> at org.apache.commons.pool2.impl.GenericObjectPool.create( >>>>>>>>> GenericObjectPool.java:861) >>>>>>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject( >>>>>>>>> GenericObjectPool.java:435) >>>>>>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject( >>>>>>>>> GenericObjectPool.java:363) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java: >>>>>>>>> 73) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterProcess.callRemoteFunction( >>>>>>>>> RemoteInterpreterProcess.java:144) >>>>>>>>> ... 13 more >>>>>>>>> Caused by: org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create( >>>>>>>>> ClientFactory.java:51) >>>>>>>>> ... 20 more >>>>>>>>> Caused by: java.net.ConnectException: Connection refused >>>>>>>>> (Connection refused) >>>>>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>>>>> at java.net.AbstractPlainSocketImpl.doConnect( >>>>>>>>> AbstractPlainSocketImpl.java:350) >>>>>>>>> at java.net.AbstractPlainSocketImpl.connectToAddress( >>>>>>>>> AbstractPlainSocketImpl.java:206) >>>>>>>>> at java.net.AbstractPlainSocketImpl.connect( >>>>>>>>> AbstractPlainSocketImpl.java:188) >>>>>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>>>>>> at java.net.Socket.connect(Socket.java:589) >>>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182) >>>>>>>>> ... 21 more >>>>>>>>> ERROR [2018-03-05 13:15:32,884] ({pool-2-thread-2} >>>>>>>>> NotebookServer.java[afterStatusChange]:2219) - Error >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterProcess.callRemoteFunction( >>>>>>>>> RemoteInterpreterProcess.java:152) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreter.internal_create(RemoteInterpreter.java:154) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open( >>>>>>>>> RemoteInterpreter.java:126) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreter.getFormType(RemoteInterpreter.java:265) >>>>>>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph. >>>>>>>>> java:423) >>>>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:182) >>>>>>>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run( >>>>>>>>> RemoteScheduler.java:307) >>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter. >>>>>>>>> call(Executors.java:511) >>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor. >>>>>>>>> java:180) >>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ >>>>>>>>> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker( >>>>>>>>> ThreadPoolExecutor.java:1149) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>>>>>>>> ThreadPoolExecutor.java:624) >>>>>>>>> at java.lang.Thread.run(Thread.java:748) >>>>>>>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create( >>>>>>>>> ClientFactory.java:53) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create( >>>>>>>>> ClientFactory.java:37) >>>>>>>>> at org.apache.commons.pool2.BasePooledObjectFactory.makeObject( >>>>>>>>> BasePooledObjectFactory.java:60) >>>>>>>>> at org.apache.commons.pool2.impl.GenericObjectPool.create( >>>>>>>>> GenericObjectPool.java:861) >>>>>>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject( >>>>>>>>> GenericObjectPool.java:435) >>>>>>>>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject( >>>>>>>>> GenericObjectPool.java:363) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java: >>>>>>>>> 73) >>>>>>>>> at org.apache.zeppelin.interpreter.remote. >>>>>>>>> RemoteInterpreterProcess.callRemoteFunction( >>>>>>>>> RemoteInterpreterProcess.java:144) >>>>>>>>> ... 13 more >>>>>>>>> Caused by: org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187) >>>>>>>>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create( >>>>>>>>> ClientFactory.java:51) >>>>>>>>> ... 20 more >>>>>>>>> Caused by: java.net.ConnectException: Connection refused >>>>>>>>> (Connection refused) >>>>>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>>>>> at java.net.AbstractPlainSocketImpl.doConnect( >>>>>>>>> AbstractPlainSocketImpl.java:350) >>>>>>>>> at java.net.AbstractPlainSocketImpl.connectToAddress( >>>>>>>>> AbstractPlainSocketImpl.java:206) >>>>>>>>> at java.net.AbstractPlainSocketImpl.connect( >>>>>>>>> AbstractPlainSocketImpl.java:188) >>>>>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>>>>>>>> at java.net.Socket.connect(Socket.java:589) >>>>>>>>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182) >>>>>>>>> ... 21 more >>>>>>>>> WARN [2018-03-05 13:15:32,885] ({pool-2-thread-2} >>>>>>>>> NotebookServer.java[afterStatusChange]:2227) - Job >>>>>>>>> 20180302-135047_99102973 is finished, status: ERROR, exception: >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused), >>>>>>>>> result: >>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>> INFO [2018-03-05 13:15:32,910] ({pool-2-thread-2} >>>>>>>>> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U >>>>>>>>> INFO [2018-03-05 13:15:32,914] ({pool-2-thread-2} >>>>>>>>> SchedulerFactory.java[jobFinished]:115) - Job >>>>>>>>> 20180302-135047_99102973 finished by scheduler org.apache.zeppelin. >>>>>>>>> interpreter.remote.RemoteInterpreter-shared_session >>>>>>>>> >>>>>>>>> 2018-03-02 15:07 GMT+09:00 小野圭二 <onoke...@gmail.com>: >>>>>>>>> >>>>>>>>>> Hi Jeff, >>>>>>>>>> >>>>>>>>>> No one graved 63039 so far on Mac. >>>>>>>>>> Yes, this issue has been reproduced as changing the port NO >>>>>>>>>> randomly. >>>>>>>>>> >>>>>>>>>> Ono >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2018-03-02 14:49 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> It is possible that 63039 is used by other processes, as Mac >>>>>>>>>>> can not detect whether this port is used or not from my experience. >>>>>>>>>>> >>>>>>>>>>> And can you reproduce this issue consistently ? Because each >>>>>>>>>>> time zeppelin will chose a random port as long as you don't specify >>>>>>>>>>> the >>>>>>>>>>> portRange, so if you reproduce it consistently, it might due to >>>>>>>>>>> other >>>>>>>>>>> issues. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 小野圭二 <onoke...@gmail.com>于2018年3月2日周五 下午1:32写道： >>>>>>>>>>> >>>>>>>>>>>> Hi there, >>>>>>>>>>>> >>>>>>>>>>>> I am so confusing now, let me know why if You know this, >>>>>>>>>>>> please. >>>>>>>>>>>> >>>>>>>>>>>> The env: >>>>>>>>>>>> Mac: High Sierra ver.10.13.3 >>>>>>>>>>>> Zep: 0.7.3 >>>>>>>>>>>> Spark: 2.2.0 >>>>>>>>>>>> Java: 1.8 >>>>>>>>>>>> notebook script: %pyspark print 'aaa' (just >>>>>>>>>>>> it for test) >>>>>>>>>>>> >>>>>>>>>>>> In this env, i got the error in log as >>>>>>>>>>>> >>>>>>>>>>>> ERROR ({pool-2-thread-2} Job.java[run]:184) - Job failed >>>>>>>>>>>> >>>>>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>>>>> org.apache.zeppelin.interpreter.InterpreterException: >>>>>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>>>>> java.net.ConnectException: Connection refused (Connection refused) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Indeed, before running notebook, >>>>>>>>>>>> RemoteInterpreterManagedProcess.java >>>>>>>>>>>> prepares a Port for callback to Thrift. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> INFO ({pool-2-thread-1} >>>>>>>>>>>> RemoteInterpreterManagedProcess.java[start]:130) >>>>>>>>>>>> - Thrift server for callback will start. Port: 63039 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I guess, the reason why the error was Zeppelin could not >>>>>>>>>>>> attache to the Thrift PORT. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> And my confusion is, when i run the same module on CentOS, it >>>>>>>>>>>> worked fine. >>>>>>>>>>>> CentOS: 7 >>>>>>>>>>>> Zep: same >>>>>>>>>>>> Spark: same >>>>>>>>>>>> Java: same >>>>>>>>>>>> notebook script: same >>>>>>>>>>>> >>>>>>>>>>>> And there was no the Thift port description in the log file. >>>>>>>>>>>> From this consequence, the fine working was cause of did not use >>>>>>>>>>>> Thrift >>>>>>>>>>>> port on CentOS. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Do You know actually there is different behavior between Mac >>>>>>>>>>>> and CentOS? And i am so appreciated if You know the workaround on >>>>>>>>>>>> Mac. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Best, >>>>>>>>>>>> Ono >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>> >>
