It looks like we do have two versions: ./zeppelin/spark/spark-dependencies/target/spark-2.4.0/pom.xml: <libthrift.version>0.9.3</libthrift.version>
./zeppelin/pom.xml: <libthrift.version>0.9.2</libthrift.version> I’m always hesitant to rebuild blindly with changes. Did you align your versions to 0.9.3 and have success? Can anyone weigh in on this discrepancy? 11:33 AM ./zeppelin/pom.xml: <libthrift.version>0.9.2</libthrift.version> PAUL BRENNER Head of Data Science pbren...@placeiq.com | (217) 390-3033 | www.placeiq.com twitter @placeiq linkedin /placeiq On Sep 4, 2019, 7:57 AM -0400, James Srinivasan <james.sriniva...@gmail.com>, wrote: > Is there a chance you have differing versions of libthrift on your classpath? > I've seen errors (admittedly not looking like yours) for that reason? > > James > > > On Sun, 1 Sep 2019 at 00:57, Paul Brenner <pbren...@placeiq.com> wrote: > > > We are using a recent build post 0.81 so 0.82? > > > > > > Restarting the interpreter helps in 75% of the cases, but the remaining > > > few just won’t recover until we shut everything down and restart all if > > > zeppelin. I suspect there is some other process that we aren’t > > > identifying that needs killing. > > > > > > > > > > > > PAUL BRENNER > > > Head of Data Science > > > pbren...@placeiq.com | (217) 390-3033 | www.placeiq.com > > > twitter @placeiq linkedin /placeiq > > > On Aug 31, 2019, 7:50 PM -0400, Jeff Zhang <zjf...@gmail.com>, wrote: > > > > Which version do you use ? And does restarting interpreter work for you > > > > ? > > > > > > > > > Paul Brenner <pbren...@placeiq.com> 于2019年8月30日周五 下午11:45写道: > > > > > > We run into the error pasted below semi-regularly and often the > > > > > > only solution is to do a full restart of zeppelin to get the > > > > > > user/notebook back up and running. I’m at a loss of where to even > > > > > > look to try to debug this or address it. Anyone have any ideas or > > > > > > any way to point me in the right direction? > > > > > > > > > > > > This is what the notebook shows: > > > > > > org.apache.thrift.transport.TTransportException at > > > > > > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > > > > > > at > > > > > > org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) > > > > > > at > > > > > > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:274) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:258) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:233) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:229) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:228) > > > > > > at > > > > > > org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:437) > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at > > > > > > org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315) > > > > > > at > > > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > > > > > > > This is what I pulled from the log: > > > > > > > > > > > > ERROR [2019-08-30 15:41:56,923] ({pool-2-thread-24} > > > > > > Job.java[run]:190) - Job failed > > > > > > java.lang.RuntimeException: > > > > > > org.apache.thrift.transport.TTransportException > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:139) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:228) > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:437) > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > at > > > > > > org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315) > > > > > > at > > > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > Caused by: org.apache.thrift.transport.TTransportException > > > > > > at > > > > > > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > > > > > > at > > > > > > org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) > > > > > > at > > > > > > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:274) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:258) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:233) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:229) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135) > > > > > > ... 11 more > > > > > > INFO [2019-08-30 15:41:56,954] ({Exec Default Executor} > > > > > > RemoteInterpreterManagedProcess.java[onProcessComplete]:243) - > > > > > > Interpreter process exited 0 > > > > > > DEBUG [2019-08-30 15:41:56,957] ({pool-2-thread-24} > > > > > > RemoteScheduler.java[run]:332) - Job ABORT, > > > > > > 20190821-204527_149556857, > > > > > > org.apache.thrift.transport.TTransportException > > > > > > at > > > > > > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > > > > > > at > > > > > > org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) > > > > > > at > > > > > > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:274) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:258) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:233) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:229) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:228) > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:437) > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > at > > > > > > org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315) > > > > > > at > > > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > ERROR [2019-08-30 15:41:56,959] ({pool-2-thread-24} > > > > > > NotebookServer.java[afterStatusChange]:2308) - Error > > > > > > java.lang.RuntimeException: > > > > > > org.apache.thrift.transport.TTransportException > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:139) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:228) > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:437) > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > at > > > > > > org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315) > > > > > > at > > > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > Caused by: org.apache.thrift.transport.TTransportException > > > > > > at > > > > > > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > > > > > > at > > > > > > org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) > > > > > > at > > > > > > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:274) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:258) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:233) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:229) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135) > > > > > > ... 11 more > > > > > > WARN [2019-08-30 15:41:56,959] ({pool-2-thread-24} > > > > > > NotebookServer.java[afterStatusChange]:2316) - Job > > > > > > 20190821-204527_149556857 is finished, status: ERROR, exception: > > > > > > java.lang.RuntimeException: > > > > > > org.apache.thrift.transport.TTransportException, result: %text > > > > > > org.apache.thrift.transport.TTransportException > > > > > > at > > > > > > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > > > > > > at > > > > > > org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) > > > > > > at > > > > > > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) > > > > > > at > > > > > > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:274) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:258) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:233) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter$4.call(RemoteInterpreter.java:229) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135) > > > > > > at > > > > > > org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:228) > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:437) > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > at > > > > > > org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315) > > > > > > at > > > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > > > > > at > > > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > > > > > > > > > > > > > PAUL BRENNER > > > > > > Head of Data Science > > > > > > pbren...@placeiq.com | (217) 390-3033 | www.placeiq.com > > > > > > twitter @placeiq linkedin /placeiq > > > > > > > > > > > > -- > > > > Best Regards > > > > > > > > Jeff Zhang