Alex Bush created ZEPPELIN-3656:
-----------------------------------
Summary: Livy Code Completion does not work when using Shared
Interpreter
Key: ZEPPELIN-3656
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3656
Project: Zeppelin
Issue Type: Bug
Components: livy-interpreter
Affects Versions: 0.8.0
Environment: {{Zeppelin 0.8.0}}
Livy 0.5
Spark 2.1
Kerberos+SSL
Reporter: Alex Bush
Fix For: 0.9.0, 0.8.1
When attempting to use code completion with Livy 0.5 I get a
NullPointerException:
{code:java}
2018-07-24 14:05:05,504 ERROR org.apache.thrift.server.TThreadPoolServer: Error
occurred during processing of message.
java.lang.NullPointerException
at
org.apache.zeppelin.livy.BaseLivyInterpreter.callCompletion(BaseLivyInterpreter.java:284)
at
org.apache.zeppelin.livy.BaseLivyInterpreter.completion(BaseLivyInterpreter.java:271)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.completion(LazyOpenInterpreter.java:138)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.completion(RemoteInterpreterServer.java:736)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$completion.getResult(RemoteInterpreterService.java:1940)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$completion.getResult(RemoteInterpreterService.java:1925)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-07-24 14:05:05,508 INFO org.apache.zeppelin.socket.NotebookServer: Fail to
get completion
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:139)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:358)
at org.apache.zeppelin.notebook.Paragraph.completion(Paragraph.java:270)
at org.apache.zeppelin.notebook.Note.completion(Note.java:729)
at
org.apache.zeppelin.socket.NotebookServer.completion(NotebookServer.java:1397)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:303)
at
org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
at
org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161)
at
org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:309)
at
org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214)
at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
at
org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632)
at
org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_completion(RemoteInterpreterService.java:372)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.completion(RemoteInterpreterService.java:355)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter$8.call(RemoteInterpreter.java:362)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter$8.call(RemoteInterpreter.java:359)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:135)
... 21 more
{code}
Debugging this shows that it is the call to sessionInfo on this line:
[https://github.com/apache/zeppelin/blob/v0.8.0/livy/src/main/java/org/apache/zeppelin/livy/BaseLivyInterpreter.java#L284]
The code should be referencing sessionInfo via the SessionInfo method to handle
the case of shared interpreter.
It would be nice if this could be fixed in 0.8.1 and 0.9.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
