Jiri Kremser created SPARK-22594: ------------------------------------ Summary: Handling spark-submit and master version mismatch Key: SPARK-22594 URL: https://issues.apache.org/jira/browse/SPARK-22594 Project: Spark Issue Type: Bug Components: Spark Core, Spark Shell, Spark Submit Affects Versions: 2.2.0, 2.1.0 Reporter: Jiri Kremser Priority: Minor
When using spark-submit in different version than the remote Spark master, the execution fails on during the message deserialization with this log entry / exception: {code} Error while invoking RpcHandler#receive() for one-way message. java.io.InvalidClassException: org.apache.spark.rpc.RpcEndpointRef; local class incompatible: stream classdesc serialVersionUID = 1835832137613908542, local class serialVersionUID = -1329125091869941550 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75) at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:108) at org.apache.spark.rpc.netty.NettyRpcEnv$$anonfun$deserialize$1$$anonfun$apply$1.apply(NettyRpcEnv.scala:271) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) at org.apache.spark.rpc.netty.NettyRpcEnv.deserialize(NettyRpcEnv.scala:320) at org.apache.spark.rpc.netty.NettyRpcEnv$$anonfun$deserialize$1.apply(NettyRpcEnv.scala:270) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) at org.apache.spark.rpc.netty.NettyRpcEnv.deserialize(NettyRpcEnv.scala:269) at org.apache.spark.rpc.netty.RequestMessage$.apply(NettyRpcEnv.scala:604) at org.apache.spark.rpc.netty.NettyRpcHandler.internalReceive(NettyRpcEnv.scala:655) at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:647) at org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:209) at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:114) at org.apache.spark.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118) ... {code} This is quite ok and it can be read as version mismatch between the client and server, however there is no such a message on the client (spark-submit) side, so if the submitter doesn't have an access to the spark master or spark UI, there is no way to figure out what is wrong. I propose sending a {{RpcFailure}} message back from server to client with some more informative error. I'd use the {{OneWayMessage}} instead of {{RpcFailure}}, because there was no counterpart {{RpcRequest}}, but I had no luck sending it using the {{reverseClient.send()}}. I think some internal protocol is assumed when sending messages server2client. I have a patch prepared. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org