Re: Spark shell broken

Andrew Palumbo Tue, 24 Feb 2015 15:01:00 -0800

ok finally pushed it to the spark-1.2 branch. the shell on 1.2 should beworking now also.


On 02/24/2015 05:45 PM, Andrew Musselman wrote:

Using Spark v1.1.0 cleared up those errors.


On Tue, Feb 24, 2015 at 2:42 PM, Andrew Palumbo <[email protected]> wrote:

yeah- i was working off of an old branch and couldn't figure out how to
get the mahout/spark-1.2 branch. I got it.  just double checking everything
and will push it shortly.


On 02/24/2015 05:32 PM, Dmitriy Lyubimov wrote:

all i am saying if spark side tests work imo there's no need to go thru PR
process, you can push your squashed patch directly, but if you feel like
to
do PR please go ahead.

On Tue, Feb 24, 2015 at 2:30 PM, Andrew Palumbo <[email protected]>
wrote:

  or rather i have [email protected]:andrewpalumbo/mahout.git as my origin

and
there's no spark-1.2 branch there

On 02/24/2015 05:25 PM, Andrew Palumbo wrote:

  sorry- trying to get the shell fix out but having trouble pulling the

remote spark 1.2 branch. i think its cause i have git.apache.org as my
remote rather than github.com/apache/mahout


On 02/24/2015 05:15 PM, Andrew Musselman wrote:

  Makes sense; I'm still getting those errors after restarting my rebuilt

spark..

On Tue, Feb 24, 2015 at 2:12 PM, Dmitriy Lyubimov <[email protected]>
wrote:

   IIRC MAHOUT_LOCAL doesn't mean a thing with spark mode. It is purely
MR

thing.

with shell, important things are MASTER (spark setting) and also other
spark specific settings  that should be added in MAHOUT_OPT (but the
latter
is broken in public branch with spark shell -- one of the bugs that
need
fixing).

On Tue, Feb 24, 2015 at 2:03 PM, Andrew Musselman <
[email protected]> wrote:

   Ah, I had an old build of spark server running.

On Tue, Feb 24, 2015 at 1:56 PM, Andrew Musselman <
[email protected]> wrote:

   I only have one spark build that I know of, and it's still
reporting

errors in Mahout local mode.

$ echo $SPARK_HOME
/home/akm/spark

$ export MAHOUT_LOCAL="TRUE"
$ bin/mahout spark-shell
MAHOUT_LOCAL is set, so we don't add HADOOP_CONF_DIR to classpath.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in

   [jar:file:/home/akm/mahout/mrlegacy/target/mahout-

mrlegacy-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/

StaticLoggerBinder.class]


  SLF4J: Found binding in

   [jar:file:/home/akm/mahout/spark/target/mahout-spark_2.

10-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

  SLF4J: Found binding in

   [jar:file:/home/akm/spark/assembly/target/scala-2.10/

spark-assembly-1.1.1-hadoop1.1.1.jar!/org/slf4j/impl/

StaticLoggerBinder.class]


  SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an

explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.
Log4jLoggerFactory]

                            _                 _
            _ __ ___   __ _| |__   ___  _   _| |_
           | '_ ` _ \ / _` | '_ \ / _ \| | | | __|
           | | | | | | (_| | | | | (_) | |_| | |_
           |_| |_| |_|\__,_|_| |_|\___/ \__,_|\__|  version 1.0


Using Scala version 2.10.0 (Java HotSpot(TM) 64-Bit Server VM, Java
1.7.0_03)
Type in expressions to have them evaluated.
Type :help for more information.
15/02/24 13:42:42 WARN Utils: Your hostname, ubuntu resolves to a

  loopback

  address: 127.0.1.1; using 5.5.8.1 instead (on interface as0t1)

15/02/24 13:42:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind
to
another address
15/02/24 13:42:54 WARN NativeCodeLoader: Unable to load
native-hadoop
library for your platform... using builtin-java classes where

  applicable

Created spark context..

Mahout distributed context is available as "implicit val sdc".
mahout> 15/02/24 13:43:32 ERROR Remoting:
org.apache.spark.storage.BlockManagerId; local class incompatible:

  stream

classdesc serialVersionUID = 1677335532749418220, local class

serialVersionUID = -7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.
BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:44:05 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:44:38 ERROR TaskSchedulerImpl: Lost an executor 0

(already

removed): remote Akka client disassociated
15/02/24 13:45:15 ERROR Remoting:

  org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:45:48 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:46:21 ERROR TaskSchedulerImpl: Lost an executor 1

(already

removed): remote Akka client disassociated
15/02/24 13:46:57 ERROR Remoting:

  org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:47:30 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:48:04 ERROR TaskSchedulerImpl: Lost an executor 2

(already

removed): remote Akka client disassociated
15/02/24 13:48:40 ERROR Remoting:

  org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:49:14 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:49:47 ERROR TaskSchedulerImpl: Lost an executor 3

(already

removed): remote Akka client disassociated
15/02/24 13:50:23 ERROR Remoting:

  org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:50:56 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:51:30 ERROR TaskSchedulerImpl: Lost an executor 4

(already

removed): remote Akka client disassociated
15/02/24 13:52:07 ERROR Remoting:

  org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

   java.io.ObjectInputStream.defaultReadFields(
ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(
ObjectInputStream.java:1888)
       at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
       at

   akka.serialization.JavaSerializer$$anonfun$1.
apply(Serializer.scala:136)

        at scala.util.DynamicVariable.withValue(DynamicVariable.

scala:57)

       at

  akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

   akka.serialization.Serialization$$anonfun$deserialize$1.apply(
Serialization.scala:104)

        at scala.util.Try$.apply(Try.scala:161)

       at

  akka.serialization.Serialization.deserialize(

Serialization.scala:98)

        at

akka.remote.MessageSerializer$.deserialize(
MessageSerializer.scala:23)

       at

   akka.remote.DefaultMessageDispatcher.
payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
       at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
       at

   akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(
Endpoint.scala:764)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

       at akka.actor.ActorCell.invoke(ActorCell.scala:456)
       at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
       at akka.dispatch.Mailbox.run(Mailbox.scala:219)
       at

   akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at

   scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

   scala.concurrent.forkjoin.ForkJoinPool.runWorker(
ForkJoinPool.java:1979)

at

   scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)

  15/02/24 13:52:40 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

  local class incompatible: stream classdesc serialVersionUID =

1677335532749418220, local class serialVersionUID =

  -7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

  -7366074099953117729

       at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
       at

java.io.ObjectInputStream.readNonProxyDesc(
ObjectInputStream.java:1601)

       at

  java.io.ObjectInputStream.readClassDesc(

ObjectInputStream.java:1514)

        at

   java.io.ObjectInputStream.readOrdinaryObject(
ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
       at

Re: Spark shell broken

Reply via email to