This is probably a versioning issue, are you sure your code is compiling and running against the same versions? On Oct 14, 2015 2:19 PM, "Shreeharsha G Neelakantachar" < shreeharsh...@in.ibm.com> wrote:
> Hi, > I have Terasort being executed on spark1.4.1 with hadoop 2.7 for a > datasize of 1TB. When i change my os user from spark1 to hduser, i am > observing below exception. Please let me know what is wrong here. I tried > to update scala-2.10 to 2.11 and compiled Terasort scala code using sbt. > But nothing is helping here. I had created the spark1 (spark1 to spark4) > user for a multi-tenancy run. > > 2015-10-13 03:42:54,840 [sparkDriver-akka.actor.default-dispatcher-2] INFO > org.apache.spark.scheduler.TaskSetManager - Starting task 199.0 in stage > 0.0 (TID 199, 9.37.251.65, ANY, 1544 bytes) > 2015-10-13 03:42:54,843 [task-result-getter-2] WARN > org.apache.spark.scheduler.TaskSetManager - Lost task 173.0 in stage 0.0 > (TID 173, 9.37.251.65): java.io.InvalidClassException: > scala.reflect.ClassTag$$anon$1; local class incompatible: stream classdesc > serialVersionUID = -4937928798201944954, local class serialVersionUID = > -8102093212602380348 > at > java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:621) > at > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) > at > java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) > at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993) > at > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) > at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993) > at > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) > at > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) > at > org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:69) > at > org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:95) > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:194) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > > Any help in this regard would be of great help. > > Regards > Shreeharsha GN > >