Finally got it working by increasing the spark version in maven to 1.2.1 On Thu, Apr 9, 2015 at 12:30 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
> I changed the JDK to Oracle but I still get this error. Not sure what it > means by "Stream class is incompatible with local class". I am using the > following build on the server "spark-1.2.1-bin-hadoop2.4" > > 15/04/09 15:26:24 ERROR JobScheduler: Error running job streaming job > 1428607584000 ms.0 > org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 > in stage 15.0 failed 4 times, most recent failure: Lost task 0.3 in stage > 15.0 (TID 66, ip-10-241-251-232.us-west-2.compute.internal): > java.io.InvalidClassException: org.apache.spark.Aggregator; local class > incompatible: stream classdesc serialVersionUID = 5032037208639381169, > local class serialVersionUID = -9085606473104903453 > at > java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) > > On Wed, Apr 8, 2015 at 3:40 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> bq. one is Oracle and the other is OpenJDK >> >> I don't have experience with mixed JDK's. >> Can you try with using single JDK ? >> >> Cheers >> >> On Wed, Apr 8, 2015 at 3:26 PM, Mohit Anchlia <mohitanch...@gmail.com> >> wrote: >> >>> For the build I am using java version "1.7.0_65" which seems to be the >>> same as the one on the spark host. However one is Oracle and the other is >>> OpenJDK. Does that make any difference? >>> >>> On Wed, Apr 8, 2015 at 1:24 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>> >>>> What version of Java do you use to build ? >>>> >>>> Cheers >>>> >>>> On Wed, Apr 8, 2015 at 12:43 PM, Mohit Anchlia <mohitanch...@gmail.com> >>>> wrote: >>>> >>>>> I am seeing the following, is this because of my maven version? >>>>> >>>>> 15/04/08 15:42:22 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID >>>>> 0, ip-10-241-251-232.us-west-2.compute.internal): >>>>> java.io.InvalidClassException: org.apache.spark.Aggregator; local class >>>>> incompatible: stream classdesc serialVersionUID = 5032037208639381169, >>>>> local class serialVersionUID = -9085606473104903453 >>>>> at >>>>> java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) >>>>> at >>>>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622) >>>>> at >>>>> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) >>>>> at >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) >>>>> at >>>>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) >>>>> at >>>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) >>>>> at >>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) >>>>> at >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) >>>>> at >>>>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) >>>>> >>>>> >>>>> <dependency> >>>>> >>>>> <groupId>org.apache.spark</groupId> >>>>> >>>>> <artifactId>spark-streaming_2.10</artifactId> >>>>> >>>>> <version>1.2.0</version> >>>>> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> >>>>> <groupId>org.apache.spark</groupId> >>>>> >>>>> <artifactId>spark-core_2.10</artifactId> >>>>> >>>>> <version>1.2.0</version> >>>>> >>>>> </dependency> >>>>> >>>>> >>>> >>> >> >
