I changed the JDK to Oracle but I still get this error. Not sure what it
means by "Stream class is incompatible with local class". I am using the
following build on the server "spark-1.2.1-bin-hadoop2.4"
15/04/09 15:26:24 ERROR JobScheduler: Error running job streaming job
1428607584000 ms.0
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0
in stage 15.0 failed 4 times, most recent failure: Lost task 0.3 in stage
15.0 (TID 66, ip-10-241-251-232.us-west-2.compute.internal):
java.io.InvalidClassException: org.apache.spark.Aggregator; local class
incompatible: stream classdesc serialVersionUID = 5032037208639381169,
local class serialVersionUID = -9085606473104903453
at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
On Wed, Apr 8, 2015 at 3:40 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> bq. one is Oracle and the other is OpenJDK
>
> I don't have experience with mixed JDK's.
> Can you try with using single JDK ?
>
> Cheers
>
> On Wed, Apr 8, 2015 at 3:26 PM, Mohit Anchlia <mohitanch...@gmail.com>
> wrote:
>
>> For the build I am using java version "1.7.0_65" which seems to be the
>> same as the one on the spark host. However one is Oracle and the other is
>> OpenJDK. Does that make any difference?
>>
>> On Wed, Apr 8, 2015 at 1:24 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>
>>> What version of Java do you use to build ?
>>>
>>> Cheers
>>>
>>> On Wed, Apr 8, 2015 at 12:43 PM, Mohit Anchlia <mohitanch...@gmail.com>
>>> wrote:
>>>
>>>> I am seeing the following, is this because of my maven version?
>>>>
>>>> 15/04/08 15:42:22 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID
>>>> 0, ip-10-241-251-232.us-west-2.compute.internal):
>>>> java.io.InvalidClassException: org.apache.spark.Aggregator; local class
>>>> incompatible: stream classdesc serialVersionUID = 5032037208639381169,
>>>> local class serialVersionUID = -9085606473104903453
>>>> at
>>>> java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617)
>>>> at
>>>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
>>>> at
>>>> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
>>>> at
>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
>>>> at
>>>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
>>>> at
>>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
>>>> at
>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
>>>> at
>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
>>>> at
>>>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
>>>>
>>>>
>>>> <dependency>
>>>>
>>>> <groupId>org.apache.spark</groupId>
>>>>
>>>> <artifactId>spark-streaming_2.10</artifactId>
>>>>
>>>> <version>1.2.0</version>
>>>>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>>
>>>> <groupId>org.apache.spark</groupId>
>>>>
>>>> <artifactId>spark-core_2.10</artifactId>
>>>>
>>>> <version>1.2.0</version>
>>>>
>>>> </dependency>
>>>>
>>>>
>>>
>>
>
