Ok, I found the mistake!. Wow, it really come to me by inspiration, if not I don't know how this suddenly come to me.
The problem is that I packed my application in a jar file with the old "spark-assembly-0.8.0-incubating-hadoop1.0.4.jar" in it. So that is the reason why workers and master have different versions of the file. I re-packeted it and run it just fine!. Thanks for all the help, Eduardo. On Thu, Oct 3, 2013 at 4:05 PM, Eduardo Berrocal <eberr...@hawk.iit.edu>wrote: > And by .java, I mean .scala > > > On Thu, Oct 3, 2013 at 4:03 PM, Eduardo Berrocal <eberr...@hawk.iit.edu>wrote: > >> That directory is not present on version 0.8.0 (the one I am using). >> >> However, there are the files for FlatMapFunction in spark 0.8.0: >> >> $ find ./ -name FlatMapFunction* >> >> ./assembly/target/scala-2.9.3/cache/assembly/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction2.class >> >> ./assembly/target/scala-2.9.3/cache/assembly/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction.class >> >> ./core/src/main/scala/org/apache/spark/api/java/function/FlatMapFunction2.scala >> >> ./core/src/main/scala/org/apache/spark/api/java/function/FlatMapFunction.scala >> >> ./core/target/scala-2.9.3/classes/org/apache/spark/api/java/function/FlatMapFunction2.class >> >> ./core/target/scala-2.9.3/classes/org/apache/spark/api/java/function/FlatMapFunction.class >> >> ./examples/target/scala-2.9.3/cache/examples/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction2.class >> >> ./examples/target/scala-2.9.3/cache/examples/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction.class >> >> All of the FlatMapFunction.class files have a modification time bigger >> than the .java file, which I suspect means that they have been created from >> the new version of the java file (all of them!) >> >> Cheers, >> Eduardo. >> >> >> On Thu, Oct 3, 2013 at 3:38 PM, Martin Weindel >> <martin.wein...@gmail.com>wrote: >> >>> Hi Eduardo, >>> >>> if you are using Spark 0.7.3, I remember that I had to replace the class >>> file additional at >>> spark-0.7.3/core/target/scala-2.9.3/classes/spark/api/java/function/ >>> >>> Martin >>> >>> Am 03.10.2013 22:35, schrieb Eduardo Berrocal: >>> >>> Hi Martin, >>> >>> Yes, that is what is seems. However, it is unlikely that is the case, >>> because I have all spark classes on my home, which is mounted on NFS to all >>> nodes. Unless there is something else I am missing... >>> >>> Edu >>> >>> >>> On Thu, Oct 3, 2013 at 3:29 PM, Martin Weindel <martin.wein...@gmail.com >>> > wrote: >>> >>>> Hi Eduardo, >>>> >>>> it seems to me that your second problem is caused by inconsistent, i.e. >>>> different classes in master and worker JVMs. >>>> Are you sure, that you have replaced the changed FlatMapFunction on all >>>> worker nodes and also on master? >>>> >>>> Regards, >>>> Martin >>>> >>>> 13/10/03 13:27:44 INFO cluster.ClusterTaskSetManager: Lost TID 0 (task >>>> 1.0:0) >>>> 13/10/03 13:27:44 INFO cluster.ClusterTaskSetManager: Loss was due to >>>> java.io.InvalidClassException >>>> java.io.InvalidClassException: >>>> org.apache.spark.api.java.function.FlatMapFunction; local class >>>> incompatible: stream classdesc serialVersionUID = -1748278142466443391, >>>> local class serialVersionUID = 2220150375729402137 >>>> >>>> >>>> >>> >>> >> >
