Hi all,
since I'm having issues with Java serialization I wanted to switch to
Kryo to serialize everything, including closures.
Hence, before creating my spark context, I set the
|spark.closure.serializer| property:
|System.setProperty("spark.closure.serializer","spark.KryoSerializer")|
The problem is that now my program crashes. You can find its last words,
including |DEBUG| statements and stack trace, in this gist
<https://gist.github.com/Cecca/6552078>.
Since removing the line that sets the serializer property for closures
makes the program work again, I guess it's a problem in serializing (or
deserializing) closures.
Is there something else I should do besides setting that property?
Thank you very much
Matteo
--
Matteo Ceccarello, Computer Engineering student
Universita' di Padova, Dipartimento di Ingegneria dell'Informazione,
URL: http://www.dei.unipd.it/~ceccarel
LinkedIn: http://it.linkedin.com/pub/matteo-ceccarello/60/322/931/
