Re: Kryo Serializer on Worker doesn't work by default.

2015-07-08 Thread Eugene Morozov
What I seem to be don’t get is how my code ends up being on Worker node. My understanding was that jar file, which I use to start the job should automatically be copied into Worker nodes and added to classpath. It seems to be not the case. But if my jar is not copied into Worker nodes, then how

Kryo Serializer on Worker doesn't work by default.

2015-07-08 Thread Eugene Morozov
Hello. I have an issue with CustomKryoRegistrator, which causes ClassNotFound on Worker. The issue is resolved if call SparkConf.setJar with path to the same jar I run. It is a workaround, but it requires to specify the same jar file twice. The first time I use it to actually run the job, and