Re: Bug is KryoSerializer under Mesos [work-around included]

2014-05-15 Thread Soren Macbeth
Hi Matei, Yes, I'm 100% positive the jar on the executors is the same version. I am building everything and deploying myself. Additionally, while debugging the issue, I forked spark's git repo and added additional logging, which I could see in the driver and executors. These debugging jars

Re: Bug is KryoSerializer under Mesos [work-around included]

2014-05-12 Thread Matei Zaharia
Hey Soren, are you sure that the JAR you used on the executors is for the right version of Spark? Maybe they’re running an older version. The Kryo serializer should be initialized the same way on both. Matei On May 12, 2014, at 10:39 AM, Soren Macbeth so...@yieldbot.com wrote: I finally

Bug is KryoSerializer under Mesos [work-around included]

2014-05-12 Thread Soren Macbeth
I finally managed to track down the source of the kryo issues that I was having under mesos. What happens is the for a reason that I haven't tracked down yet, a handful of the scala collection classes from chill-scala down get registered by the mesos executors, but they do all get registered in