Re: Spark on Yarn: Kryo throws ClassNotFoundException for class included in fat jar

2015-09-08 Thread Nicholas R. Peterson
; I wouldn't build on this. local mode & yarn are different so that jars >> you use in spark submit are handled differently >> >> On 8 September 2015 at 15:43, Nicholas R. Peterson >> wrote: >> >>> Thans, Igor; I've got it running again right now

Re: Spark on Yarn: Kryo throws ClassNotFoundException for class included in fat jar

2015-09-08 Thread Nicholas R. Peterson
> On 8 September 2015 at 06:38, Nicholas R. Peterson > wrote: > >> I'm trying to run a Spark 1.4.1 job on my CDH5.4 cluster, through Yarn. >> Serialization is set to use Kryo. >> >> I have a large object which I send to the executors as a Broadcast. The

Spark on Yarn: Kryo throws ClassNotFoundException for class included in fat jar

2015-09-07 Thread Nicholas R. Peterson
I'm trying to run a Spark 1.4.1 job on my CDH5.4 cluster, through Yarn. Serialization is set to use Kryo. I have a large object which I send to the executors as a Broadcast. The object seems to serialize just fine. When it attempts to deserialize, though, Kryo throws a ClassNotFoundException... fo