I'm also getting this - Ryan we both seem to be running into this issue with elasticsearch-hadoop :)
I tried spark.files.userClassPathFirst true on command line and that doesn;t work If I put it that line in spark/conf/spark-defaults it works but now I'm getting: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/InputFormat think I may need to add hadoop-client to my assembly, but any other ideas welcome. Ryan, will let you know how I get on On Mon, Aug 4, 2014 at 10:28 AM, Sean Owen <so...@cloudera.com> wrote: > I'm guessing you have the Jackson classes in your assembly but so does > Spark. Its classloader wins, and does not contain the class present in > your app's version of Jackson. Try spark.files.userClassPathFirst ? > > On Mon, Aug 4, 2014 at 6:28 AM, Ryan Braley <r...@traintracks.io> wrote: > > Hi Folks, > > > > I have an assembly jar that I am submitting using spark-submit script > on a > > cluster I created with the spark-ec2 script. I keep running into the > > java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass > > error on my workers even though jar tf clearly shows that class being a > part > > of my assembly jar. I have the spark program working locally. > > Here is the error log: > > https://gist.github.com/rbraley/cf5cd3457a89b1c0ac88 > > > > Anybody have any suggestions of things I can try? It seems > > > http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201406.mbox/%3c1403899110.65393.yahoomail...@web160503.mail.bf1.yahoo.com%3E > > that this is a similar error. I am open to recompiling spark to fix this, > > but I would like to run my job on my cluster rather than just locally. > > > > Thanks, > > Ryan > > > > > > Ryan Braley | Founder > > http://traintracks.io/ > > > > US: +1 (206) 866 5661 > > CN: +86 156 1153 7598 > > Coding the future. Decoding the game. > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
