Hi Suhas, Without seeing the actual output of the stacktrace, I'd suspect that spark-submit is doing some classpath magic that is covering some dependencies you may not have included. Depending on your use case, it might be easier to deal with this by just having maven output a pre-built jar-with-dependencies for you, so you can just run off a single jar file.
Might want to check out the maven-assembly-plugin or maven-shade-plugin http://stackoverflow.com/questions/574594/how-can-i-create-an-executable-jar-with-dependencies-using-maven Thanks, Natty Jonathan "Natty" Natkins StreamSets | Customer Engagement Engineer mobile: 609.577.1600 | linkedin <http://www.linkedin.com/in/nattyice> On Mon, Jan 19, 2015 at 10:03 AM, Su She <[email protected]> wrote: > Hello All, > > I posted a question last week ( > http://mail-archives.apache.org/mod_mbox/kafka-users/201501.mbox/browser) > but I'm not able to respond to the thread for some reason? > > The suggestion on the last post was that I am missing some jar files in my > classpath, but I am using the following jar files: > > * kafka_2.10-0.8.2-beta.jar, kafka-clients-0.8.2-beta.jar, > metrics-core-2.2.0.jar, scala-library-2.10.4.jar, slf4j-api-1.5.6.jar, > slf4j-log4j12-1.5.6.jar* > > These are the jar files that I used when I ran the Kafka producer using the > Spark submit script which looked like this: > > > */opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/lib/spark/bin/spark-submit > --class SimpleApp --local[2] --jars $(echo > /home/ec2-user/sparkApps/lib/*.jar | tr ' ' ',') > /home/ec2-user/target/simple-project-1.0.jar* > > Is there a reason I would need additional jar files now that i'm using the > java -cp -jar command? > > Thanks! >
