Hi Suhas,

Without seeing the actual output of the stacktrace, I'd suspect that
spark-submit is doing some classpath magic that is covering some
dependencies you may not have included. Depending on your use case, it
might be easier to deal with this by just having maven output a pre-built
jar-with-dependencies for you, so you can just run off a single jar file.

Might want to check out the maven-assembly-plugin or maven-shade-plugin

http://stackoverflow.com/questions/574594/how-can-i-create-an-executable-jar-with-dependencies-using-maven

Thanks,
Natty

Jonathan "Natty" Natkins
StreamSets | Customer Engagement Engineer
mobile: 609.577.1600 | linkedin <http://www.linkedin.com/in/nattyice>


On Mon, Jan 19, 2015 at 10:03 AM, Su She <[email protected]> wrote:

> Hello All,
>
> I posted a question last week (
> http://mail-archives.apache.org/mod_mbox/kafka-users/201501.mbox/browser)
> but I'm not able to respond to the thread for some reason?
>
> The suggestion on the last post was that I am missing some jar files in my
> classpath, but I am using the following jar files:
>
> * kafka_2.10-0.8.2-beta.jar, kafka-clients-0.8.2-beta.jar,
> metrics-core-2.2.0.jar, scala-library-2.10.4.jar, slf4j-api-1.5.6.jar,
> slf4j-log4j12-1.5.6.jar*
>
> These are the jar files that I used when I ran the Kafka producer using the
> Spark submit script which looked like this:
>
>
> */opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/lib/spark/bin/spark-submit
>  --class SimpleApp --local[2]  --jars $(echo
> /home/ec2-user/sparkApps/lib/*.jar | tr ' ' ',')
> /home/ec2-user/target/simple-project-1.0.jar*
>
> Is there a reason I would need additional jar files now that i'm using the
> java -cp -jar command?
>
> Thanks!
>

Reply via email to