Re: java.sql.SQLException: No suitable driver found

2014-12-22 Thread Michael Orr 
Here is a script I use to submit a directory of jar files. It assumes jar files 
are in target/dependency or lib/

DRIVER_PATH=
DEPEND_PATH=
if [ -d lib ]; then
  DRIVER_PATH=lib
  DEPEND_PATH=lib
else
  DRIVER_PATH=target
  DEPEND_PATH=target/dependency
fi

DEPEND_JARS=log4j.properties
for f in `ls $DEPEND_PATH`; do DEPEND_JARS=$DEPEND_JARS,$DEPEND_PATH/$f; done

$SPARK_HOME/bin/spark-submit \
  --class $1 \
  --master yarn-client \
  --num-executors 1 \
  --driver-memory 4g \
  --executor-memory 16g \
  --executor-cores 4 \
  --jars $DEPEND_JARS \
  $DRIVER_PATH/core-ingest-*.jar ${*:2}


You would run it with a command like:

./run.sh class.to.submit arg1 arg2 …


 On Dec 22, 2014, at 1:11 AM, durga durgak...@gmail.com wrote:
 
 One more question.
 
 How would I submit additional jars to the spark-submit job. I used --jars
 option, it seems it is not working as explained earlier.
 
 Thanks for the help,
 
 -D
 
 
 
 --
 View this message in context: 
 http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20805.html
 Sent from the Apache Spark User List mailing list archive at Nabble.com.
 
 -
 To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
 For additional commands, e-mail: user-h...@spark.apache.org
 


-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: java.sql.SQLException: No suitable driver found

2014-12-21 Thread Michael Armbrust 
With JDBC you often need to load the class so it can register the driver at
the beginning of your program.  Usually this is something like:

Class.forName(com.mysql.jdbc.Driver);


On Fri, Dec 19, 2014 at 3:47 PM, durga durgak...@gmail.com wrote:

 Hi I am facing an issue with mysql jars with spark-submit.

 I am not running in yarn mode.

 spark-submit --jars $(echo mysql-connector-java-5.1.34-bin.jar | tr ' '
 ',')
 --class com.abc.bcd.GetDBSomething myjar.jar abc bcd

 Any help is really appreciated.

 Thanks,
 -D

 14/12/19 23:42:10 INFO SparkUI: Started SparkUI at
 http://ec2-54-90-85-197.compute-1.amazonaws.com:4040
 14/12/19 23:42:10 INFO SparkContext: Added JAR
 file:/root/abc/mysql-connector-java-5.1.34-bin.jar at
 http://10.61.187.176:57956/jars/mysql-connector-java-5.1.34-bin.jar with
 timestamp 1419032530456
 14/12/19 23:42:10 INFO SparkContext: Added JAR file:/root/abc/myjar.jar at
 http://10.61.187.176:57956/jars/filesplitter_2.10-1.0.jar with timestamp
 1419032530459
 14/12/19 23:42:10 INFO AppClient$ClientActor: Connecting to master
 spark://ec2-54-90-85-197.compute-1.amazonaws.com:7077...
 Exception in thread main java.sql.SQLException: No suitable driver found
 for jdbc:mysql://192.168.20.45:3306/abcdb?user=rootpassword=admin



 --
 View this message in context:
 http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792.html
 Sent from the Apache Spark User List mailing list archive at Nabble.com.

 -
 To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
 For additional commands, e-mail: user-h...@spark.apache.org

Re: java.sql.SQLException: No suitable driver found

2014-12-21 Thread durga 
Hi All,

I tried to make combined.jar in shell script . it is working when I am using
spark-shell. But for the spark-submit it is same issue.

Help is highly appreciated.

Thanks
-D



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20804.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: java.sql.SQLException: No suitable driver found

2014-12-21 Thread durga 
One more question.

How would I submit additional jars to the spark-submit job. I used --jars
option, it seems it is not working as explained earlier.

Thanks for the help,

-D



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20805.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org