Can you try the patch from:
[SPARK-6913][SQL] Fixed "java.sql.SQLException: No suitable driver found"
Cheers
On Sat, Mar 28, 2015 at 12:41 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com> wrote:
> This is from my Hive installation
>
> -sh-4.1$ ls /apache/hive/lib | grep derby
>
> derby-10.10.1.1.jar
>
> derbyclient-10.10.1.1.jar
>
> derbynet-10.10.1.1.jar
>
>
> -sh-4.1$ ls /apache/hive/lib | grep datanucleus
>
> datanucleus-api-jdo-3.2.6.jar
>
> datanucleus-core-3.2.10.jar
>
> datanucleus-rdbms-3.2.9.jar
>
>
> -sh-4.1$ ls /apache/hive/lib | grep mysql
>
> mysql-connector-java-5.0.8-bin.jar
>
> -sh-4.1$
>
>
> $ hive --version
>
> Hive 0.13.0.2.1.3.6-2
>
> Subversion
> git://ip-10-0-0-90.ec2.internal/grid/0/jenkins/workspace/BIGTOP-HDP_RPM_REPO-HDP-2.1.3.6-centos6/bigtop/build/hive/rpm/BUILD/hive-0.13.0.2.1.3.6
> -r 87da9430050fb9cc429d79d95626d26ea382b96c
>
>
> $
>
>
>
> On Sat, Mar 28, 2015 at 1:05 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com>
> wrote:
>
>> I tried with a different version of driver but same error
>>
>> ./bin/spark-submit -v --master yarn-cluster --driver-class-path
>> /apache/hadoop/share/hadoop/common/hadoop-common-2.4.1-EBAY-2.jar:/apache/hadoop/lib/hadoop-lzo-0.6.0.jar:/apache/hadoop-2.4.1-2.1.3.0-2-EBAY/share/hadoop/yarn/lib/guava-11.0.2.jar
>> --jars
>> /home/dvasthimal/spark1.3/spark-avro_2.10-1.0.0.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar,
>> */home/dvasthimal/spark1.3/mysql-connector-java-5.0.8-bin.jar* --files
>> $SPARK_HOME/conf/hive-site.xml --num-executors 1 --driver-memory 4g
>> --driver-java-options "-XX:MaxPermSize=2G" --executor-memory 2g
>> --executor-cores 1 --queue hdmi-express --class
>> com.ebay.ep.poc.spark.reporting.SparkApp spark_reporting-1.0-SNAPSHOT.jar
>> startDate=2015-02-16 endDate=2015-02-16
>> input=/user/dvasthimal/epdatasets/successdetail1/part-r-00000.avro
>> subcommand=successevents2 output=/user/dvasthimal/epdatasets/successdetail2
>>
>> On Sat, Mar 28, 2015 at 12:47 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com>
>> wrote:
>>
>>> This is what am seeing
>>>
>>>
>>>
>>> ./bin/spark-submit -v --master yarn-cluster --driver-class-path
>>> /apache/hadoop/share/hadoop/common/hadoop-common-2.4.1-EBAY-2.jar:/apache/hadoop/lib/hadoop-lzo-0.6.0.jar:/apache/hadoop-2.4.1-2.1.3.0-2-EBAY/share/hadoop/yarn/lib/guava-11.0.2.jar
>>> --jars
>>> /home/dvasthimal/spark1.3/spark-avro_2.10-1.0.0.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar
>>> --files $SPARK_HOME/conf/hive-site.xml --num-executors 1 --driver-memory
>>> 4g --driver-java-options "-XX:MaxPermSize=2G" --executor-memory 2g
>>> --executor-cores 1 --queue hdmi-express --class
>>> com.ebay.ep.poc.spark.reporting.SparkApp spark_reporting-1.0-SNAPSHOT.jar
>>> startDate=2015-02-16 endDate=2015-02-16
>>> input=/user/dvasthimal/epdatasets/successdetail1/part-r-00000.avro
>>> subcommand=successevents2 output=/user/dvasthimal/epdatasets/successdetail2
>>>
>>>
>>> Caused by:
>>> org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException:
>>> The specified datastore driver ("com.mysql.jdbc.Driver") was not found in
>>> the CLASSPATH. Please check your CLASSPATH specification, and the name of
>>> the driver.
>>>
>>>
>>>
>>>
>>> ./bin/spark-submit -v --master yarn-cluster --driver-class-path
>>> /apache/hadoop/share/hadoop/common/hadoop-common-2.4.1-EBAY-2.jar:/apache/hadoop/lib/hadoop-lzo-0.6.0.jar:/apache/hadoop-2.4.1-2.1.3.0-2-EBAY/share/hadoop/yarn/lib/guava-11.0.2.jar
>>> --jars
>>> /home/dvasthimal/spark1.3/spark-avro_2.10-1.0.0.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar,*/home/dvasthimal/spark1.3/mysql-connector-java-5.1.34.jar
>>> *--files $SPARK_HOME/conf/hive-site.xml --num-executors 1
>>> --driver-memory 4g --driver-java-options "-XX:MaxPermSize=2G"
>>> --executor-memory 2g --executor-cores 1 --queue hdmi-express --class
>>> com.ebay.ep.poc.spark.reporting.SparkApp spark_reporting-1.0-SNAPSHOT.jar
>>> startDate=2015-02-16 endDate=2015-02-16
>>> input=/user/dvasthimal/epdatasets/successdetail1/part-r-00000.avro
>>> subcommand=successevents2 output=/user/dvasthimal/epdatasets/successdetail2
>>>
>>>
>>> Caused by: java.sql.SQLException: No suitable driver found for
>>> jdbc:mysql://db_host_name.vip.ebay.com:3306/HDB
>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>
>>>
>>> Looks like the driver jar that i got in is not correct,
>>>
>>> On Sat, Mar 28, 2015 at 12:34 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com>
>>> wrote:
>>>
>>>> Could someone please share the spark-submit command that shows their
>>>> mysql jar containing driver class used to connect to Hive MySQL meta store.
>>>>
>>>> Even after including it through
>>>>
>>>> --driver-class-path
>>>> /home/dvasthimal/spark1.3/mysql-connector-java-5.1.34.jar
>>>> OR (AND)
>>>> --jars /home/dvasthimal/spark1.3/mysql-connector-java-5.1.34.jar
>>>>
>>>> I keep getting "Suitable driver not found for
>>>>
>>>>
>>>> Command
>>>> ========
>>>>
>>>> ./bin/spark-submit -v --master yarn-cluster --driver-class-path
>>>> */home/dvasthimal/spark1.3/mysql-connector-java-5.1.34.jar*:/apache/hadoop/share/hadoop/common/hadoop-common-2.4.1-EBAY-2.jar:/apache/hadoop/lib/hadoop-lzo-0.6.0.jar:/apache/hadoop-2.4.1-2.1.3.0-2-EBAY/share/hadoop/yarn/lib/guava-11.0.2.jar
>>>> --jars
>>>> /home/dvasthimal/spark1.3/spark-avro_2.10-1.0.0.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar,
>>>> */home/dvasthimal/spark1.3/mysql-connector-java-5.1.34.ja*r --files
>>>> $SPARK_HOME/conf/hive-site.xml --num-executors 1 --driver-memory 4g
>>>> --driver-java-options "-XX:MaxPermSize=2G" --executor-memory 2g
>>>> --executor-cores 1 --queue hdmi-express --class
>>>> com.ebay.ep.poc.spark.reporting.SparkApp spark_reporting-1.0-SNAPSHOT.jar
>>>> startDate=2015-02-16 endDate=2015-02-16
>>>> input=/user/dvasthimal/epdatasets/successdetail1/part-r-00000.avro
>>>> subcommand=successevents2 output=/user/dvasthimal/epdatasets/successdetail2
>>>> Logs
>>>> ====
>>>>
>>>> Caused by: java.sql.SQLException: No suitable driver found for
>>>> jdbc:mysql://hostname:3306/HDB
>>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>> at java.sql.DriverManager.getConnection(DriverManager.java:187)
>>>> at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
>>>> at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
>>>> ... 68 more
>>>> ...
>>>> ...
>>>>
>>>> 15/03/27 23:56:08 INFO yarn.Client: Uploading resource
>>>> file:/home/dvasthimal/spark1.3/mysql-connector-java-5.1.34.jar -> hdfs://
>>>> apollo-phx-nn.vip.ebay.com:8020/user/dvasthimal/.sparkStaging/application_1426715280024_119815/mysql-connector-java-5.1.34.jar
>>>>
>>>> ...
>>>>
>>>> ...
>>>>
>>>>
>>>>
>>>> -sh-4.1$ jar -tvf ../mysql-connector-java-5.1.34.jar | grep Driver
>>>> 61 Fri Oct 17 08:05:36 GMT-07:00 2014
>>>> META-INF/services/java.sql.Driver
>>>> 3396 Fri Oct 17 08:05:22 GMT-07:00 2014
>>>> com/mysql/fabric/jdbc/FabricMySQLDriver.class
>>>> * 692 Fri Oct 17 08:05:22 GMT-07:00 2014 com/mysql/jdbc/Driver.class*
>>>> 1562 Fri Oct 17 08:05:20 GMT-07:00 2014
>>>> com/mysql/jdbc/NonRegisteringDriver$ConnectionPhantomReference.class
>>>> 17817 Fri Oct 17 08:05:20 GMT-07:00 2014
>>>> com/mysql/jdbc/NonRegisteringDriver.class
>>>> 690 Fri Oct 17 08:05:24 GMT-07:00 2014
>>>> com/mysql/jdbc/NonRegisteringReplicationDriver.class
>>>> 731 Fri Oct 17 08:05:24 GMT-07:00 2014
>>>> com/mysql/jdbc/ReplicationDriver.class
>>>> 336 Fri Oct 17 08:05:24 GMT-07:00 2014 org/gjt/mm/mysql/Driver.class
>>>> You have new mail in /var/spool/mail/dvasthimal
>>>> -sh-4.1$ cat conf/hive-site.xml | grep Driver
>>>> <name>javax.jdo.option.ConnectionDriverName</name>
>>>> * <value>com.mysql.jdbc.Driver</value>*
>>>> <description>Driver class name for a JDBC metastore</description>
>>>> -sh-4.1$
>>>>
>>>> --
>>>> Deepak
>>>>
>>>>
>>>
>>>
>>> --
>>> Deepak
>>>
>>>
>>
>>
>> --
>> Deepak
>>
>>
>
>
> --
> Deepak
>
>
