[ https://issues.apache.org/jira/browse/SPARK-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-6622: ----------------------------- Component/s: (was: Spark Submit) SQL > Spark SQL cannot communicate with Hive meta store > ------------------------------------------------- > > Key: SPARK-6622 > URL: https://issues.apache.org/jira/browse/SPARK-6622 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.3.0 > Reporter: Deepak Kumar V > Labels: Hive > Attachments: exception.txt > > > I have multiple tables (among them is dw_bid) that are created through Apache > Hive. I have data in avro on HDFS that i want to join with dw_bid table, > this join needs to be done using Spark SQL. > Spark SQL is unable to communicate with Apache Hive Meta store and fails with > exception > org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test > connection to the given database. JDBC url = > jdbc:mysql://hostname.vip.company.com:3306/HDB, username = hiveuser. > Terminating connection pool (set lazyInit to true if you expect to start your > database after your app). Original Exception: ------ > java.sql.SQLException: No suitable driver found for > jdbc:mysql://hostname.vip. company.com:3306/HDB > at java.sql.DriverManager.getConnection(DriverManager.java:596) > Spark Submit Command > ./bin/spark-submit -v --master yarn-cluster --driver-class-path > /apache/hadoop/share/hadoop/common/hadoop-common-2.4.1-EBAY-2.jar:/apache/hadoop-2.4.1-2.1.3.0-2-EBAY/share/hadoop/yarn/lib/guava-11.0.2.jar > --jars > /apache/hadoop/lib/hadoop-lzo-0.6.0.jar,/home/dvasthimal/spark1.3/mysql-connector-java-5.1.35-bin.jar,/home/dvasthimal/spark1.3/spark-avro_2.10-1.0.0.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar,/home/dvasthimal/spark1.3/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar,$SPARK_HOME/conf/hive-site.xml > --num-executors 1 --driver-memory 4g --driver-java-options > "-XX:MaxPermSize=2G" --executor-memory 2g --executor-cores 1 --queue > hdmi-express --class com.ebay.ep.poc.spark.reporting.SparkApp > spark_reporting-1.0-SNAPSHOT.jar startDate=2015-02-16 endDate=2015-02-16 > input=/user/dvasthimal/epdatasets/successdetail1/part-r-00000.avro > subcommand=successevents2 output=/user/dvasthimal/epdatasets/successdetail2 > MySQL Java Conector Versions tried > mysql-connector-java-5.0.8-bin.jar (Picked from Apache Hive installation lib > folder) > mysql-connector-java-5.1.34.jar > mysql-connector-java-5.1.35.jar > Spark Version: 1.3.0 - Prebuilt for Hadoop 2.4.x > (http://d3kbcqa49mib13.cloudfront.net/spark-1.3.0-bin-hadoop2.4.tgz) > $ hive --version > Hive 0.13.0.2.1.3.6-2 > Subversion > git://ip-10-0-0-90.ec2.internal/grid/0/jenkins/workspace/BIGTOP-HDP_RPM_REPO-HDP-2.1.3.6-centos6/bigtop/build/hive/rpm/BUILD/hive-0.13.0.2.1.3.6 > -r 87da9430050fb9cc429d79d95626d26ea382b96c -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org