So which one will be used during query execution ? What we observed that no matter what queries are getting executed on Hive with derby .
Also there is no dialect defined for HSQLDB , do you think this will work ? On Tue, Nov 10, 2015 at 5:07 AM, Michael Armbrust <mich...@databricks.com> wrote: > We have two copies of hive running in order to support multiple versions > of hive with a single version of Spark. You are see log messages for the > version that we use for execution (it just creates a temporary derby > metastore). > > On Mon, Nov 9, 2015 at 3:32 PM, mayurladwa <mayur.la...@blackrock.com> > wrote: > >> Hello, this question is around the hive thrift server that comes packaged >> with spark 1.5.1, I am trying to change the default metastore from derby. >> >> From googling I see the more commonly documented alternatives to derby are >> MySQL, but as it exposes a JDBC interface I want to try and get this >> working >> with HSQL (2.0). >> >> I'm overriding the following in the hive-site.xml: >> >> /<property> >> <name>javax.jdo.option.ConnectionURL</name> >> <value>jdbc:hsqldb:hsql://myhost:58090/default</value> >> <description>JDBC connect string for a JDBC metastore</description> >> </property> >> >> <property> >> <name>javax.jdo.option.ConnectionDriverName</name> >> <value>org.hsqldb.jdbc.JDBCDriver</value> >> <description>Driver class name for a JDBC metastore</description> >> </property> >> >> <property> >> <name>javax.jdo.option.ConnectionUserName</name> >> <value>user</value> >> <description>username to use against metastore database</description> >> </property> >> >> <property> >> <name>javax.jdo.option.ConnectionPassword</name> >> <value>pwd</value> >> <description>password to use against metastore database</description> >> </property> >> / >> >> What's really strange is that I see some hive tables created in my HSQL >> database when my spark hive thrift server is running, but when I do a >> query >> I see it switches back to derby! I get logs like this: >> >> /15/11/09 11:59:16 DEBUG ObjectStore: *Overriding >> javax.jdo.option.ConnectionURL value null from jpox.properties with >> jdbc:hsqldb:hsql://myhost:58090/default*/ >> >> And then later I see this: >> >> /15/11/09 11:59:18 DEBUG ObjectStore: *Overriding >> javax.jdo.option.ConnectionURL value null from jpox.properties with >> >> jdbc:derby:;databaseName=/tmp/spark-acb48194-09a7-4beb-b5fc-*ffc0216449c8/metastore;create=true/ >> ... >> /15/11/09 11:59:18 DEBUG Transaction: Transaction committed in 1 ms >> 15/11/09 11:59:18 INFO MetaStoreDirectSql: Using direct SQL, *underlying >> DB >> is DERBY* >> 15/11/09 11:59:18 DEBUG ObjectStore: RawStore: >> org.apache.hadoop.hive.metastore.ObjectStore@7ecab68e, with >> PersistenceManager: org.datanucleus.api.jdo.JDOPersistenceManager@61e68dab >> created in the thread with id: 1 >> 15/11/09 11:59:18 INFO ObjectStore: Initialized ObjectStore/ >> >> So not totally sure how this is getting switched back to derby, or why it >> thinks later on that the jpox.properties I am overriding in the >> hive-site.xml area is suddenly null? >> >> Any help would be much appreciated :) >> >> Many thanks >> >> Mayur >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Overriding-Derby-in-hive-site-xml-giving-strange-results-tp25333.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
