Hi James,
I am doing something similar with the difference that everything runs on
docker [1].
I am using Hive 3.1 (HDP though) but things work fine at least with
in-memory derby.
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:memory:metastore;create=true</value>
</property>
Best,
Stamatis
[1] https://github.com/zabetak/hs2-embedded
On Wed, Feb 17, 2021 at 10:55 PM James Baiera <james.bai...@elastic.co>
wrote:
> Hey folks,
>
> I have a project where I test with Hive using an embedded HiveServer2
> instance within a JVM running integration tests. This has worked for Hive
> 1.2.2 in the past, and I've been able to get it to work with Hive 2.3.8,
> but have been having trouble getting it working on Hive 3.0+
>
> The error I keep running into is that the metastore tables are not present
> in the local embedded metastore. I have enabled both
> "hive.metastore.schema.verification" to be "false" and
> "datanucleus.schema.autoCreateAll" to be "true", but it seems like the
> latter setting is being ignored. Instead of starting up, the HiveServer2
> fails while trying to read from the DBS table:
>
> Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
> javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID"
> from "DBS"".
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> ~[datanucleus-api-jdo-4.2.4.jar:?]
> at
> org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
> ~[datanucleus-api-jdo-4.2.4.jar:?]
> at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:216)
> ~[datanucleus-api-jdo-4.2.4.jar:?]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.runTestQuery(MetaStoreDirectSql.java:276)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:184)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:498)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:420)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:375)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
> [hadoop-common-3.1.2.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
> [hadoop-common-3.1.2.jar:?]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:59)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:718)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:696)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:690)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:767)
> [hive-exec-3.1.2.jar:3.1.2]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:538)
> [hive-exec-3.1.2.jar:3.1.2]
> <....>
>
> Looking into the documentation, it seems that many things mention using
> the schematool to set up the metastore the first time, but this is an
> embedded use case, and there is no Hive installation locally to use for
> this.
>
> I've also tried using the HiveJDBC driver with "jdbc:hive2:///" as the url
> to run on an embedded server, and I am getting the same errors.
>
> Is this use case not supported anymore in Hive 3? Am I missing something
> here?
>
