[
https://issues.apache.org/jira/browse/FLINK-27999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Anderson updated FLINK-27999:
-----------------------------------
Fix Version/s: 1.15.1
(was: 1.15.2)
> NoSuchMethodError when using Hive 3 dialect
> -------------------------------------------
>
> Key: FLINK-27999
> URL: https://issues.apache.org/jira/browse/FLINK-27999
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.15.0
> Reporter: luoyuxia
> Assignee: luoyuxia
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.16.0, 1.15.1
>
>
> VirtualColumn.VIRTUAL_COLUMN_NAMES uses different types in Hive2 / Hive3,
> causing NoSuchMethodErrors when the Hive dialect is used.
> When use 'mvn test -PHive3.1.1' Hive connector, it will throw the following
> exception "Caused by: MetaException(message:Required table missing : "DBS" in
> Catalog "" Schema "". DataNucleus requires this table to perform its
> persistence operations. Either your MetaData is incorrect, or you need to
> enable "datanucleus.schema.autoCreateTables")".
> From the error message, we can know the reason is the configuration
> "datanucleus.schema.autoCreateTables" is not true. But when create
> HiveCatalog, we do really set the configuration
> "datanucleus.schema.autoCreateTables" to true in the hive-site.xml.
> After some debuging, I found the reason is that some test modify a static
> variable, and then boil the other test.
> When running HiveCatalogFactoryTest, it will call
> HiveCatalog#createHiveConf(@Nullable String hiveConfDir, @Nullable String
> hadoopConfDir) to create a HiveConf for HiveCatalog. In this method, it will
> set the static variable "hiveSiteURL" to null.
> Then, if we run "HiveCatalogHiveMetadataTest", it will call
> HiveTestUtils#createHiveConf() to create HiveConf. The following code will
> create a empty HiveConf;
>
> {code:java}
> HiveConf hiveConf = new HiveConf(); {code}
> But in the initialize function of HiveConf, it first will apply all default
> variables, so "datanucleus.schema.autoCreateTables" will be set to false. And
> the "hiveSiteURL" is null, so skip add the resource. Then, it will check
> ""hive.metastore.schema.verification" is true or false. If it's true, it will
> set "datanucleus.schema.autoCreateTables" to false. This have a higher
> priority and thus overwrite the value we configure in hive-site.xml.
> To fix it, we only need to reset the static variable "hiveSiteURL" with our
> hive-site.xml.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
