[
https://issues.apache.org/jira/browse/HIVE-25020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luca Toscano updated HIVE-25020:
--------------------------------
Description:
Hi everybody,
I am running Apache Bigtop 1.5 on Debian, and I am currently in the process of
migrating everything from Debian 9 (stretch) to 10 (buster). Part of the
upgrade involves the libmysql-dev vs libmariadb-dev packages, since the former
is not provided on Buster (hence the only jdbc driver available is
org.mariadb.jdbc.Driver).
Bigtop 1.5 ships with hive 2.3.6, Buster ships with libmariadb-java 2.3.0.
When I upgraded the hive-site.xml file with
javax.jdo.option.ConnectionDriverName=org.mariadb.jdbc.Driver I noticed some
issues on the Metastore side, for example
{code}
2021-04-13T06:46:15,564 ERROR [pool-11-thread-144]
metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter:
value = MANAGED_TABLE for column "" : Could not set parameter at position 1
(values was 'MANAGED_TABLE')
Query - conn:30368(M) - "SELECT DISTINCT
'org.apache.hadoop.hive.metastore.model.MTable' AS
NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID
FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE
LOWER(B0.`NAME`) LIKE 'default' ESCAPE '\\' AND ((((A0.TBL_TYPE = ?) OR
(A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?))"
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableMeta(ObjectStore.java:1265)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
at com.sun.proxy.$Proxy25.getTableMeta(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_meta(HiveMetaStore.java:2006)
at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy26.get_table_meta(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11429)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11413)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:594)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:589)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:589)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
The above was originated from Hue while trying to fetch metadata about tables
into a database. I tried to deploy libmysql-dev (stretch package deployed on
buster), reverting javax.jdo.option.ConnectionDriverName to
com.mysql.jdbc.Driver and all the issues disappeared. This is a workaround that
can work as interim solution, but I'd like to know if what I encountered above
is expected or if there is a suggested way to fix it (that could also be
included in Hive 3.x, it will be packaged in the next Bigtop version so it
would be perfectly acceptable for me).
was:
Hi everybody,
I am running Apache Bigtop 1.5 on Debian, and I am currently in the process of
migrating everything from Debian 9 (stretch) to 10 (buster). Part of the
upgrade involves the libmysql-dev vs libmariadb-dev packages, since the former
is not provided on Buster (hence the only jdbc driver available is
org.mariadb.jdbc.Driver).
Bigtop 1.5 ships with hive 2.3.6, Buster ships with libmariadb-java 2.3.0.
When I upgraded the hive-site.xml file with
javax.jdo.option.ConnectionDriverName=org.mariadb.jdbc.Driver I noticed some
issues on the Metastore side, for example
```
2021-04-13T06:46:15,564 ERROR [pool-11-thread-144]
metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter:
value = MANAGED_TABLE for column "" : Could not set parameter at position 1
(values was 'MANAGED_TABLE')
Query - conn:30368(M) - "SELECT DISTINCT
'org.apache.hadoop.hive.metastore.model.MTable' AS
NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID
FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE
LOWER(B0.`NAME`) LIKE 'default' ESCAPE '\\' AND ((((A0.TBL_TYPE = ?) OR
(A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?))"
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableMeta(ObjectStore.java:1265)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
at com.sun.proxy.$Proxy25.getTableMeta(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_meta(HiveMetaStore.java:2006)
at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy26.get_table_meta(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11429)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11413)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:594)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:589)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:589)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
The above was originated from Hue while trying to fetch metadata about tables
into a database. I tried to deploy libmysql-dev (stretch package deployed on
buster), reverting javax.jdo.option.ConnectionDriverName to
com.mysql.jdbc.Driver and all the issues disappeared. This is a workaround that
can work as interim solution, but I'd like to know if what I encountered above
is expected or if there is a suggested way to fix it (that could also be
included in Hive 3.x, it will be packaged in the next Bigtop version so it
would be perfectly acceptable for me).
> Prepared statement related exceptions from Metastore when using
> org.mariadb.jdbc.Driver
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-25020
> URL: https://issues.apache.org/jira/browse/HIVE-25020
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 2.3.6
> Reporter: Luca Toscano
> Priority: Major
>
> Hi everybody,
> I am running Apache Bigtop 1.5 on Debian, and I am currently in the process
> of migrating everything from Debian 9 (stretch) to 10 (buster). Part of the
> upgrade involves the libmysql-dev vs libmariadb-dev packages, since the
> former is not provided on Buster (hence the only jdbc driver available is
> org.mariadb.jdbc.Driver).
> Bigtop 1.5 ships with hive 2.3.6, Buster ships with libmariadb-java 2.3.0.
> When I upgraded the hive-site.xml file with
> javax.jdo.option.ConnectionDriverName=org.mariadb.jdbc.Driver I noticed some
> issues on the Metastore side, for example
> {code}
> 2021-04-13T06:46:15,564 ERROR [pool-11-thread-144]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter:
> value = MANAGED_TABLE for column "" : Could not set parameter at position 1
> (values was 'MANAGED_TABLE')
> Query - conn:30368(M) - "SELECT DISTINCT
> 'org.apache.hadoop.hive.metastore.model.MTable' AS
> NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID
> FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE
> LOWER(B0.`NAME`) LIKE 'default' ESCAPE '\\' AND ((((A0.TBL_TYPE = ?) OR
> (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?))"
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
> at
> org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getTableMeta(ObjectStore.java:1265)
> at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> at com.sun.proxy.$Proxy25.getTableMeta(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_meta(HiveMetaStore.java:2006)
> at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy26.get_table_meta(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11429)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11413)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:594)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:589)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:589)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The above was originated from Hue while trying to fetch metadata about tables
> into a database. I tried to deploy libmysql-dev (stretch package deployed on
> buster), reverting javax.jdo.option.ConnectionDriverName to
> com.mysql.jdbc.Driver and all the issues disappeared. This is a workaround
> that can work as interim solution, but I'd like to know if what I encountered
> above is expected or if there is a suggested way to fix it (that could also
> be included in Hive 3.x, it will be packaged in the next Bigtop version so it
> would be perfectly acceptable for me).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
