Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?
On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwa...@mapr.com wrote:
Hi Anup,
From the stack trace which you have shared it looks like you are trying to use
Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.
Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.
[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51
[2]: https://github.com/apache/drill/pull/1111
Thanks,
Sorabh
________________________________
From: Anup Tiwari <anup.tiw...@games24x7.com>
Sent: Monday, February 12, 2018 9:21 AM
To: user@drill.apache.org
Subject: Re: Unable to setup hive plugin in Drill 1.11.0
I don't know where to look for logs as i have tried multiple time creating
hive plugin. But one time i can see below stack is printed in drillbit.log(i
don't know if it came due to plugin creation or not as i have tried multiple
time but nothing got printed in drillbit.log except below which is present in
logs only once) :-
Error
org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up
new storage plugin configuration for config org.apache.drill.exec.
store.hive.HiveStoragePluginConfig@f149f108 at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)
[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)
[drill-java-exec-1.11.0.jar:1.11.0] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72] at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java
:81) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher
Provider.java:195) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104
) [jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
[jersey-common-2.8.jar:na] at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)
[jersey-server-2.8.jar:na] at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345)
[jersey-container-servlet-core-2.8.jar:na] at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220)
[jersey-container-servlet-core-2.8.jar:na] at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.<init>(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 45 common frames omitted
On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiw...@games24x7.com wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{ "type" : "hive", "enabled" : true, "configProps" : {
"hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
"hive.metastore.sasl.enabled" : "false", "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000" } }
Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari
Sent with Mixmax
Regards,
Anup Tiwari
Regards,
Anup Tiwari
