Hi Asim,
You may give it a shot by adding this uber jar to Drill 3rd party directory (Remove previously copied jars). For truststore, try giving absolute path. The test was to validate if hive uber jar works with your Hive setup. Thanks, Arjun ________________________________ From: Asim Kanungo <asim...@gmail.com> Sent: Tuesday, March 13, 2018 10:48 AM To: user@drill.apache.org Subject: Re: hive connection as generic jdbc Hi Arjun, I have tried with the hive jdbc uber jar and able to make a successful connection. java -cp "hive-jdbc-uber-2.6.3.0-235.jar:sqlline-1.1.9-drill-r7.jar:jline-2.10.jar" sqlline.SqlLine -d org.apache.hive.jdbc.HiveDriver -u "jdbc:hive2://knox server name:port/default;ssl=true;sslTrustStore=location and filename of jks file;trustStorePassword=********;transportMode=http;httpPath=path " -n username-p ******** -e "show tables;" As we have SSL enabled system so I have to give the extra details in the URL and it worked. Does that mean it should work for adding it as a generic JDBC.How is this test related to my issue ? Thanks Asim On Mon, Mar 12, 2018 at 10:36 PM, Arjun kr <arjun...@outlook.com> wrote: > Hi Asim, > > > You may try using hive uber jar in case you have not tried it. See if > below link helps. > > > https://github.com/timveil/hive-jdbc-uber-jar/releases [https://avatars0.githubusercontent.com/u/3260845?s=400&v=4]<https://github.com/timveil/hive-jdbc-uber-jar/releases> Releases · timveil/hive-jdbc-uber-jar · GitHub<https://github.com/timveil/hive-jdbc-uber-jar/releases> github.com hive-jdbc-uber-jar - Hive JDBC "uber" or "standalone" jar based on the latest Hortonworks Data Platform (HDP) > > > It would be ideal to test this uber jar with a sample JDBC application > before trying with Drill. > > > java -cp "hive-jdbc-uber-2.6.3.0-235.jar:$DRILL_HOME/jars/3rdparty/ > sqlline-1.1.9-drill-r7.jar:$DRILL_HOME/jars/3rdparty/jline-2.10.jar" > sqlline.SqlLine -d org.apache.hive.jdbc.HiveDriver -u "<JDBC URL>" -n > <user> -p <password> -e "show tables;" > > > Thanks, > > > Arjun > > > > ________________________________ > From: Asim Kanungo <asim...@gmail.com> > Sent: Tuesday, March 13, 2018 8:37 AM > To: user@drill.apache.org > Subject: Re: hive connection as generic jdbc > > Thanks Kunal... > > Here are the details. > { > "type": "jdbc", > "driver": "org.apache.hive.jdbc.HiveDriver", > "url": "jdbc:hive2://knox > address:port/default?ssl=true&transportMode=http&httpPath= > pathdetail&sslTrustStore=mytruststore.jks&trustStorePassword=******", > "username": "XXXXXXX", > "password": "**********", > "enabled": true > } > > Please note that we have a SSL enabled system, so have used the Truststore > settings. > Please let me know if you need any more details. > > Thanks > Asim > > > On Sun, Mar 11, 2018 at 11:55 PM, Kunal Khatua <ku...@apache.org> wrote: > > > I'm not sure how this is resolved since Hive is directly accessed by > Drill > > using the Hive storage plugin, instead of via the JDBC storage plugin. > > Perhaps you can share the parameters of the JDBC storage plugin you used, > > so that folks more familiar with the JDBC storage plugin can help. > > I'll see what I can find out in the meanwhile. > > ~ Kunal > > > > On Sat, Mar 10, 2018 at 7:23 PM, Asim Kanungo <asim...@gmail.com> wrote: > > > > > Hi Kunal, > > > > > > I have tried the steps and getting the below error:- > > > > > > 2018-03-08 22:39:59,234 [qtp433826182-75] ERROR > > > o.a.d.e.server.rest.StorageResources - Unable to create/ update > plugin: > > > test > > > org.apache.drill.common.exceptions.ExecutionSetupException: Failure > > > setting > > > up new storage plugin configuration for config > > > org.apache.drill.exec.store.jdbc.JdbcStorageConfig@8ef5d26f > > > at > > > org.apache.drill.exec.store.StoragePluginRegistryImpl.create( > > > StoragePluginRegistryImpl.java:355) > > > ~[drill-java-exec-1.12.0.jar:1.12.0] > > > at > > > org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate( > > > StoragePluginRegistryImpl.java:239) > > > ~[drill-java-exec-1.12.0.jar:1.12.0] > > > at > > > org.apache.drill.exec.server.rest.PluginConfigWrapper. > > > createOrUpdateInStorage(PluginConfigWrapper.java:57) > > > ~[drill-java-exec-1.12.0.jar:1.12.0] > > > at > > > org.apache.drill.exec.server.rest.StorageResources. > > > createOrUpdatePluginJSON(StorageResources.java:162) > > > [drill-java-exec-1.12.0.jar:1.12.0] > > > at > > > org.apache.drill.exec.server.rest.StorageResources. > createOrUpdatePlugin( > > > StorageResources.java:177) > > > [drill-java-exec-1.12.0.jar:1.12.0] > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > ~[na:1.8.0_73] > > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke( > > NativeMethodAccessorImpl.java: > > > 62) > > > ~[na:1.8.0_73] > > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke( > > > DelegatingMethodAccessorImpl.java:43) > > > ~[na:1.8.0_73] > > > at java.lang.reflect.Method.invoke(Method.java:497) > > ~[na:1.8.0_73] > > > at > > > org.glassfish.jersey.server.model.internal. > > ResourceMethodInvocationHandle > > > rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.internal. > > AbstractJavaResourceMethodDisp > > > atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.internal. > > AbstractJavaResourceMethodDisp > > > atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.internal. > > JavaResourceMethodDispatcherPr > > > ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr > > > ovider.java:195) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.internal. > > AbstractJavaResourceMethodDisp > > > atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.ResourceMethodInvoker. > > > invoke(ResourceMethodInvoker.java:387) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.ResourceMethodInvoker. > > > apply(ResourceMethodInvoker.java:331) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.model.ResourceMethodInvoker. > > > apply(ResourceMethodInvoker.java:103) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.ServerRuntime$1.run( > ServerRuntime.java:269) > > > [jersey-server-2.8.jar:na] > > > at org.glassfish.jersey.internal. > Errors$1.call(Errors.java:271) > > > [jersey-common-2.8.jar:na] > > > at org.glassfish.jersey.internal. > Errors$1.call(Errors.java:267) > > > [jersey-common-2.8.jar:na] > > > at org.glassfish.jersey.internal.Errors.process(Errors.java: > 315) > > > [jersey-common-2.8.jar:na] > > > at org.glassfish.jersey.internal.Errors.process(Errors.java: > 297) > > > [jersey-common-2.8.jar:na] > > > at org.glassfish.jersey.internal.Errors.process(Errors.java: > 267) > > > [jersey-common-2.8.jar:na] > > > at > > > org.glassfish.jersey.process.internal.RequestScope. > > > runInScope(RequestScope.java:297) > > > [jersey-common-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.ServerRuntime.process( > > ServerRuntime.java:252) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.server.ApplicationHandler.handle( > > > ApplicationHandler.java:1023) > > > [jersey-server-2.8.jar:na] > > > at > > > org.glassfish.jersey.servlet.WebComponent.service( > WebComponent.java:372) > > > [jersey-container-servlet-core-2.8.jar:na] > > > at > > > org.glassfish.jersey.servlet.ServletContainer.service( > > > ServletContainer.java:382) > > > [jersey-container-servlet-core-2.8.jar:na] > > > at > > > org.glassfish.jersey.servlet.ServletContainer.service( > > > ServletContainer.java:345) > > > [jersey-container-servlet-core-2.8.jar:na] > > > at > > > org.glassfish.jersey.servlet.ServletContainer.service( > > > ServletContainer.java:220) > > > [jersey-container-servlet-core-2.8.jar:na] > > > at > > > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) > > > [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle( > > ServletHandler.java:551) > > > [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.session.SessionHandler. > > > doHandle(SessionHandler.java:219) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.handler.ContextHandler. > > > doHandle(ContextHandler.java:1111) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doScope( > > ServletHandler.java:478) > > > [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.session.SessionHandler. > > > doScope(SessionHandler.java:183) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.handler.ContextHandler. > > > doScope(ContextHandler.java:1045) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle( > > > ScopedHandler.java:141) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle( > > > HandlerWrapper.java:97) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at org.eclipse.jetty.server.Server.handle(Server.java:462) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.server.HttpConnection.onFillable( > > > HttpConnection.java:232) > > > [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.io.AbstractConnection$2.run( > > AbstractConnection.java:534) > > > [jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( > > > QueuedThreadPool.java:607) > > > [jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( > > > QueuedThreadPool.java:536) > > > [jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] > > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_73] > > > *Caused by: java.lang.NoSuchFieldError: INSTANCE* > > > at > > > org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>( > > > SSLConnectionSocketFactory.java:144) > > > ~[hive-jdbc.jar:1.2.1000.2.5.3.0-37] > > > at > > > org.apache.hive.jdbc.HiveConnection.getHttpClient( > > HiveConnection.java:365) > > > ~[hive-jdbc.jar:1.2.1000.2.5.3.0-37] > > > at > > > org.apache.hive.jdbc.HiveConnection.createHttpTransport( > > > HiveConnection.java:244) > > > ~[hive-jdbc.jar:1.2.1000.2.5.3.0-37] > > > at > > > org.apache.hive.jdbc.HiveConnection.openTransport( > > HiveConnection.java:191) > > > ~[hive-jdbc.jar:1.2.1000.2.5.3.0-37] > > > at > > > org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:155) > > > ~[hive-jdbc.jar:1.2.1000.2.5.3.0-37] > > > at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver. > java:105) > > > ~[hive-jdbc.jar:1.2.1000.2.5.3.0-37] > > > at > > > org.apache.commons.dbcp.DriverConnectionFactory.createConnection( > > > DriverConnectionFactory.java:38) > > > ~[commons-dbcp-1.4.jar:1.4] > > > at > > > org.apache.commons.dbcp.PoolableConnectionFactory.makeObject( > > > PoolableConnectionFactory.java:582) > > > ~[commons-dbcp-1.4.jar:1.4] > > > at > > > org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory( > > > BasicDataSource.java:1556) > > > ~[commons-dbcp-1.4.jar:1.4] > > > at > > > org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor > > > y(BasicDataSource.java:1545) > > > ~[commons-dbcp-1.4.jar:1.4] > > > at > > > org.apache.commons.dbcp.BasicDataSource.createDataSource( > > > BasicDataSource.java:1388) > > > ~[commons-dbcp-1.4.jar:1.4] > > > at > > > org.apache.commons.dbcp.BasicDataSource.getConnection( > > > BasicDataSource.java:1044) > > > ~[commons-dbcp-1.4.jar:1.4] > > > at > > > org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool. > > > get(JdbcUtils.java:73) > > > ~[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23] > > > at > > > org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect( > > > JdbcSchema.java:138) > > > ~[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23] > > > at > > > org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>( > > > JdbcStoragePlugin.java:103) > > > ~[drill-jdbc-storage-1.12.0.jar:1.12.0] > > > at sun.reflect.NativeConstructorAccessorImpl. > newInstance0(Native > > > Method) ~[na:1.8.0_73] > > > at > > > sun.reflect.NativeConstructorAccessorImpl.newInstance( > > > NativeConstructorAccessorImpl.java:62) > > > ~[na:1.8.0_73] > > > at > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > > > DelegatingConstructorAccessorImpl.java:45) > > > ~[na:1.8.0_73] > > > at java.lang.reflect.Constructor.newInstance(Constructor.java: > > 422) > > > ~[na:1.8.0_73] > > > at > > > org.apache.drill.exec.store.StoragePluginRegistryImpl.create( > > > StoragePluginRegistryImpl.java:346) > > > ~[drill-java-exec-1.12.0.jar:1.12.0] > > > ... 45 common frames omitted > > > > > > The JAR file for Hive and HTTP is added. and the > > > "drill.exec.sys.store.provider.local.path" is set in > > > the drill-override.conf file. After searching in google I found that > this > > > issue occurs when there is a version mismatch or if one class is > present > > in > > > two or more JAR files. I have not very much idea in Java, so can you > > please > > > let me know any particular JAR which has to be removed or added to > > resolve > > > this issue. > > > Thanks again for your time and help !!!! > > > > > > Thanks > > > Asim > > > > > > On Wed, Mar 7, 2018 at 10:24 PM, Asim Kanungo <asim...@gmail.com> > wrote: > > > > > > > Thanks Kunal for clarifying... > > > > I am still learning the things, so as per my first project I am > trying > > to > > > > create a successful connection. > > > > I will work on the optimization in the second phase. > > > > Thanks for your valuable tips, let me try to create the hive > connection > > > > through JDBC then. > > > > I suppose I need to put the hive jdbc drivers in the 3rd party > > directory, > > > > please let me know if you have the list of the drivers or jar I need > to > > > put > > > > in the 3rd party directory. > > > > > > > > Thanks > > > > Asim > > > > > > > > On Wed, Mar 7, 2018 at 6:06 PM, Kunal Khatua <kunalkha...@gmail.com> > > > > wrote: > > > > > > > >> You should be able to connect to a Hive cluster via JDBC. However, > the > > > >> benefit of using Drill co-located on the same cluster is that Drill > > can > > > >> directly access the data based on locality information from Hive and > > > >> process across the distributed FS cluster. > > > >> > > > >> With JDBC, any filters you have, will (most probably) not be pushed > > down > > > >> to Hive. So, you'll end up loading the unfiltered data through a > > single > > > >> data channel from the Hive cluster, into your Drill cluster, before > it > > > can > > > >> start processing. > > > >> > > > >> If using JDBC is the only option, it might be worth using the > 'create > > > >> table as' (or the temporary table variant) to offload that data into > > > your > > > >> Drill cluster and then execute your analytical queries against this > > > >> offloaded dataset. > > > >> > > > >> On 3/7/2018 2:46:55 PM, Asim Kanungo <asim...@gmail.com> wrote: > > > >> Hi Team, > > > >> > > > >> Can I connect to hive database as a generic jdbc protocol like I am > > > doing > > > >> for other RDBMS.Or DRILL can only connect to hive residing in the > same > > > >> cluster where Hadoop is installed. > > > >> > > > >> I am talking about the cases where DRILL is installed in one cluster > > and > > > >> my > > > >> Hadoop cluster is different. Can you please guide if I can connect > to > > > that > > > >> hive, using jdbc protocol. > > > >> I have tried, but it failed with the error "unable to add/update > > > storage". > > > >> I am not sure why it has failed as it is not giving any other > > messages, > > > >> and > > > >> I know version 13 will contain more message. > > > >> > > > >> Please advise. > > > >> > > > >> Thanks > > > >> Asim > > > >> > > > > > > > > > > > > > >
