Hive Query Timeout in hive-jdbc
Hi Team, I am trying to connect to hiveServer via hive-jdbc. Can we configure client side timeout at each query executed inside each jdbc connection. (When I looked at HiveStatement.setQueryTimeout method it says operation unsupported). Is there any other way of timing out and cancelling the connection and throwing Exception, if it alive for over a period of 4 mins or so (configurable at client side). PS : Queries that I am executing over jdbc are simple ddl statements. (hive external table create statements and drop table statements). Regards, Satya Harish.
Re: Hive Query Timeout in hive-jdbc
hive does not support timeout on the client side. and I think it is not recommended that if the client exit with timeout exception, the hiveserver side may also running the job. this will result in inconsistent state. On Tue, Feb 2, 2016 at 4:49 PM, Satya Harish Appana < satyaharish.app...@gmail.com> wrote: > Hi Team, > > I am trying to connect to hiveServer via hive-jdbc. > Can we configure client side timeout at each query executed inside each > jdbc connection. (When I looked at HiveStatement.setQueryTimeout method it > says operation unsupported). > Is there any other way of timing out and cancelling the connection and > throwing Exception, if it alive for over a period of 4 mins or so > (configurable at client side). > > PS : Queries that I am executing over jdbc are simple ddl statements. > (hive external table create statements and drop table statements). > > > Regards, > Satya Harish. >
Re: Hive Query Timeout in hive-jdbc
Actually, Hive doesn't support timeout, but Tez and MapReduce does. Therefore, you can set a timeout on these tools to kill failed queries. Hope this helps, Loïc Loïc CHANEL System & virtualization engineer TO - XaaS Ind - Worldline (Villeurbanne, France) 2016-02-02 11:10 GMT+01:00 董亚军: > hive does not support timeout on the client side. > > and I think it is not recommended that if the client exit with timeout > exception, the hiveserver side may also running the job. this will result > in inconsistent state. > > On Tue, Feb 2, 2016 at 4:49 PM, Satya Harish Appana < > satyaharish.app...@gmail.com> wrote: > >> Hi Team, >> >> I am trying to connect to hiveServer via hive-jdbc. >> Can we configure client side timeout at each query executed inside each >> jdbc connection. (When I looked at HiveStatement.setQueryTimeout method it >> says operation unsupported). >> Is there any other way of timing out and cancelling the connection and >> throwing Exception, if it alive for over a period of 4 mins or so >> (configurable at client side). >> >> PS : Queries that I am executing over jdbc are simple ddl statements. >> (hive external table create statements and drop table statements). >> >> >> Regards, >> Satya Harish. >> > >
Re: Hive Query Timeout in hive-jdbc
Queries I am running over Hive JDBC are ddl statements(none of the queries are select or insert. which will result in an execution engine(tez/mr) job to be launched.. all the queries are create external table .. and drop table .. and alter table add partitions). On Tue, Feb 2, 2016 at 3:54 PM, Loïc Chanelwrote: > Actually, Hive doesn't support timeout, but Tez and MapReduce does. > Therefore, you can set a timeout on these tools to kill failed queries. > Hope this helps, > > Loïc > > Loïc CHANEL > System & virtualization engineer > TO - XaaS Ind - Worldline (Villeurbanne, France) > > 2016-02-02 11:10 GMT+01:00 董亚军 : > >> hive does not support timeout on the client side. >> >> and I think it is not recommended that if the client exit with timeout >> exception, the hiveserver side may also running the job. this will result >> in inconsistent state. >> >> On Tue, Feb 2, 2016 at 4:49 PM, Satya Harish Appana < >> satyaharish.app...@gmail.com> wrote: >> >>> Hi Team, >>> >>> I am trying to connect to hiveServer via hive-jdbc. >>> Can we configure client side timeout at each query executed inside each >>> jdbc connection. (When I looked at HiveStatement.setQueryTimeout method it >>> says operation unsupported). >>> Is there any other way of timing out and cancelling the connection and >>> throwing Exception, if it alive for over a period of 4 mins or so >>> (configurable at client side). >>> >>> PS : Queries that I am executing over jdbc are simple ddl statements. >>> (hive external table create statements and drop table statements). >>> >>> >>> Regards, >>> Satya Harish. >>> >> >> > -- Regards, Satya Harish Appana, Software Development Engineer II, Flipkart,Bangalore, Ph:+91-9538797174.
Re: Hive Query Timeout in hive-jdbc
Then indeed Tez and MR timeout won't be any help, sorry. I would be very interested in your solution though. Regards, Loïc Loïc CHANEL System & virtualization engineer TO - XaaS Ind - Worldline (Villeurbanne, France) 2016-02-02 11:27 GMT+01:00 Satya Harish Appana: > Queries I am running over Hive JDBC are ddl statements(none of the queries > are select or insert. which will result in an execution engine(tez/mr) job > to be launched.. all the queries are create external table .. and drop > table .. and alter table add partitions). > > > On Tue, Feb 2, 2016 at 3:54 PM, Loïc Chanel > wrote: > >> Actually, Hive doesn't support timeout, but Tez and MapReduce does. >> Therefore, you can set a timeout on these tools to kill failed queries. >> Hope this helps, >> >> Loïc >> >> Loïc CHANEL >> System & virtualization engineer >> TO - XaaS Ind - Worldline (Villeurbanne, France) >> >> 2016-02-02 11:10 GMT+01:00 董亚军 : >> >>> hive does not support timeout on the client side. >>> >>> and I think it is not recommended that if the client exit with timeout >>> exception, the hiveserver side may also running the job. this will result >>> in inconsistent state. >>> >>> On Tue, Feb 2, 2016 at 4:49 PM, Satya Harish Appana < >>> satyaharish.app...@gmail.com> wrote: >>> Hi Team, I am trying to connect to hiveServer via hive-jdbc. Can we configure client side timeout at each query executed inside each jdbc connection. (When I looked at HiveStatement.setQueryTimeout method it says operation unsupported). Is there any other way of timing out and cancelling the connection and throwing Exception, if it alive for over a period of 4 mins or so (configurable at client side). PS : Queries that I am executing over jdbc are simple ddl statements. (hive external table create statements and drop table statements). Regards, Satya Harish. >>> >>> >> > > > -- > > > Regards, > Satya Harish Appana, > Software Development Engineer II, > Flipkart,Bangalore, > Ph:+91-9538797174. >
Re: Query Timeout
Hi Ibrar What is the value given for --hiveconf *hbase.master*=# OR --hiveconf *hbase.zookeeper.quorum*= It seems from the error that HBase Server configuration are not correct. Regards Sanjiv Singh Mob : +091 9990-447-339 On Wed, Jun 17, 2015 at 4:31 PM, Ibrar Ahmed ibrar.ah...@gmail.com wrote: I am able to fix that issue, but got another error [127.0.0.1:1] hive CREATE TABLE IF NOT EXISTS pagecounts_hbase (rowkey STRING, pageviews STRING, bytes STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES (' hbase.table.name' = 'pagecounts'); [Hive Error]: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.lang.IllegalArgumentException: Not a host:port pair: PBUF ibrar-virtual-machine �� �߯��) �� at org.apache.hadoop.hbase.util.Addressing.parseHostname(Addressing.java:60) at org.apache.hadoop.hbase.ServerName.init(ServerName.java:96) at org.apache.hadoop.hbase.ServerName.parseVersionedServerName(ServerName.java:278) at org.apache.hadoop.hbase.MasterAddressTracker.bytesToServerName(MasterAddressTracker.java:77) at org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:61) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:631) at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:106) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:84) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:162) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:554) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:547) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:613) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1472) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1239) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1057) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) On Wed, Jun 17, 2015 at 3:51 PM, Ibrar Ahmed ibrar.ah...@gmail.com wrote: Hi, Whats wrong with my settings? [127.0.0.1:1] hive CREATE TABLE IF NOT EXISTS pagecounts_hbase (rowkey STRING, pageviews STRING, bytes STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES (' hbase.table.name' = 'pagecounts'); [Hive Error]: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:127) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:84) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:162) at
Query Timeout
Hi, Whats wrong with my settings? [127.0.0.1:1] hive CREATE TABLE IF NOT EXISTS pagecounts_hbase (rowkey STRING, pageviews STRING, bytes STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES (' hbase.table.name' = 'pagecounts'); [Hive Error]: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:127) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:84) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:162) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:554) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:547) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:613) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1472) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1239) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1057) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:88) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:162) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:554) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:547) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:613) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1472) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1239) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1057) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at
Re: Query Timeout
I am able to fix that issue, but got another error [127.0.0.1:1] hive CREATE TABLE IF NOT EXISTS pagecounts_hbase (rowkey STRING, pageviews STRING, bytes STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES (' hbase.table.name' = 'pagecounts'); [Hive Error]: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:java.lang.IllegalArgumentException: Not a host:port pair: PBUF ibrar-virtual-machine���߯��)�� at org.apache.hadoop.hbase.util.Addressing.parseHostname(Addressing.java:60) at org.apache.hadoop.hbase.ServerName.init(ServerName.java:96) at org.apache.hadoop.hbase.ServerName.parseVersionedServerName(ServerName.java:278) at org.apache.hadoop.hbase.MasterAddressTracker.bytesToServerName(MasterAddressTracker.java:77) at org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:61) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:631) at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:106) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:84) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:162) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:554) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:547) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:613) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1472) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1239) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1057) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) On Wed, Jun 17, 2015 at 3:51 PM, Ibrar Ahmed ibrar.ah...@gmail.com wrote: Hi, Whats wrong with my settings? [127.0.0.1:1] hive CREATE TABLE IF NOT EXISTS pagecounts_hbase (rowkey STRING, pageviews STRING, bytes STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES (' hbase.table.name' = 'pagecounts'); [Hive Error]: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:127) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:84) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:162) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:554) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:547) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at