Hive Metastore: how to overwrite default log file name?
Hello, everyone, I'm using Hive 2.1.1. I can't find a way to overwrite the default log file name for Metastore service. Here is what I've tried: hive --service metastore --hiveconf hive.log.file=metastore.log Setting the hive.log.file variable seems to work for HiveServer2 (hive --service hiveserver2 --hiveconf hive.log.file=hs2.log), but not for Metastore. Is there a way to specify the log file for Metastore on command line? Thanks in advance and best regards.
query with FIST_VALUE/LAST_VALUE functions keep running forever
Hello everyone, I'm trying to run a query on an 8 node cluster with hive-0.13 (MapR 3.1.1): SELECT FIRST_VALUE(col_a) OVER (PARTITION BY col_b ORDER BY col_c) FROM test; If any partition is over 3 rows, the reduce phase of my query keeps running forever (until job is being killed by JobTracker). Is this normal behavior? A normal ORDER BY on a table of 7 millions of rows takes about 70 seconds to complete on the same cluster. Thanks and best regards, -- AVISO CONFIDENCIAL\nEste correo y la información contenida o adjunta al mismo es privada y confidencial y va dirigida exclusivamente a su destinatario. Pragsis informa a quien pueda haber recibido este correo por error que contiene información confidencial cuyo uso, copia, reproducción o distribución está expresamente prohibida. Si no es Vd. el destinatario del mismo y recibe este correo por error, le rogamos lo ponga en conocimiento del emisor y proceda a su eliminación sin copiarlo, imprimirlo o utilizarlo de ningún modo.\nCONFIDENTIALITY WARNING.\nThis message and the information contained in or attached to it are private and confidential and intended exclusively for the addressee. Pragsis informs to whom it may receive it in error that it contains privileged information and its use, copy, reproduction or distribution is prohibited. If you are not an intended recipient of this E-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, reta in or redistribute any portion of this E-mail.
Re: Number of hive-server2 threads increments after jdbc connection
Hi,
I have ipc.client.connection.maxidletime set to default (10 sec). The
hive-server2 threads (and tcp connections to zookeeper) stay active
forever. I guess these issues might be related but they are not the same.
Regards,
Dima Fadeyev
El 05/05/14 03:47, Shengjun Xin escribió:
Is it same with https://issues.apache.org/jira/browse/HIVE-6866 ?
On Tue, Apr 29, 2014 at 8:26 PM, Dima Fadeyev <mailto:dfade...@pragsis.com>> wrote:
Hi, Chinna. Thanks for your reply.
Yes, modifying code solves the problem. This is what my code looks
like (a piece of it):
Connection con =
DriverManager.getConnection("jdbc:hive2://localhost:1/default", "hive",
"hive");
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
//stmt.close();
When I uncomment the last line, the number of hive-server2 threads
doesn't keep incrementing to infinity. However I'm investigating
the issue where the code is not really my code. Is there a way to
correct this behavior from within hive-server2 without changing
the client's code?
El 29/04/14 14:05, Chinna Rao Lalam escribió:
Hi,
In your code if more connections and statements are created?. If
so, closed those connections?
After use close unused connections and statements.
Hope It Helps,
Chinna
On Tue, Apr 29, 2014 at 3:47 PM, Dima Fadeyev
mailto:dfade...@pragsis.com>> wrote:
Hello everyone,
When I run a jdbc example from
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode
against my hive server, the number of hive-server2 threads
increments. If I execute it long enough I either start seeing
exceptions
Exception in thread "main" java.sql.SQLException:
org.apache.thrift.TApplicationException: Internal error
processing ExecuteStatement
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:203)
at HiveJdbcClient.main(HiveJdbcClient.java:24)
Caused by: org.apache.thrift.TApplicationException: Internal
error processing ExecuteStatement
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.hive.service.cli.thrift.TCLIService$Client.recv_ExecuteStatement(TCLIService.java:213)
at
org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:200)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:197)
... 1 more
or I bump into a limit of connections with Zookeeper (each
hive-server2 thread maintains a connection with Zookeeper. I
have hive.support.concurrency enabled).
In either case I can't connect to hive server after that.
I've tried this on hive 0.10 (cdh 4.4) and hive 0.12 (cdh 5.0
and hdp 2.0.6) with same results.
Please, could anyone help me resolve this.
Thanks in advance.
--
Hope It Helps,
Chinna
--
Regards
Shengjun
Re: Number of hive-server2 threads increments after jdbc connection
Hi, Chinna. Thanks for your reply.
Yes, modifying code solves the problem. This is what my code looks like
(a piece of it):
Connection con =
DriverManager.getConnection("jdbc:hive2://localhost:1/default",
"hive", "hive");
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
//stmt.close();
When I uncomment the last line, the number of hive-server2 threads
doesn't keep incrementing to infinity. However I'm investigating the
issue where the code is not really my code. Is there a way to correct
this behavior from within hive-server2 without changing the client's code?
El 29/04/14 14:05, Chinna Rao Lalam escribió:
Hi,
In your code if more connections and statements are created?. If so,
closed those connections?
After use close unused connections and statements.
Hope It Helps,
Chinna
On Tue, Apr 29, 2014 at 3:47 PM, Dima Fadeyev <mailto:dfade...@pragsis.com>> wrote:
Hello everyone,
When I run a jdbc example from
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode
against my hive server, the number of hive-server2 threads
increments. If I execute it long enough I either start seeing
exceptions
Exception in thread "main" java.sql.SQLException:
org.apache.thrift.TApplicationException: Internal error processing
ExecuteStatement
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:203)
at HiveJdbcClient.main(HiveJdbcClient.java:24)
Caused by: org.apache.thrift.TApplicationException: Internal error
processing ExecuteStatement
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.hive.service.cli.thrift.TCLIService$Client.recv_ExecuteStatement(TCLIService.java:213)
at
org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:200)
at
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:197)
... 1 more
or I bump into a limit of connections with Zookeeper (each
hive-server2 thread maintains a connection with Zookeeper. I have
hive.support.concurrency enabled).
In either case I can't connect to hive server after that.
I've tried this on hive 0.10 (cdh 4.4) and hive 0.12 (cdh 5.0 and
hdp 2.0.6) with same results.
Please, could anyone help me resolve this.
Thanks in advance.
--
Hope It Helps,
Chinna
Number of hive-server2 threads increments after jdbc connection
Hello everyone, When I run a jdbc example from https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode against my hive server, the number of hive-server2 threads increments. If I execute it long enough I either start seeing exceptions Exception in thread "main" java.sql.SQLException: org.apache.thrift.TApplicationException: Internal error processing ExecuteStatement at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:203) at HiveJdbcClient.main(HiveJdbcClient.java:24) Caused by: org.apache.thrift.TApplicationException: Internal error processing ExecuteStatement at org.apache.thrift.TApplicationException.read(TApplicationException.java:108) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71) at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_ExecuteStatement(TCLIService.java:213) at org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:200) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:197) ... 1 more or I bump into a limit of connections with Zookeeper (each hive-server2 thread maintains a connection with Zookeeper. I have hive.support.concurrency enabled). In either case I can't connect to hive server after that. I've tried this on hive 0.10 (cdh 4.4) and hive 0.12 (cdh 5.0 and hdp 2.0.6) with same results. Please, could anyone help me resolve this. Thanks in advance.
