Jingxuan Fu created HIVE-26211:
----------------------------------
Summary: "hive.server2.webui.max.historic.queries" should be
avoided to be set too large, otherwise it will cause blocking
Key: HIVE-26211
URL: https://issues.apache.org/jira/browse/HIVE-26211
Project: Hive
Issue Type: Bug
Affects Versions: 3.1.2
Environment: Hive 3.1.2
os.name=Linux
os.arch=amd64
os.version=5.4.0-72-generic
java.version=1.8.0_162
java.vendor=Oracle Corporation
Reporter: Jingxuan Fu
Assignee: Jingxuan Fu
In hive-default.xml.template
<property>
<name>hive.server2.webui.max.historic.queries</name>
<value>25</value>
<description>The maximum number of past queries to show in HiverSever2
WebUI.</description>
</property>
Set hive.server2.webui.max.historic.queries to a relatively large value, take
20000000 as an example, start hiveserver2, it can start hiveserver normally,
and logging without exception.
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-09 20:03:41: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d
Next, if you use beeline as a client to connect to hive and send a request for
database related operations, for example, if you query all the databases, after
successfully executing "show databases", beeline blocks and no other operations
can be performed.
liky@ljq1:/opt/hive$ beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline> !connect jdbc:hive2://192.168.1.194:10000/default
Connecting to jdbc:hive2://192.168.1.194:10000/default
Enter username for jdbc:hive2://192.168.1.194:10000/default: hive
Enter password for jdbc:hive2://192.168.1.194:10000/default: *****
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.194:10000/default> show databases
. . . . . . . . . . . . . . . . . . . . . .> ;
INFO : Compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from
deserializer)], properties:null)
INFO : Completed compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.393 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.109 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
+----------------+
| database_name |
+----------------+
| default |
+----------------+
1 row selected (1.374 seconds)
Also, on the hiveserver side, a runtime null pointer exception is thrown, and
the observation log throws no warnings or errors.
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hbase-2.2.2/hbase/maven/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-09 20:03:41: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d
OK
Exception in thread "HiveServer2-Handler-Pool: Thread-41"
java.lang.RuntimeException: java.lang.NullPointerException
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy42.close(Unknown Source)
at
org.apache.hive.service.cli.session.SessionManager.closeSession(SessionManager.java:552)
at
org.apache.hive.service.cli.CLIService.closeSession(CLIService.java:241)
at
org.apache.hive.service.cli.thrift.ThriftBinaryCLIService$1.deleteContext(ThriftBinaryCLIService.java:141)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:300)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at
org.apache.hive.service.cli.operation.OperationManager.getQueryId(OperationManager.java:189)
at
org.apache.hive.service.cli.operation.OperationManager.removeOperation(OperationManager.java:206)
at
org.apache.hive.service.cli.operation.OperationManager.closeOperation(OperationManager.java:287)
at
org.apache.hive.service.cli.session.HiveSessionImpl.close(HiveSessionImpl.java:751)
at
org.apache.hive.service.cli.session.HiveSessionImplwithUGI.close(HiveSessionImplwithUGI.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
... 14 more
If you open another beeline as a client to initiate a database operation
request to the hiveserver, the beeline will also block after executing a
request and the hiveserver will throw the same exception.
If you call java API write a program to establish a jdbc connection as a client
to access hive, take JDBCDemo as an example.
import demo.utils.JDBCUtils;
public class JDBCDemo{
public static void main(String[] args) throws Exception {
JDBCUtils.init();
JDBCUtils.createDatabase();
JDBCUtils.showDatabases();
JDBCUtils.createTable();
JDBCUtils.showTables();
JDBCUtils.descTable();
JDBCUtils.loadData();
JDBCUtils.selectData();
JDBCUtils.countData();
JDBCUtils.dropDatabase();
JDBCUtils.dropTable();
JDBCUtils.destory();
}
}
Starting the hiveserver and running the written program will block directly
after the first database operation request, and the terminal will not throw any
hints, in line with beeline's blocking node, the hiveserver throws the same
exceptions as before.
liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Running: drop database if exists hive_jdbc_test
Running: create database hive_jdbc_test
^Cliky@ljq1:~/hive_jdbc_test$
In summary, when the value of hive.server2.webui.max.historic.queries is set to
a larger value, {*}it blocks all clients that initiate the request{*}, and the
hive server side only throws a runtime null pointer exception, and the
observation log throws no related warnings and errors, causing some impact.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
