[jira] [Updated] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26213:
---
Issue Type: Bug (was: Improvement)
> "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it
> will raise an error
> ---
>
> Key: HIVE-26213
> URL: https://issues.apache.org/jira/browse/HIVE-26213
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: Hive 3.1.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_162
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Minor
>
> In hive-default.xml.template
> {code:java}
>
> hive.limit.pushdown.memory.usage
> 0.1
>
> Expects value between 0.0f and 1.0f.
> The fraction of available memory to be used for buffering rows in
> Reducesink operator for limit pushdown optimization.
>
> {code}
> Based on the description of hive-default.xml.template,
> hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
> hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
> memory of all buffered lines for the limit pushdown optimization, and
> successfully start hiveserver2.
> Then, call the java api to write a program to establish a jdbc connection as
> a client to access hive, using JDBCDemo as an example.
> {code:java}
> import demo.utils.JDBCUtils;
> public class JDBCDemo{
> public static void main(String[] args) throws Exception
> { JDBCUtils.init(); JDBCUtils.createDatabase();
> JDBCUtils.showDatabases(); JDBCUtils.createTable();
> JDBCUtils.showTables(); JDBCUtils.descTable(); JDBCUtils.loadData();
> JDBCUtils.selectData(); JDBCUtils.countData(); JDBCUtils.dropDatabase();
> JDBCUtils.dropTable(); JDBCUtils.destory(); }
> }
> {code}
> After running the client program, both the client and the hiveserver throw
> exceptions.
> {code:java}
> 2022-05-09 19:05:36: Starting HiveServer2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Hive Session ID = 67a6db8d-f957-4d5d-ac18-28403adab7f3
> Hive Session ID = f9f8772c-5765-4c3e-bcff-ca605c667be7
> OK
> OK
> OK
> OK
> OK
> OK
> OK
> Loading data to table default.emp
> OK
> FAILED: SemanticException Invalid memory usage value 1.0 for
> hive.limit.pushdown.memory.usage{code}
> {code:java}
> liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Running: drop database if exists hive_jdbc_test
> Running: create database hive_jdbc_test
> Running: show databases
> default
> hive_jdbc_test
> Running: drop table if exists emp
> Running: create table emp(
> empno int,
> ename string,
> job string,
> mgr int,
> hiredate string,
> sal double,
> comm double,
> deptno int
> )
> row format delimited fields terminated by '\t'
> Running: show tables
> emp
> Running: desc emp
> empno int
> ename string
> job string
> mgr int
> hiredate string
> sal double
> comm double
> deptno int
> Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
> overwrite into table emp
> Running: select * from emp
> Exception in thread "main" org.apache.hive.service.cli.HiveSQLException:
> Error while compiling statement: FAILED: SemanticException Invalid memory
> usage value 1.0 for hive.limit.pushdown.memory.usage
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366)
> at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509)
> at
[jira] [Updated] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26213:
---
Issue Type: Improvement (was: Bug)
> "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it
> will raise an error
> ---
>
> Key: HIVE-26213
> URL: https://issues.apache.org/jira/browse/HIVE-26213
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.1.2
> Environment: Hive 3.1.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_162
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Major
>
> In hive-default.xml.template
> {code:java}
>
> hive.limit.pushdown.memory.usage
> 0.1
>
> Expects value between 0.0f and 1.0f.
> The fraction of available memory to be used for buffering rows in
> Reducesink operator for limit pushdown optimization.
>
> {code}
> Based on the description of hive-default.xml.template,
> hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
> hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
> memory of all buffered lines for the limit pushdown optimization, and
> successfully start hiveserver2.
> Then, call the java api to write a program to establish a jdbc connection as
> a client to access hive, using JDBCDemo as an example.
> {code:java}
> import demo.utils.JDBCUtils;
> public class JDBCDemo{
> public static void main(String[] args) throws Exception
> { JDBCUtils.init(); JDBCUtils.createDatabase();
> JDBCUtils.showDatabases(); JDBCUtils.createTable();
> JDBCUtils.showTables(); JDBCUtils.descTable(); JDBCUtils.loadData();
> JDBCUtils.selectData(); JDBCUtils.countData(); JDBCUtils.dropDatabase();
> JDBCUtils.dropTable(); JDBCUtils.destory(); }
> }
> {code}
> After running the client program, both the client and the hiveserver throw
> exceptions.
> {code:java}
> 2022-05-09 19:05:36: Starting HiveServer2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Hive Session ID = 67a6db8d-f957-4d5d-ac18-28403adab7f3
> Hive Session ID = f9f8772c-5765-4c3e-bcff-ca605c667be7
> OK
> OK
> OK
> OK
> OK
> OK
> OK
> Loading data to table default.emp
> OK
> FAILED: SemanticException Invalid memory usage value 1.0 for
> hive.limit.pushdown.memory.usage{code}
> {code:java}
> liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Running: drop database if exists hive_jdbc_test
> Running: create database hive_jdbc_test
> Running: show databases
> default
> hive_jdbc_test
> Running: drop table if exists emp
> Running: create table emp(
> empno int,
> ename string,
> job string,
> mgr int,
> hiredate string,
> sal double,
> comm double,
> deptno int
> )
> row format delimited fields terminated by '\t'
> Running: show tables
> emp
> Running: desc emp
> empno int
> ename string
> job string
> mgr int
> hiredate string
> sal double
> comm double
> deptno int
> Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
> overwrite into table emp
> Running: select * from emp
> Exception in thread "main" org.apache.hive.service.cli.HiveSQLException:
> Error while compiling statement: FAILED: SemanticException Invalid memory
> usage value 1.0 for hive.limit.pushdown.memory.usage
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366)
> at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509)
> at
[jira] [Updated] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26213:
---
Priority: Minor (was: Major)
> "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it
> will raise an error
> ---
>
> Key: HIVE-26213
> URL: https://issues.apache.org/jira/browse/HIVE-26213
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.1.2
> Environment: Hive 3.1.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_162
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Minor
>
> In hive-default.xml.template
> {code:java}
>
> hive.limit.pushdown.memory.usage
> 0.1
>
> Expects value between 0.0f and 1.0f.
> The fraction of available memory to be used for buffering rows in
> Reducesink operator for limit pushdown optimization.
>
> {code}
> Based on the description of hive-default.xml.template,
> hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
> hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
> memory of all buffered lines for the limit pushdown optimization, and
> successfully start hiveserver2.
> Then, call the java api to write a program to establish a jdbc connection as
> a client to access hive, using JDBCDemo as an example.
> {code:java}
> import demo.utils.JDBCUtils;
> public class JDBCDemo{
> public static void main(String[] args) throws Exception
> { JDBCUtils.init(); JDBCUtils.createDatabase();
> JDBCUtils.showDatabases(); JDBCUtils.createTable();
> JDBCUtils.showTables(); JDBCUtils.descTable(); JDBCUtils.loadData();
> JDBCUtils.selectData(); JDBCUtils.countData(); JDBCUtils.dropDatabase();
> JDBCUtils.dropTable(); JDBCUtils.destory(); }
> }
> {code}
> After running the client program, both the client and the hiveserver throw
> exceptions.
> {code:java}
> 2022-05-09 19:05:36: Starting HiveServer2
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Hive Session ID = 67a6db8d-f957-4d5d-ac18-28403adab7f3
> Hive Session ID = f9f8772c-5765-4c3e-bcff-ca605c667be7
> OK
> OK
> OK
> OK
> OK
> OK
> OK
> Loading data to table default.emp
> OK
> FAILED: SemanticException Invalid memory usage value 1.0 for
> hive.limit.pushdown.memory.usage{code}
> {code:java}
> liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Running: drop database if exists hive_jdbc_test
> Running: create database hive_jdbc_test
> Running: show databases
> default
> hive_jdbc_test
> Running: drop table if exists emp
> Running: create table emp(
> empno int,
> ename string,
> job string,
> mgr int,
> hiredate string,
> sal double,
> comm double,
> deptno int
> )
> row format delimited fields terminated by '\t'
> Running: show tables
> emp
> Running: desc emp
> empno int
> ename string
> job string
> mgr int
> hiredate string
> sal double
> comm double
> deptno int
> Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
> overwrite into table emp
> Running: select * from emp
> Exception in thread "main" org.apache.hive.service.cli.HiveSQLException:
> Error while compiling statement: FAILED: SemanticException Invalid memory
> usage value 1.0 for hive.limit.pushdown.memory.usage
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366)
> at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509)
> at
[jira] [Updated] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26213:
---
Description:
In hive-default.xml.template
{code:java}
hive.limit.pushdown.memory.usage
0.1
Expects value between 0.0f and 1.0f.
The fraction of available memory to be used for buffering rows in
Reducesink operator for limit pushdown optimization.
{code}
Based on the description of hive-default.xml.template,
hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
memory of all buffered lines for the limit pushdown optimization, and
successfully start hiveserver2.
Then, call the java api to write a program to establish a jdbc connection as a
client to access hive, using JDBCDemo as an example.
{code:java}
import demo.utils.JDBCUtils;
public class JDBCDemo{
public static void main(String[] args) throws Exception
{ JDBCUtils.init(); JDBCUtils.createDatabase();
JDBCUtils.showDatabases(); JDBCUtils.createTable(); JDBCUtils.showTables();
JDBCUtils.descTable(); JDBCUtils.loadData(); JDBCUtils.selectData();
JDBCUtils.countData(); JDBCUtils.dropDatabase(); JDBCUtils.dropTable();
JDBCUtils.destory(); }
}
{code}
After running the client program, both the client and the hiveserver throw
exceptions.
{code:java}
2022-05-09 19:05:36: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 67a6db8d-f957-4d5d-ac18-28403adab7f3
Hive Session ID = f9f8772c-5765-4c3e-bcff-ca605c667be7
OK
OK
OK
OK
OK
OK
OK
Loading data to table default.emp
OK
FAILED: SemanticException Invalid memory usage value 1.0 for
hive.limit.pushdown.memory.usage{code}
{code:java}
liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Running: drop database if exists hive_jdbc_test
Running: create database hive_jdbc_test
Running: show databases
default
hive_jdbc_test
Running: drop table if exists emp
Running: create table emp(
empno int,
ename string,
job string,
mgr int,
hiredate string,
sal double,
comm double,
deptno int
)
row format delimited fields terminated by '\t'
Running: show tables
emp
Running: desc emp
empno int
ename string
job string
mgr int
hiredate string
sal double
comm double
deptno int
Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
overwrite into table emp
Running: select * from emp
Exception in thread "main" org.apache.hive.service.cli.HiveSQLException: Error
while compiling statement: FAILED: SemanticException Invalid memory usage value
1.0 for hive.limit.pushdown.memory.usage
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366)
at
org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293)
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509)
at demo.utils.JDBCUtils.selectData(JDBCUtils.java:98)
at demo.test.JDBCDemo.main(JDBCDemo.java:19){code}
Setting hive.limit.pushdown.memory.usage to 0.0 has no exception.
So, setting hive.limit.pushdown.memory.usage to 1.0 is not desirable,
*hive-default.xml.template is not clear enough for the description of the
boundary of the value, it is better to use the interval to indicate the value
that is [0.0,1.0).*
was:
In hive-default.xml.template
{code:java}
hive.limit.pushdown.memory.usage
0.1
Expects value between 0.0f and 1.0f.
The fraction of available memory to be used for buffering rows in
Reducesink operator for limit pushdown optimization.
{code}
Based on the description of hive-default.xml.template,
hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
memory of
[jira] [Updated] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26213:
---
Description:
In hive-default.xml.template
{code:java}
hive.limit.pushdown.memory.usage
0.1
Expects value between 0.0f and 1.0f.
The fraction of available memory to be used for buffering rows in
Reducesink operator for limit pushdown optimization.
{code}
Based on the description of hive-default.xml.template,
hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
memory of all buffered lines for the limit pushdown optimization, and
successfully start hiveserver2.
Then, call the java api to write a program to establish a jdbc connection as a
client to access hive, using JDBCDemo as an example.
hive.limit.pushdown.memory.usage
0.1
Expects value between 0.0f and 1.0f.
The fraction of available memory to be used for buffering rows in
Reducesink operator for limit pushdown optimization.
Based on the description of hive-default.xml.template,
hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
memory of all buffered lines for the limit pushdown optimization, and
successfully start hiveserver2.
Then, call the java api to write a program to establish a jdbc connection as a
client to access hive, using JDBCDemo as an example.
import demo.utils.JDBCUtils;
public class JDBCDemo{
public static void main(String[] args) throws Exception {
JDBCUtils.init();
JDBCUtils.createDatabase();
JDBCUtils.showDatabases();
JDBCUtils.createTable();
JDBCUtils.showTables();
JDBCUtils.descTable();
JDBCUtils.loadData();
JDBCUtils.selectData();
JDBCUtils.countData();
JDBCUtils.dropDatabase();
JDBCUtils.dropTable();
JDBCUtils.destory();
}
}
After running the client program, both the client and the hiveserver throw
exceptions.
{code:java}
2022-05-09 19:05:36: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 67a6db8d-f957-4d5d-ac18-28403adab7f3
Hive Session ID = f9f8772c-5765-4c3e-bcff-ca605c667be7
OK
OK
OK
OK
OK
OK
OK
Loading data to table default.emp
OK
FAILED: SemanticException Invalid memory usage value 1.0 for
hive.limit.pushdown.memory.usage{code}
{code:java}
liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Running: drop database if exists hive_jdbc_test
Running: create database hive_jdbc_test
Running: show databases
default
hive_jdbc_test
Running: drop table if exists emp
Running: create table emp(
empno int,
ename string,
job string,
mgr int,
hiredate string,
sal double,
comm double,
deptno int
)
row format delimited fields terminated by '\t'
Running: show tables
emp
Running: desc emp
empno int
ename string
job string
mgr int
hiredate string
sal double
comm double
deptno int
Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
overwrite into table emp
Running: select * from emp
Exception in thread "main" org.apache.hive.service.cli.HiveSQLException: Error
while compiling statement: FAILED: SemanticException Invalid memory usage value
1.0 for hive.limit.pushdown.memory.usage
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366)
at
org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293)
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509)
at demo.utils.JDBCUtils.selectData(JDBCUtils.java:98)
at demo.test.JDBCDemo.main(JDBCDemo.java:19){code}
Setting hive.limit.pushdown.memory.usage to 0.0 has no exception.
So, setting hive.limit.pushdown.memory.usage to 1.0 is
[jira] [Updated] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26213:
---
Description:
In hive-default.xml.template
hive.limit.pushdown.memory.usage
0.1
Expects value between 0.0f and 1.0f.
The fraction of available memory to be used for buffering rows in
Reducesink operator for limit pushdown optimization.
Based on the description of hive-default.xml.template,
hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
memory of all buffered lines for the limit pushdown optimization, and
successfully start hiveserver2.
Then, call the java api to write a program to establish a jdbc connection as a
client to access hive, using JDBCDemo as an example.
import demo.utils.JDBCUtils;
public class JDBCDemo{
public static void main(String[] args) throws Exception {
JDBCUtils.init();
JDBCUtils.createDatabase();
JDBCUtils.showDatabases();
JDBCUtils.createTable();
JDBCUtils.showTables();
JDBCUtils.descTable();
JDBCUtils.loadData();
JDBCUtils.selectData();
JDBCUtils.countData();
JDBCUtils.dropDatabase();
JDBCUtils.dropTable();
JDBCUtils.destory();
}
}
After running the client program, both the client and the hiveserver throw
exceptions.
2022-05-09 19:05:36: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 67a6db8d-f957-4d5d-ac18-28403adab7f3
Hive Session ID = f9f8772c-5765-4c3e-bcff-ca605c667be7
OK
OK
OK
OK
OK
OK
OK
Loading data to table default.emp
OK
FAILED: SemanticException Invalid memory usage value 1.0 for
hive.limit.pushdown.memory.usage
liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Running: drop database if exists hive_jdbc_test
Running: create database hive_jdbc_test
Running: show databases
default
hive_jdbc_test
Running: drop table if exists emp
Running: create table emp(
empno int,
ename string,
job string,
mgr int,
hiredate string,
sal double,
comm double,
deptno int
)
row format delimited fields terminated by '\t'
Running: show tables
emp
Running: desc emp
empno int
ename string
job string
mgr int
hiredate string
sal double
comm double
deptno int
Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
overwrite into table emp
Running: select * from emp
Exception in thread "main" org.apache.hive.service.cli.HiveSQLException: Error
while compiling statement: FAILED: SemanticException Invalid memory usage value
1.0 for hive.limit.pushdown.memory.usage
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366)
at
org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509)
at demo.utils.JDBCUtils.selectData(JDBCUtils.java:98)
at demo.test.JDBCDemo.main(JDBCDemo.java:19)
Setting hive.limit.pushdown.memory.usage to 0.0 has no exception.
So, setting hive.limit.pushdown.memory.usage to 1.0 is not desirable,
*hive-default.xml.template is not clear enough for the description of the
boundary of the value, it is better to use the interval to indicate the value
that is [0.0,1.0).*
was:
In hive-default.xml.template
{code:java}
hive.limit.pushdown.memory.usage 0.1
Expects value between 0.0f and 1.0f. The fraction of available
memory to be used for buffering rows in Reducesink operator for limit pushdown
optimization.
{code}
Based on the description of hive-default.xml.template,
hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
memory of all buffered
[jira] [Assigned] (HIVE-26213) "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it will raise an error
[
https://issues.apache.org/jira/browse/HIVE-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu reassigned HIVE-26213:
--
> "hive.limit.pushdown.memory.usage" better not be equal to 1.0, otherwise it
> will raise an error
> ---
>
> Key: HIVE-26213
> URL: https://issues.apache.org/jira/browse/HIVE-26213
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: Hive 3.1.2
> os.name=Linux
> os.arch=amd64
> os.version=5.4.0-72-generic
> java.version=1.8.0_162
> java.vendor=Oracle Corporation
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Major
>
> In hive-default.xml.template
>
>
>
>
> {code:java}
> hive.limit.pushdown.memory.usage 0.1
> Expects value between 0.0f and 1.0f. The fraction of available
> memory to be used for buffering rows in Reducesink operator for limit
> pushdown optimization.
> {code}
>
>
> Based on the description of hive-default.xml.template,
> hive.limit.pushdown.memory.usage expects a value between 0.0 and 1.0, setting
> hive.limit.pushdown.memory.usage to 1.0 means that it expects the available
> memory of all buffered lines for the limit pushdown optimization, and
> successfully start hiveserver2.
> Then, call the java api to write a program to establish a jdbc connection as
> a client to access hive, using JDBCDemo as an example.
>
> {code:java}
> import demo.utils.JDBCUtils; public class JDBCDemo{ public static void
> main(String[] args) throws Exception { JDBCUtils.init();
> JDBCUtils.createDatabase(); JDBCUtils.showDatabases();
> JDBCUtils.createTable(); JDBCUtils.showTables(); JDBCUtils.descTable();
> JDBCUtils.loadData(); JDBCUtils.selectData(); JDBCUtils.countData();
> JDBCUtils.dropDatabase(); JDBCUtils.dropTable(); JDBCUtils.destory(); } }
> {code}
> After running the client program, both the client and the hiveserver throw
> exceptions.
>
> {code:java}
> 2022-05-09 19:05:36: Starting HiveServer2 SLF4J: Class path contains multiple
> SLF4J bindings. SLF4J: Found binding in
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory] Hive Session ID =
> 67a6db8d-f957-4d5d-ac18-28403adab7f3 Hive Session ID =
> f9f8772c-5765-4c3e-bcff-ca605c667be7 OK OK OK OK OK OK OK Loading data to
> table default.emp OK FAILED: SemanticException Invalid memory usage value 1.0
> for hive.limit.pushdown.memory.usage{code}
>
>
>
> {code:java}
> liky@ljq1:~/hive_jdbc_test$ ./startJDBC_0.sh SLF4J: Class path contains
> multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.17.1/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/liky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory] Running: drop database if
> exists hive_jdbc_test Running: create database hive_jdbc_test Running: show
> databases default hive_jdbc_test Running: drop table if exists emp Running:
> create table emp( empno int, ename string, job string, mgr int, hiredate
> string, sal double, comm double, deptno int ) row format delimited fields
> terminated by '\t' Running: show tables emp Running: desc emp empno int ename
> string job string mgr int hiredate string sal double comm double deptno int
> Running: load data local inpath '/home/liky/hiveJDBCTestData/data.txt'
> overwrite into table emp Running: select * from emp Exception in thread
> "main" org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: SemanticException Invalid memory usage value 1.0 for
> hive.limit.pushdown.memory.usage at
> org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:380) at
> org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:366) at
> org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:354)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:293) at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:509) at
> demo.utils.JDBCUtils.selectData(JDBCUtils.java:98) at
> demo.test.JDBCDemo.main(JDBCDemo.java:19){code}
>
>
> Setting hive.limit.pushdown.memory.usage
[jira] [Updated] (HIVE-26211) "hive.server2.webui.max.historic.queries" should be avoided to be set too large, otherwise it will cause blocking
[
https://issues.apache.org/jira/browse/HIVE-26211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26211:
---
Description:
In hive-default.xml.template
{code:java}
hive.server2.webui.max.historic.queries
25
The maximum number of past queries to show in HiverSever2
WebUI.
{code}
Set hive.server2.webui.max.historic.queries to a relatively large value, take
2000 as an example, start hiveserver2, it can start hiveserver normally,
and logging without exception.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-09 20:03:41: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d{code}
Next, if you use beeline as a client to connect to hive and send a request for
database related operations, for example, if you query all the databases, after
successfully executing "show databases", beeline blocks and no other operations
can be performed.
{code:java}
liky@ljq1:/opt/hive$ beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline> !connect jdbc:hive2://192.168.1.194:1/default
Connecting to jdbc:hive2://192.168.1.194:1/default
Enter username for jdbc:hive2://192.168.1.194:1/default: hive
Enter password for jdbc:hive2://192.168.1.194:1/default: *
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.194:1/default> show databases
. . . . . . . . . . . . . . . . . . . . . .> ;
INFO : Compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from
deserializer)], properties:null)
INFO : Completed compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.393 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.109 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
database_name
default
1 row selected (1.374 seconds)
{code}
Also, on the hiveserver side, a runtime null pointer exception is thrown, and
the observation log throws no warnings or errors.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jira] [Updated] (HIVE-26211) "hive.server2.webui.max.historic.queries" should be avoided to be set too large, otherwise it will cause blocking
[
https://issues.apache.org/jira/browse/HIVE-26211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26211:
---
Description:
In hive-default.xml.template
{code:java}
hive.server2.webui.max.historic.queries
25
The maximum number of past queries to show in HiverSever2
WebUI.
{code}
Set hive.server2.webui.max.historic.queries to a relatively large value, take
2000 as an example, start hiveserver2, it can start hiveserver normally,
and logging without exception.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-09 20:03:41: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d{code}
Next, if you use beeline as a client to connect to hive and send a request for
database related operations, for example, if you query all the databases, after
successfully executing "show databases", beeline blocks and no other operations
can be performed.
{code:java}
liky@ljq1:/opt/hive$ beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline> !connect jdbc:hive2://192.168.1.194:1/default
Connecting to jdbc:hive2://192.168.1.194:1/default
Enter username for jdbc:hive2://192.168.1.194:1/default: hive
Enter password for jdbc:hive2://192.168.1.194:1/default: *
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.194:1/default> show databases
. . . . . . . . . . . . . . . . . . . . . .> ;
INFO : Compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from
deserializer)], properties:null)
INFO : Completed compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.393 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.109 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
database_name
default
1 row selected (1.374 seconds)
{code}
Also, on the hiveserver side, a runtime null pointer exception is thrown, and
the observation log throws no warnings or errors.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jira] [Updated] (HIVE-26211) "hive.server2.webui.max.historic.queries" should be avoided to be set too large, otherwise it will cause blocking
[
https://issues.apache.org/jira/browse/HIVE-26211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26211:
---
Description:
In hive-default.xml.template
{code:java}
hive.server2.webui.max.historic.queries
25
The maximum number of past queries to show in HiverSever2
WebUI.
{code}
Set hive.server2.webui.max.historic.queries to a relatively large value, take
2000 as an example, start hiveserver2, it can start hiveserver normally,
and logging without exception.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-09 20:03:41: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d{code}
Next, if you use beeline as a client to connect to hive and send a request for
database related operations, for example, if you query all the databases, after
successfully executing "show databases", beeline blocks and no other operations
can be performed.
{code:java}
liky@ljq1:/opt/hive$ beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline> !connect jdbc:hive2://192.168.1.194:1/default
Connecting to jdbc:hive2://192.168.1.194:1/default
Enter username for jdbc:hive2://192.168.1.194:1/default: hive
Enter password for jdbc:hive2://192.168.1.194:1/default: *
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.194:1/default> show databases
. . . . . . . . . . . . . . . . . . . . . .> ;
INFO : Compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from
deserializer)], properties:null)
INFO : Completed compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.393 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.109 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
database_name
default
1 row selected (1.374 seconds)
{code}
Also, on the hiveserver side, a runtime null pointer exception is thrown, and
the observation log throws no warnings or errors.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jira] [Updated] (HIVE-26211) "hive.server2.webui.max.historic.queries" should be avoided to be set too large, otherwise it will cause blocking
[
https://issues.apache.org/jira/browse/HIVE-26211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingxuan Fu updated HIVE-26211:
---
Description:
In hive-default.xml.template
{code:java}
hive.server2.webui.max.historic.queries
25
The maximum number of past queries to show in HiverSever2
WebUI.
{code}
Set hive.server2.webui.max.historic.queries to a relatively large value, take
2000 as an example, start hiveserver2, it can start hiveserver normally,
and logging without exception.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-09 20:03:41: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f
Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d{code}
Next, if you use beeline as a client to connect to hive and send a request for
database related operations, for example, if you query all the databases, after
successfully executing "show databases", beeline blocks and no other operations
can be performed.
{code:java}
liky@ljq1:/opt/hive$ beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 3.1.2 by Apache Hive
beeline> !connect jdbc:hive2://192.168.1.194:1/default
Connecting to jdbc:hive2://192.168.1.194:1/default
Enter username for jdbc:hive2://192.168.1.194:1/default: hive
Enter password for jdbc:hive2://192.168.1.194:1/default: *
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.194:1/default> show databases
. . . . . . . . . . . . . . . . . . . . . .> ;
INFO : Compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from
deserializer)], properties:null)
INFO : Completed compiling
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.393 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): show
databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); Time
taken: 0.109 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
database_name
default
1 row selected (1.374 seconds)
{code}
Also, on the hiveserver side, a runtime null pointer exception is thrown, and
the observation log throws no warnings or errors.
{code:java}
liky@ljq1:/usr/local/hive/conf$ hiveserver2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Error: Could not find or load main class
org.apache.hadoop.hbase.util.GetJavaProperty
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jira] [Assigned] (HIVE-26211) "hive.server2.webui.max.historic.queries" should be avoided to be set too large, otherwise it will cause blocking
[ https://issues.apache.org/jira/browse/HIVE-26211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingxuan Fu reassigned HIVE-26211: -- > "hive.server2.webui.max.historic.queries" should be avoided to be set too > large, otherwise it will cause blocking > - > > Key: HIVE-26211 > URL: https://issues.apache.org/jira/browse/HIVE-26211 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 > Environment: Hive 3.1.2 > os.name=Linux > os.arch=amd64 > os.version=5.4.0-72-generic > java.version=1.8.0_162 > java.vendor=Oracle Corporation >Reporter: Jingxuan Fu >Assignee: Jingxuan Fu >Priority: Major > > In hive-default.xml.template > > hive.server2.webui.max.historic.queries > 25 > The maximum number of past queries to show in HiverSever2 > WebUI. > > Set hive.server2.webui.max.historic.queries to a relatively large value, take > 2000 as an example, start hiveserver2, it can start hiveserver normally, > and logging without exception. > liky@ljq1:/usr/local/hive/conf$ hiveserver2 > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > 2022-05-09 20:03:41: Starting HiveServer2 > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > Hive Session ID = 0b419706-4026-4a8b-80fe-b79fecbccd4f > Hive Session ID = 0f9e28d7-0081-4b2f-a743-4093c38c152d > Next, if you use beeline as a client to connect to hive and send a request > for database related operations, for example, if you query all the databases, > after successfully executing "show databases", beeline blocks and no other > operations can be performed. > liky@ljq1:/opt/hive$ beeline > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > Beeline version 3.1.2 by Apache Hive > beeline> !connect jdbc:hive2://192.168.1.194:1/default > Connecting to jdbc:hive2://192.168.1.194:1/default > Enter username for jdbc:hive2://192.168.1.194:1/default: hive > Enter password for jdbc:hive2://192.168.1.194:1/default: * > Connected to: Apache Hive (version 3.1.2) > Driver: Hive JDBC (version 3.1.2) > Transaction isolation: TRANSACTION_REPEATABLE_READ > 0: jdbc:hive2://192.168.1.194:1/default> show databases > . . . . . . . . . . . . . . . . . . . . . .> ; > INFO : Compiling > command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): > show databases > INFO : Concurrency mode is disabled, not creating a lock manager > INFO : Semantic Analysis Completed (retrial = false) > INFO : Returning Hive schema: > Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, > comment:from deserializer)], properties:null) > INFO : Completed compiling > command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); > Time taken: 0.393 seconds > INFO : Concurrency mode is disabled, not creating a lock manager > INFO : Executing > command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b): > show databases > INFO : Starting task [Stage-0:DDL] in serial mode > INFO : Completed executing > command(queryId=liky_20220509202542_15382019-f07b-40ff-840d-1f720df77d8b); > Time taken: 0.109 seconds > INFO : OK > INFO : Concurrency mode is disabled, not creating a lock manager > ++ > | database_name | > ++ > | default | > ++ > 1 row selected (1.374 seconds) > Also, on the hiveserver side, a runtime null pointer exception is thrown, and > the
