[jira] [Created] (HIVE-26297) Refactoring ColumnStatsAggregator classes to reduce warnings

2022-06-07 Thread Alessandro Solimando (Jira) 
Alessandro Solimando created HIVE-26297:
---

 Summary: Refactoring ColumnStatsAggregator classes to reduce 
warnings
 Key: HIVE-26297
 URL: https://issues.apache.org/jira/browse/HIVE-26297
 Project: Hive
  Issue Type: Sub-task
  Components: Standalone Metastore, Statistics
Affects Versions: 4.0.0-alpha-2
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando


The interest of reducing warnings is to be able to focus on the important ones.

Some of the bugs fixed while writing unit-tests were highlighted as warnings 
(potential NPEs and rounding issues), but it was hard to see them among the 
many other (less severe) warnings.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HIVE-26296) RuntimeException when executing EXPLAIN CBO JOINCOST on query with JDBC tables

2022-06-07 Thread Stamatis Zampetakis (Jira) 
Stamatis Zampetakis created HIVE-26296:
--

 Summary: RuntimeException when executing EXPLAIN CBO JOINCOST on 
query with JDBC tables
 Key: HIVE-26296
 URL: https://issues.apache.org/jira/browse/HIVE-26296
 Project: Hive
  Issue Type: Bug
  Components: CBO, HiveServer2
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis


Consider a JDBC database with two tables _author_ and _book_.
{code:sql}
CREATE EXTERNAL TABLE author
(
id int,
fname varchar(20),
lname varchar(20)
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
...
"hive.sql.table" = "author"
);

CREATE EXTERNAL TABLE book
(
id int,
title varchar(100),
author int
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
"hive.sql.database.type" = "MYSQL",
"hive.sql.jdbc.driver" = "com.mysql.jdbc.Driver",
...
"hive.sql.table" = "book"
);
{code}

Executing an {{EXPLAIN CBO JOINCOST}} with a query joining two JDBC tables 
fails with {{RuntimeException}} while trying to compute the selectivity of the 
join.
{code:sql}
EXPLAIN CBO JOINCOST 
SELECT a.lname, b.title FROM author a JOIN book b ON a.id=b.author;
{code}

+Stacktrace+
{noformat}
java.lang.RuntimeException: Unexpected Join type: 
org.apache.calcite.adapter.jdbc.JdbcRules$JdbcJoin
at 
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdSelectivity.computeInnerJoinSelectivity(HiveRelMdSelectivity.java:156)
at 
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdSelectivity.getSelectivity(HiveRelMdSelectivity.java:68)
at GeneratedMetadataHandler_Selectivity.getSelectivity_$(Unknown Source)
at GeneratedMetadataHandler_Selectivity.getSelectivity(Unknown Source)
at 
org.apache.calcite.rel.metadata.RelMetadataQuery.getSelectivity(RelMetadataQuery.java:426)
at 
org.apache.calcite.rel.metadata.RelMdUtil.getJoinRowCount(RelMdUtil.java:736)
at 
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:195)
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
at 
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:212)
at 
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:140)
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
at 
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:212)
at 
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:191)
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
at 
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:212)
at 
org.apache.calcite.rel.externalize.RelWriterImpl.explain_(RelWriterImpl.java:100)
at 
org.apache.calcite.rel.externalize.RelWriterImpl.done(RelWriterImpl.java:144)
at 
org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:246)
at org.apache.calcite.plan.RelOptUtil.toString(RelOptUtil.java:2308)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:648)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12699)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:460)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:317)
at 
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:180)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:317)
at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:106)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:495)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:447)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:412)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:406)
at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:200)
at org.apache.hadoop

[jira] [Created] (HIVE-26295) Hive LB based on zookeeper occured some probability of connect failured

2022-06-07 Thread hansonhe (Jira) 
hansonhe created HIVE-26295:
---

 Summary: Hive LB based on zookeeper occured some probability of 
connect failured
 Key: HIVE-26295
 URL: https://issues.apache.org/jira/browse/HIVE-26295
 Project: Hive
  Issue Type: Bug
Reporter: hansonhe


(1) I set LB properties  in hive-site.xml
hive.server2.support.dynamic.service.discovery=true
hive.server2.active.passive.ha.enable=false
 (2) My hive production cluster info 
hive  version: Apache 3.1.2
hadoop version: Apache 3.1.4
zookeeper version: Apache 3.5.9
URL：  
jdbc:hive2://host1:2181,host2:2181,host3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
my hive cluster have 3 hiveserver2 :  host1:1,host2:1,host3:1
（3）After lots of test connect to use beeline -u '${URL}'  -n 'hive' -p ''  
, some probability of connect failured or connect success.When connect 
success，it can connect anyone of 3 hiveserver2 randomly and successfully. When 
connect failed，failed logs  as following :


22/06/07 11:14:59 INFO X509Util: Setting -D 
jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS 
renegotiation
22/06/07 11:14:59 INFO ClientCnxnSocket: jute.maxbuffer value is 4194304 Bytes
22/06/07 11:14:59 INFO ClientCnxn: zookeeper.request.timeout value is 0. 
feature enabled=
22/06/07 11:14:59 INFO ClientCnxn: Opening socket connection to server 
host3/10.2.52.73:2181. Will not attempt to authenticate using SASL (unknown 
error)
22/06/07 11:14:59 INFO ClientCnxn: Socket connection established, initiating 
session, client: /10.2.52.110:61210, server: 
sh1-bigdata-master03/10.2.52.73:2181
22/06/07 11:14:59 INFO ClientCnxn: Session establishment complete on server 
host3/10.2.52.73:2181, sessionid = 0x301b42644c8001c, negotiated timeout = 6
22/06/07 11:14:59 INFO ConnectionStateManager: State change: CONNECTED
22/06/07 11:14:59 INFO CuratorFrameworkImpl: backgroundOperationsLoop exiting
22/06/07 11:14:59 INFO ZooKeeper: Session: 0x301b42644c8001c closed
22/06/07 11:14:59 INFO ClientCnxn: EventThread shut down for session: 
0x301b42644c8001c
Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read 
HiveServer2 configs from ZooKeeper (state=,code=0)
Beeline version 3.1.2 by Apache Hive



--
This message was sent by Atlassian Jira
(v8.20.7#820007)