[jira] [Created] (HIVE-26013) Parquet predicate filters are not properly propogated to task configs at runtime
Rajesh Balamohan created HIVE-26013:
---
Summary: Parquet predicate filters are not properly propogated to
task configs at runtime
Key: HIVE-26013
URL: https://issues.apache.org/jira/browse/HIVE-26013
Project: Hive
Issue Type: Bug
Reporter: Rajesh Balamohan
Hive ParquetRecordReader sets the predicate filter in the config for parquet
libs to read.
Ref:
[https://github.com/apache/hive/blob/master/ql%2Fsrc%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhive%2Fql%2Fio%2Fparquet%2FParquetRecordReaderBase.java#L188]
{code:java}
ParquetInputFormat.setFilterPredicate(conf, p);
{code}
This internally sets {color:#FF}"parquet.private.read.filter.predicate"
{color}variable in config.
Ref:
[https://github.com/apache/parquet-mr/blob/master/parquet-hadoop%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fparquet%2Fhadoop%2FParquetInputFormat.java#L231]
Config set in compilation phase isn't visible at runtime for the tasks. This
causes filters to be lost and tasks run with excessive IO.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HIVE-26014) Remove redundant HushableRandomAccessFileAppender
Stamatis Zampetakis created HIVE-26014:
--
Summary: Remove redundant HushableRandomAccessFileAppender
Key: HIVE-26014
URL: https://issues.apache.org/jira/browse/HIVE-26014
Project: Hive
Issue Type: Improvement
Components: HiveServer2
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
[HushableRandomAccessFileAppender|https://github.com/apache/hive/blob/d3cd596aa15ebedd58f99628d43a03eb2f5f3909/ql/src/java/org/apache/hadoop/hive/ql/log/HushableRandomAccessFileAppender.java]
was introduced by HIVE-17826 to avoid exceptions originating from attempts to
write to a closed appender.
After the changes in HIVE-24590, the life-cycle (opening/closing/deleting) of
appenders is managed by the Log4j framework and not explicitly by Hive as it
used to be before. With HIVE-24590 in place, it is no longer possible to have
the exception in HIVE-17826 cause appenders are opened and closed when
necessary.
Due to the above, the {{HushableRandomAccessFileAppender}} is completely
redundant and can be removed in favor of the {{RandomAccessFileAppender}}
already provided by the Log4j framework.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HIVE-26015) HBase table with Ranger authentication fails; needs URLEncoding
Steve Carlin created HIVE-26015: --- Summary: HBase table with Ranger authentication fails; needs URLEncoding Key: HIVE-26015 URL: https://issues.apache.org/jira/browse/HIVE-26015 Project: Hive Issue Type: New Feature Reporter: Steve Carlin A Create table statement is failing for HBase going through Ranger. The stack trace shows a problem with the getURIForAuth method. The table is creating someting like this: CREATE EXTERNAL TABLE `mytesttbl`( `field1` string COMMENT 'from deserializer',`field2` string COMMENT 'from deserializer',`field3` string COMMENT 'from deserializer',`field4` string COMMENT 'from deserializer',`field5` string COMMENT 'from deserializer',`field6` int COMMENT 'from deserializer', `field7` string COMMENT 'from deserializer', `field8` int COMMENT 'from deserializer') ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( 'hbase.columns.mapping'=':key,field1,field2,field3,field4,field5#b,field6,cf:field7#b','serialization.format'='1') TBLPROPERTIES ( 'hbase.table.name'='mytesttbl'); Essentially, the SERDEPROPERTIES contain hash tabs which is causing a problem when creating a URI -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HIVE-26016) Remove duplicate table exists check in create_table_core api of HMSHandler
Yu Weiqiang created HIVE-26016: -- Summary: Remove duplicate table exists check in create_table_core api of HMSHandler Key: HIVE-26016 URL: https://issues.apache.org/jira/browse/HIVE-26016 Project: Hive Issue Type: Improvement Components: Hive Reporter: Yu Weiqiang Assignee: Yu Weiqiang Currently create_table_core() will check if table exists before opening transaction, it does not need to check again after opening transaction, can remove the unnecessary check code. -- This message was sent by Atlassian Jira (v8.20.1#820001)
