[jira] [Created] (HIVE-26013) Parquet predicate filters are not properly propogated to task configs at runtime
Rajesh Balamohan created HIVE-26013: --- Summary: Parquet predicate filters are not properly propogated to task configs at runtime Key: HIVE-26013 URL: https://issues.apache.org/jira/browse/HIVE-26013 Project: Hive Issue Type: Bug Reporter: Rajesh Balamohan Hive ParquetRecordReader sets the predicate filter in the config for parquet libs to read. Ref: [https://github.com/apache/hive/blob/master/ql%2Fsrc%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhive%2Fql%2Fio%2Fparquet%2FParquetRecordReaderBase.java#L188] {code:java} ParquetInputFormat.setFilterPredicate(conf, p); {code} This internally sets {color:#FF}"parquet.private.read.filter.predicate" {color}variable in config. Ref: [https://github.com/apache/parquet-mr/blob/master/parquet-hadoop%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fparquet%2Fhadoop%2FParquetInputFormat.java#L231] Config set in compilation phase isn't visible at runtime for the tasks. This causes filters to be lost and tasks run with excessive IO. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HIVE-26014) Remove redundant HushableRandomAccessFileAppender
Stamatis Zampetakis created HIVE-26014: -- Summary: Remove redundant HushableRandomAccessFileAppender Key: HIVE-26014 URL: https://issues.apache.org/jira/browse/HIVE-26014 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Stamatis Zampetakis Assignee: Stamatis Zampetakis [HushableRandomAccessFileAppender|https://github.com/apache/hive/blob/d3cd596aa15ebedd58f99628d43a03eb2f5f3909/ql/src/java/org/apache/hadoop/hive/ql/log/HushableRandomAccessFileAppender.java] was introduced by HIVE-17826 to avoid exceptions originating from attempts to write to a closed appender. After the changes in HIVE-24590, the life-cycle (opening/closing/deleting) of appenders is managed by the Log4j framework and not explicitly by Hive as it used to be before. With HIVE-24590 in place, it is no longer possible to have the exception in HIVE-17826 cause appenders are opened and closed when necessary. Due to the above, the {{HushableRandomAccessFileAppender}} is completely redundant and can be removed in favor of the {{RandomAccessFileAppender}} already provided by the Log4j framework. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HIVE-26015) HBase table with Ranger authentication fails; needs URLEncoding
Steve Carlin created HIVE-26015: --- Summary: HBase table with Ranger authentication fails; needs URLEncoding Key: HIVE-26015 URL: https://issues.apache.org/jira/browse/HIVE-26015 Project: Hive Issue Type: New Feature Reporter: Steve Carlin A Create table statement is failing for HBase going through Ranger. The stack trace shows a problem with the getURIForAuth method. The table is creating someting like this: CREATE EXTERNAL TABLE `mytesttbl`( `field1` string COMMENT 'from deserializer',`field2` string COMMENT 'from deserializer',`field3` string COMMENT 'from deserializer',`field4` string COMMENT 'from deserializer',`field5` string COMMENT 'from deserializer',`field6` int COMMENT 'from deserializer', `field7` string COMMENT 'from deserializer', `field8` int COMMENT 'from deserializer') ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ( 'hbase.columns.mapping'=':key,field1,field2,field3,field4,field5#b,field6,cf:field7#b','serialization.format'='1') TBLPROPERTIES ( 'hbase.table.name'='mytesttbl'); Essentially, the SERDEPROPERTIES contain hash tabs which is causing a problem when creating a URI -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HIVE-26016) Remove duplicate table exists check in create_table_core api of HMSHandler
Yu Weiqiang created HIVE-26016: -- Summary: Remove duplicate table exists check in create_table_core api of HMSHandler Key: HIVE-26016 URL: https://issues.apache.org/jira/browse/HIVE-26016 Project: Hive Issue Type: Improvement Components: Hive Reporter: Yu Weiqiang Assignee: Yu Weiqiang Currently create_table_core() will check if table exists before opening transaction, it does not need to check again after opening transaction, can remove the unnecessary check code. -- This message was sent by Atlassian Jira (v8.20.1#820001)