[jira] [Created] (HIVE-24459) Qtest to simulate query-based compaction
Karen Coppage created HIVE-24459: Summary: Qtest to simulate query-based compaction Key: HIVE-24459 URL: https://issues.apache.org/jira/browse/HIVE-24459 Project: Hive Issue Type: Test Reporter: Karen Coppage Assignee: Karen Coppage AFAIK all compaction tests run on a local filesystem, and none run on HDFS. Since HDFS and local filesystem behavior differs sometimes, it would be good to test query-based compaction on HDFS. Compaction threads don't run in the qtest environment so this qtest would simulate QB compaction by running the queries that QB compaction runs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24460) Refactor Get Next Event ID for DbNotificationListener
David Mollitor created HIVE-24460: - Summary: Refactor Get Next Event ID for DbNotificationListener Key: HIVE-24460 URL: https://issues.apache.org/jira/browse/HIVE-24460 Project: Hive Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor Refactor event ID generation to match notification log ID generation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24461) Provide CachedStore implementation for dataconnectors
Naveen Gangam created HIVE-24461: Summary: Provide CachedStore implementation for dataconnectors Key: HIVE-24461 URL: https://issues.apache.org/jira/browse/HIVE-24461 Project: Hive Issue Type: Sub-task Reporter: Naveen Gangam Currently, none of the connectors are cached. They are all delegated to the ObjectStore for every call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24462) JDBC: Support bearer token authentication
Gopal Vijayaraghavan created HIVE-24462: --- Summary: JDBC: Support bearer token authentication Key: HIVE-24462 URL: https://issues.apache.org/jira/browse/HIVE-24462 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 4.0.0 Reporter: Gopal Vijayaraghavan SPENGO authentication (Negotiate) authentication is the only way to authenticate a user without providing a password in JDBC. The SPN model for that fails when load-balancing is used (see HIVE-20583). Add a native JDBC equivalent for the Knox flow, but for POST requests with appropriate Authorization bearer tokens. https://knox.apache.org/books/knox-1-1-0/knoxsso_integration.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24463) Add special case for Derby and MySQL in Get Next ID DbNotificationListener
David Mollitor created HIVE-24463: - Summary: Add special case for Derby and MySQL in Get Next ID DbNotificationListener Key: HIVE-24463 URL: https://issues.apache.org/jira/browse/HIVE-24463 Project: Hive Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor * Derby does not support {{SELECT FOR UPDATE}} statements * MySQL can be optimized to use {{LAST_INSERT_ID()}} Debry tables are locked in other parts of the code already, but not in this path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24464) Evaluate the need to have directSQL implementation for data connectors
Naveen Gangam created HIVE-24464: Summary: Evaluate the need to have directSQL implementation for data connectors Key: HIVE-24464 URL: https://issues.apache.org/jira/browse/HIVE-24464 Project: Hive Issue Type: Sub-task Reporter: Naveen Gangam I expect that there will be just a handful of connectors not 100's of them like databases. But creating a placeholder item to evaluate at a future time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24465) RuntimeException Invalid Stats number of null > no of tuples when fetch column stats is enabled
Aman Sinha created HIVE-24465: - Summary: RuntimeException Invalid Stats number of null > no of tuples when fetch column stats is enabled Key: HIVE-24465 URL: https://issues.apache.org/jira/browse/HIVE-24465 Project: Hive Issue Type: Bug Components: Query Planning Reporter: Aman Sinha Assignee: Aman Sinha Seen in a downstream build. To reproduce, set hive.stats.fetch.column.stats to true and execute TPC-DS q8. The FilterSelectivityEstimator.java hits the following error: {noformat} RuntimeException: Invalid Stats number of null > no of tuples {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24466) insert queries should not launch job when condition in the query would output 0 rows
Rajesh Balamohan created HIVE-24466: --- Summary: insert queries should not launch job when condition in the query would output 0 rows Key: HIVE-24466 URL: https://issues.apache.org/jira/browse/HIVE-24466 Project: Hive Issue Type: Improvement Reporter: Rajesh Balamohan {noformat} -- This query would not generate any output and does not launch a job select * from tpcds_bin_partitioned_orc_3.store_sales where 1 = 2; -- This query generates a job (M -> R -> R) and runs for 30+ seconds in 2 node cluster to generate 0 rows. insert into table delete_orc_10.test_sales_1 select * from tpcds_bin_partitioned_orc_3.store_sales where 1 = 2; insert overwrite table delete_orc_10.test_sales_1 select * from tpcds_bin_partitioned_orc_3.store_sales where ss_sold_date_sk >=2450816+300 and ss_sold_date_sk <= (2450816+100); INFO : Status: Running (Executing on YARN cluster with App id application_1606875286859_0001) -- VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED -- Map 1 .. llap SUCCEEDED 1 100 0 0 Reducer 2 .. llap SUCCEEDED 2 200 0 5 Reducer 3 .. llap SUCCEEDED 2 200 0 9 -- VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 28.61 s -- INFO : Status: DAG finished successfully in 18.72 seconds INFO : INFO : Query Execution Summary INFO : -- INFO : OPERATIONDURATION INFO : -- INFO : Compile Query 14.06s INFO : Prepare Plan0.17s INFO : Get Query Coordinator (AM) 0.14s INFO : Submit Plan 0.03s INFO : Start DAG 0.05s INFO : Run DAG18.72s INFO : -- {noformat} It would be good to stop launching the job, when the condition is not valid in the query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24467) ConditionalTask remove other tasks exists thread safety problem
guojh created HIVE-24467: Summary: ConditionalTask remove other tasks exists thread safety problem Key: HIVE-24467 URL: https://issues.apache.org/jira/browse/HIVE-24467 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 2.3.4 Reporter: guojh When hive execute jobs in parallel(control by “hive.exec.parallel” parameter), ConditionalTasks run parallel to remove the tasks that not pick to run, because there are thread safety issues, some task may not remove from the dependent task tree. This is a very serious bug, which causes some stage task not trigger execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)