[hive] branch master updated (137cb91 -> dc5bb8b)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 137cb91 HIVE-24640:Add error message in hive proto logger for failed queries.(Harish Jaiprakash reviewed by Anishek Agarwal) add dc5bb8b disable tests No new revisions were added by this update. Summary of changes: .../apache/hadoop/hive/ql/parse/TestScheduledReplicationScenarios.java | 1 + ql/src/test/queries/clientnegative/dyn_part3.q | 1 + 2 files changed, 2 insertions(+)
[hive] branch master updated (7e7d46d -> c3fd3c2)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 7e7d46d HIVE-24673: addendum add c3fd3c2 HIVE-24221: Use vectorizable expression to combine multiple columns in semijoin bloom filters (#1544) (Stamatis Zampetakis reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hive/ql/optimizer/SemiJoinReductionMerge.java | 8 +++- .../org/apache/hadoop/hive/ql/parse/TezCompiler.java | 2 +- .../apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java | 18 ++ .../llap/dynamic_semijoin_reduction_2.q.out| 6 +++--- .../clientpositive/perf/tpcds30tb/tez/query17.q.out| 6 +++--- .../clientpositive/perf/tpcds30tb/tez/query25.q.out| 6 +++--- .../clientpositive/perf/tpcds30tb/tez/query29.q.out| 6 +++--- .../clientpositive/perf/tpcds30tb/tez/query50.q.out| 6 +++--- 8 files changed, 37 insertions(+), 21 deletions(-)
[hive] branch master updated (8e9d50c -> 7e7d46d)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 8e9d50c HIVE-23553: Upgrade ORC version to 1.6.7 (Panagiotis Garefalakis, reviewed by Mustafa Iman, Dongjoon Hyun, Jesus Camacho Rodriguez) add 7e7d46d HIVE-24673: addendum No new revisions were added by this update. Summary of changes: .../results/clientnegative/subquery_scalar_multi_rows.q.out | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-)
[hive] branch master updated (8ef7f09 -> 36a45b4)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 8ef7f09 HIVE-24353: Performance: do not throw exceptions when parsing Timestamp (Vincenz Priesnitz via David Mollitor, reviewed by Bodor Laszlo, Panagiotis Garefalakis) add 36a45b4 HIVE-24694: Early connection close to release server resources during creating (#1920) (Zhihua Deng reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../java/org/apache/hive/jdbc/HiveConnection.java | 61 +- 1 file changed, 48 insertions(+), 13 deletions(-)
[hive] branch master updated (014f084 -> 7281ab8)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 014f084 HIVE-18728 : Secure webHCat with SSL (#1699) (Oleksiy Sayankin reviewed by Zoltan Haindrich) add 7281ab8 HIVE-24295: Apply schema merge to all shared work optimizations (#1662) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: .../hive/ql/optimizer/SharedWorkOptimizer.java | 619 .../hive/ql/optimizer/TestSharedWorkOptimizer.java | 77 + .../results/clientpositive/llap/auto_join13.q.out | 26 +- .../results/clientpositive/llap/auto_join2.q.out | 34 +- .../results/clientpositive/llap/auto_join3.q.out | 24 +- .../results/clientpositive/llap/auto_join30.q.out | 44 +- .../results/clientpositive/llap/auto_join31.q.out | 22 +- .../llap/auto_join_without_localtask.q.out | 50 +- .../clientpositive/llap/bucket_map_join_tez2.q.out | 152 +- .../llap/cbo_SortUnionTransposeRule.q.out | 94 +- .../cbo_fallback_always_subquery_exception.q.out | 26 +- .../clientpositive/llap/constant_prop_3.q.out | 110 +- .../llap/correlationoptimizer2.q.out | 264 ++-- .../llap/correlationoptimizer3.q.out | 314 ++-- .../llap/correlationoptimizer6.q.out | 276 ++-- .../llap/create_view_disable_cbo.q.out |4 +- .../results/clientpositive/llap/cte_mat_6.q.out| 176 +-- .../llap/dynamic_partition_pruning_2.q.out | 78 +- .../llap/dynamic_semijoin_reduction_2.q.out| 94 +- .../results/clientpositive/llap/except_all.q.out |8 +- .../clientpositive/llap/except_distinct.q.out |4 +- .../clientpositive/llap/explainanalyze_2.q.out | 64 +- .../clientpositive/llap/explainuser_1.q.out| 94 +- .../clientpositive/llap/explainuser_2.q.out| 276 ++-- .../llap/external_jdbc_table_perf.q.out| 354 ++--- .../clientpositive/llap/filter_cond_pushdown.q.out | 92 +- .../llap/filter_join_breaktask.q.out | 64 +- .../clientpositive/llap/groupby_sort_1_23.q.out| 84 +- .../llap/groupby_sort_skew_1_23.q.out | 120 +- .../llap/hybridgrace_hashjoin_2.q.out | 128 +- .../clientpositive/llap/infer_join_preds.q.out | 24 +- .../test/results/clientpositive/llap/join13.q.out | 26 +- .../test/results/clientpositive/llap/join19.q.out | 48 +- .../test/results/clientpositive/llap/join2.q.out | 20 +- .../test/results/clientpositive/llap/join20.q.out |4 +- .../test/results/clientpositive/llap/join21.q.out | 20 +- .../test/results/clientpositive/llap/join3.q.out | 24 +- .../test/results/clientpositive/llap/join40.q.out |4 +- .../test/results/clientpositive/llap/join46.q.out | 22 +- .../clientpositive/llap/join_alt_syntax.q.out | 52 +- .../clientpositive/llap/join_cond_pushdown_1.q.out | 42 +- .../clientpositive/llap/join_cond_pushdown_3.q.out | 42 +- .../clientpositive/llap/join_filters_overlap.q.out | 84 +- .../results/clientpositive/llap/join_merging.q.out | 52 +- .../results/clientpositive/llap/keep_uniform.q.out |2 +- .../results/clientpositive/llap/leftsemijoin.q.out | 83 +- .../clientpositive/llap/limit_join_transpose.q.out | 562 +++ .../clientpositive/llap/mergejoins_mixed.q.out | 307 ++-- .../clientpositive/llap/multiMapJoin2.q.out| 185 ++- .../results/clientpositive/llap/ppd_join2.q.out| 48 +- .../results/clientpositive/llap/ppd_join3.q.out| 48 +- .../clientpositive/llap/ppd_outer_join4.q.out | 96 +- ql/src/test/results/clientpositive/llap/ptf.q.out | 212 +-- .../clientpositive/llap/ptf_streaming.q.out| 146 +- .../llap/reduce_deduplicate_extended2.q.out| 72 +- .../llap/runtime_skewjoin_mapjoin_spark.q.out | 46 +- .../results/clientpositive/llap/semijoin2.q.out| 86 +- .../results/clientpositive/llap/semijoin4.q.out| 132 +- .../results/clientpositive/llap/sharedwork.q.out | 224 +-- .../clientpositive/llap/sharedwork_semi.q.out | 58 +- .../clientpositive/llap/sharedworkresidual.q.out | 113 +- .../clientpositive/llap/skewjoin_mapjoin7.q.out| 70 +- .../clientpositive/llap/smb_mapjoin_17.q.out | 74 +- .../llap/special_character_in_tabnames_1.q.out | 420 +++--- .../special_character_in_tabnames_quotes_1.q.out | 420 +++--- .../results/clientpositive/llap/subquery_ALL.q.out | 84 +- .../results/clientpositive/llap/subquery_ANY.q.out | 22 +- .../llap/subquery_exists_having.q.out | 152 +- .../results/clientpositive/llap/subquery_in.q.out | 912 +-- .../clientpositive/llap/subquery_in_having.q.out | 226 +-- .../clientpositive/llap/subquery_multi.q.out | 1596 ++-- .../clientpositive/llap
[hive] branch master updated (0b40e85 -> 014f084)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 0b40e85 HIVE-24654: Table level replication support for Atlas metadata (Pravin Kumar Sinha, reviewed by Aasha Medhi) add 014f084 HIVE-18728 : Secure webHCat with SSL (#1699) (Oleksiy Sayankin reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../apache/hive/hcatalog/templeton/AppConfig.java | 5 +++ .../org/apache/hive/hcatalog/templeton/Main.java | 49 ++ 2 files changed, 54 insertions(+)
[hive] branch master updated (9256afb -> 2bd6a9d)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 9256afb HIVE-22944: Upgrade to Kryo5 (#1798) (Laszlo Bodor reviewed by Panagiotis Garefalakis) add 2bd6a9d HIVE-24525: Invite reviewers automatically by file name patterns (#1767) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: .github/assign-by-files.yml | 50 +++ .github/workflows/auto-assign.yml | 12 ++ 2 files changed, 62 insertions(+) create mode 100644 .github/assign-by-files.yml create mode 100644 .github/workflows/auto-assign.yml
[hive] branch HIVE-24525 updated: add error to jenkinsfile
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch HIVE-24525
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/HIVE-24525 by this push:
new 3e80817 add error to jenkinsfile
3e80817 is described below
commit 3e808170f9dec6a25f69095c8b6547bbe433b9f2
Author: Zoltan Haindrich
AuthorDate: Mon Feb 1 14:55:20 2021 +
add error to jenkinsfile
---
Jenkinsfile | 1 +
1 file changed, 1 insertion(+)
diff --git a/Jenkinsfile b/Jenkinsfile
index d9263e4..77bf272 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -1,3 +1,4 @@
+err{{
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
[hive] branch HIVE-24525 updated: clenaup
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a commit to branch HIVE-24525 in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/HIVE-24525 by this push: new bb4e964 clenaup bb4e964 is described below commit bb4e9648d8311d0c5e7647cea1cd1913f4d4554f Author: Zoltan Haindrich AuthorDate: Mon Feb 1 14:54:49 2021 + clenaup --- .github/assign-by-files.yml | 42 -- 1 file changed, 42 deletions(-) diff --git a/.github/assign-by-files.yml b/.github/assign-by-files.yml index 02faf1b..1df5287 100644 --- a/.github/assign-by-files.yml +++ b/.github/assign-by-files.yml @@ -1,47 +1,5 @@ --- # thrift changes -'**/*.thrift': - - kgyrtkirk -# parser changes -'**/*.g': - - kgyrtkirk -# metastore database changes -'**/package.jdo': - - kgyrtkirk -# these will match on various scheduled query -'**/schq/**': - - kgyrtkirk -'**/*Scheduled*': - - kgyrtkirk -# these will match sketches related stuff -'**/*[sS]ketches*': - - kgyrtkirk -# Build system 'Jenkinsfile': - kgyrtkirk -'.github/**': - - kgyrtkirk - - HiveQA -# ddl changes -'**/ddl/**': - - miklosgergely -# driver related changes -'**/ql/*@(Driver|Compiler|Executor)*.java': - - miklosgergely -# schematool changes -'**/schematool/**': - - miklosgergely -# metatool changes -'**/metatool/**': - - miklosgergely -'**/tez/**/*.java': - - abstractdog -'**/*Tez*java': - abstractdog -'**/*TopNKey*java': - - kasakrisz -'**/*CardinalityPreserving*java': - - kasakrisz -# query optimizer -'**/ql/optimizer/**': - - jcamachor
[hive] branch master updated (c3fdc2c -> 47bc287)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from c3fdc2c HIVE-24691: Ban commons-logging (again) (#1916) (Zoltan Matyus reviewed by Zoltan Haindrich) add 47bc287 HIVE-23485: Bound GroupByOperator stats using largest NDV among columns (#1926) (Stamatis Zampetakis reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../clientpositive/udaf_example_group_concat.q.out |8 +- .../src/test/results/positive/hbase_queries.q.out | 16 +- .../results/positive/kudu_complex_queries.q.out| 16 +- .../stats/annotation/StatsRulesProcFactory.java|9 +- .../results/clientpositive/cbo_rp_auto_join1.q.out | 20 +- .../results/clientpositive/llap/acid_stats5.q.out | 10 +- .../llap/acid_vectorization_original.q.out | 12 +- .../llap/annotate_stats_groupby.q.out |8 +- .../clientpositive/llap/annotate_stats_join.q.out |8 +- .../results/clientpositive/llap/antijoin.q.out | 102 +- .../results/clientpositive/llap/auto_join18.q.out | 24 +- .../llap/auto_join18_multi_distinct.q.out | 18 +- .../results/clientpositive/llap/auto_join27.q.out | 12 +- .../llap/auto_sortmerge_join_10.q.out | 10 +- .../clientpositive/llap/bucket_groupby.q.out | 112 +- .../clientpositive/llap/bucket_map_join_tez2.q.out | 74 +- .../llap/cbo_rp_annotate_stats_groupby.q.out |8 +- .../llap/cbo_rp_cross_product_check_2.q.out| 14 +- .../llap/cbo_rp_gby2_map_multi_distinct.q.out | 24 +- .../clientpositive/llap/check_constraint.q.out | 12 +- .../llap/column_table_stats_orc.q.out | 10 +- .../llap/constprog_partitioner.q.out | 12 +- .../clientpositive/llap/constprog_semijoin.q.out |6 +- .../llap/constraints_optimization.q.out| 104 +- .../llap/correlationoptimizer1.q.out | 36 +- .../llap/correlationoptimizer10.q.out | 44 +- .../llap/correlationoptimizer13.q.out | 22 +- .../llap/correlationoptimizer14.q.out | 28 +- .../llap/correlationoptimizer2.q.out | 432 +++--- .../llap/correlationoptimizer6.q.out | 60 +- .../llap/correlationoptimizer8.q.out | 142 +- .../llap/correlationoptimizer9.q.out | 36 +- .../clientpositive/llap/count_dist_rewrite.q.out | 74 +- .../results/clientpositive/llap/cross_prod_1.q.out | 20 +- .../llap/cross_product_check_1.q.out | 38 +- .../llap/cross_product_check_2.q.out | 14 +- .../results/clientpositive/llap/ctas_colname.q.out | 40 +- .../results/clientpositive/llap/cte_mat_6.q.out| 12 +- .../clientpositive/llap/distinct_groupby.q.out | 208 +-- .../clientpositive/llap/distinct_stats.q.out | 32 +- .../llap/dynamic_partition_pruning.q.out | 12 +- .../llap/dynamic_semijoin_reduction_2.q.out| 28 +- .../llap/dynpart_sort_opt_vectorization.q.out | 202 +-- .../llap/dynpart_sort_optimization.q.out | 48 +- .../llap/dynpart_sort_optimization2.q.out | 40 +- .../dynpart_sort_optimization_distribute_by.q.out |6 +- .../llap/enforce_constraint_notnull.q.out | 24 +- .../results/clientpositive/llap/except_all.q.out | 150 +-- .../clientpositive/llap/except_distinct.q.out | 150 +-- .../clientpositive/llap/explainuser_1.q.out| 238 ++-- .../clientpositive/llap/explainuser_2.q.out| 102 +- .../clientpositive/llap/explainuser_4.q.out|6 +- .../results/clientpositive/llap/filter_aggr.q.out | 12 +- .../results/clientpositive/llap/filter_union.q.out | 56 +- .../llap/fold_eq_with_case_when.q.out | 12 +- .../results/clientpositive/llap/gby_star.q.out | 44 +- .../clientpositive/llap/groupby1_limit.q.out |8 +- .../results/clientpositive/llap/groupby1_map.q.out | 14 +- .../clientpositive/llap/groupby1_map_nomap.q.out | 14 +- .../clientpositive/llap/groupby1_map_skew.q.out| 18 +- .../clientpositive/llap/groupby2_limit.q.out | 12 +- .../results/clientpositive/llap/groupby2_map.q.out | 12 +- .../llap/groupby2_map_multi_distinct.q.out | 24 +- .../clientpositive/llap/groupby2_map_skew.q.out| 16 +- .../results/clientpositive/llap/groupby3_map.q.out |6 +- .../llap/groupby3_map_multi_distinct.q.out |6 +- .../clientpositive/llap/groupby3_map_skew.q.out|6 +- .../results/clientpositive/llap/groupby6_map.q.out | 12 +- .../clientpositive/llap/groupby6_map_skew.q.out| 16 +- .../results/clientpositive/llap/groupby7_map.q.out | 28 +- .../clientpositive/llap/groupby7_map_skew.q.out| 36 +- .../clientpositive/llap/groupby8_map_skew.q.out| 32 +- .../results
[hive] 01/01: changes-so-far
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch HIVE-24525
in repository https://gitbox.apache.org/repos/asf/hive.git
commit 210e8f027cec29a914de8606f771640d415dde93
Author: Zoltan Haindrich
AuthorDate: Fri Jan 29 17:40:43 2021 +
changes-so-far
---
.github/assign-by-files.yml | 47 +++
.github/workflows/auto-assign.yml | 12 ++
2 files changed, 59 insertions(+)
diff --git a/.github/assign-by-files.yml b/.github/assign-by-files.yml
new file mode 100644
index 000..02faf1b
--- /dev/null
+++ b/.github/assign-by-files.yml
@@ -0,0 +1,47 @@
+---
+# thrift changes
+'**/*.thrift':
+ - kgyrtkirk
+# parser changes
+'**/*.g':
+ - kgyrtkirk
+# metastore database changes
+'**/package.jdo':
+ - kgyrtkirk
+# these will match on various scheduled query
+'**/schq/**':
+ - kgyrtkirk
+'**/*Scheduled*':
+ - kgyrtkirk
+# these will match sketches related stuff
+'**/*[sS]ketches*':
+ - kgyrtkirk
+# Build system
+'Jenkinsfile':
+ - kgyrtkirk
+'.github/**':
+ - kgyrtkirk
+ - HiveQA
+# ddl changes
+'**/ddl/**':
+ - miklosgergely
+# driver related changes
+'**/ql/*@(Driver|Compiler|Executor)*.java':
+ - miklosgergely
+# schematool changes
+'**/schematool/**':
+ - miklosgergely
+# metatool changes
+'**/metatool/**':
+ - miklosgergely
+'**/tez/**/*.java':
+ - abstractdog
+'**/*Tez*java':
+ - abstractdog
+'**/*TopNKey*java':
+ - kasakrisz
+'**/*CardinalityPreserving*java':
+ - kasakrisz
+# query optimizer
+'**/ql/optimizer/**':
+ - jcamachor
diff --git a/.github/workflows/auto-assign.yml
b/.github/workflows/auto-assign.yml
new file mode 100644
index 000..5ebcb6d
--- /dev/null
+++ b/.github/workflows/auto-assign.yml
@@ -0,0 +1,12 @@
+name: "Auto Assign"
+on:
+ - pull_request_target
+
+jobs:
+ assign_reviewer:
+runs-on: ubuntu-latest
+steps:
+- uses: shufo/auto-assign-reviewer-by-files@v1.1.1
+ with:
+config: '.github/assign-by-files.yml'
+token: ${{ secrets.GITHUB_TOKEN }}
[hive] branch HIVE-24525 created (now 210e8f0)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch HIVE-24525 in repository https://gitbox.apache.org/repos/asf/hive.git. at 210e8f0 changes-so-far This branch includes the following new commits: new 210e8f0 changes-so-far The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[hive] branch branch-3.1 updated: HIVE-24636: Memory leak due LogFactory retaining ClassLoaders (#1924) (Zoltan Matyus reviewed by Karen Coppage)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3.1 by this push:
new 75d69cc HIVE-24636: Memory leak due LogFactory retaining ClassLoaders
(#1924) (Zoltan Matyus reviewed by Karen Coppage)
75d69cc is described below
commit 75d69cc08b890dab5aa6115d54213baa094b239d
Author: zmatyus <47740689+zmat...@users.noreply.github.com>
AuthorDate: Fri Jan 29 17:59:17 2021 +0100
HIVE-24636: Memory leak due LogFactory retaining ClassLoaders (#1924)
(Zoltan Matyus reviewed by Karen Coppage)
---
common/src/java/org/apache/hadoop/hive/common/JavaUtils.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/common/src/java/org/apache/hadoop/hive/common/JavaUtils.java
b/common/src/java/org/apache/hadoop/hive/common/JavaUtils.java
index c011cd1..54d6b65 100644
--- a/common/src/java/org/apache/hadoop/hive/common/JavaUtils.java
+++ b/common/src/java/org/apache/hadoop/hive/common/JavaUtils.java
@@ -28,7 +28,7 @@ import java.net.URLClassLoader;
import java.util.Arrays;
import java.util.List;
-
+import org.apache.commons.logging.LogFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -138,6 +138,7 @@ public final class JavaUtils {
newOutputStream.close();
}
}
+ LogFactory.release(loader);
}
/**
[hive] branch master updated (76f3bf1 -> 789e2fa)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 76f3bf1 HIVE-23728: Run metastore verification tests during precommit (#1154) (Zoltan Haindrich reviewed by Krisztian Kasa) add 789e2fa HIVE-24458: Add a method to provide a non-normalized ExpressionTree (#1726) (Owen O'Malley reviewed by Panos Garefalakis, Zoltan Haindrich) No new revisions were added by this update. Summary of changes: pom.xml| 2 +- .../orc/TestVectorizedOrcAcidRowBatchReader.java | 4 +- .../hive/ql/io/sarg/TestConvertAstToSearchArg.java | 32 ++-- .../hive/ql/io/sarg/TestSearchArgumentImpl.java| 121 +++--- standalone-metastore/pom.xml | 2 +- storage-api/pom.xml| 2 +- .../org/apache/hadoop/hive/common/io/CacheTag.java | 2 +- .../hadoop/hive/common/type/CalendarUtils.java | 2 +- .../hadoop/hive/ql/io/sarg/ExpressionTree.java | 74 + .../hadoop/hive/ql/io/sarg/PredicateLeaf.java | 7 + .../hadoop/hive/ql/io/sarg/SearchArgument.java | 12 +- .../hadoop/hive/ql/io/sarg/SearchArgumentImpl.java | 179 + .../hive/ql/io/sarg/TestSearchArgumentFactory.java | 178 13 files changed, 407 insertions(+), 210 deletions(-) create mode 100644 storage-api/src/test/org/apache/hadoop/hive/ql/io/sarg/TestSearchArgumentFactory.java
[hive] branch master updated (8d87112 -> 76f3bf1)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.
from 8d87112 HIVE-24392: Add table id and getFileMetadata to
GetPartitionsByNamesRequest (#1909) (Yongzhi Chen, reviewed by reviewed by
Kishen Das)
add 76f3bf1 HIVE-23728: Run metastore verification tests during precommit
(#1154) (Zoltan Haindrich reviewed by Krisztian Kasa)
No new revisions were added by this update.
Summary of changes:
Jenkinsfile| 19 +++---
standalone-metastore/DEV-README| 3 ++-
.../{ITestSqlServer.java => ITestDerby.java} | 6 ++---
.../{ITestSqlServer.java => ITestMssql.java} | 2 +-
.../metastore/dbinstall/rules/DatabaseRule.java| 13 +-
.../hive/metastore/dbinstall/rules/Derby.java | 30 +-
.../hive/metastore/dbinstall/rules/Mssql.java | 5 ++--
.../hive/metastore/dbinstall/rules/Mysql.java | 9 +--
.../hive/metastore/dbinstall/rules/Oracle.java | 4 +--
.../hive/metastore/dbinstall/rules/Postgres.java | 4 +--
storage-api/pom.xml| 6 +
upgrade-acid/pom.xml | 8 --
upgrade-acid/pre-upgrade/pom.xml | 17
13 files changed, 96 insertions(+), 30 deletions(-)
copy
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/dbinstall/{ITestSqlServer.java
=> ITestDerby.java} (86%)
rename
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/dbinstall/{ITestSqlServer.java
=> ITestMssql.java} (95%)
[hive] branch master updated (1781615 -> 170d5b4)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 1781615 HIVE-24661: Do Not Stringify Exception in Logger messages (David Mollitor reviewed by Miklos Gergely) add 170d5b4 HIVE-24601: Control CBO fallback behavior via property (#1875) (Stamatis Zampetakis reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hive/conf/HiveConf.java | 7 ++ data/conf/hive-site.xml| 5 ++ data/conf/llap/hive-site.xml | 5 ++ data/conf/tez/hive-site.xml| 5 ++ .../hadoop/hive/ql/parse/CBOFallbackStrategy.java | 76 ++ .../hadoop/hive/ql/parse/CalcitePlanner.java | 21 ++ .../cbo_fallback_conservative_subquery_exception.q | 6 ++ .../cbo_fallback_never_semantic_exception.q| 5 ++ .../cbo_fallback_never_subquery_exception.q| 5 ++ .../cbo_fallback_strategy_illegal_00.q | 2 + .../cbo_fallback_strategy_illegal_01.q | 2 + .../cbo_fallback_test_subquery_exception.q | 6 ++ .../cbo_fallback_always_semantic_exception.q | 6 ++ .../cbo_fallback_always_subquery_exception.q | 8 +++ .../cbo_fallback_conservative_semantic_exception.q | 6 ++ .../cbo_fallback_test_semantic_exception.q | 6 ++ ..._fallback_conservative_subquery_exception.q.out | 1 + .../cbo_fallback_never_semantic_exception.q.out| 1 + .../cbo_fallback_never_subquery_exception.q.out| 1 + .../cbo_fallback_strategy_illegal_00.q.out | 1 + .../cbo_fallback_strategy_illegal_01.q.out | 1 + .../cbo_fallback_test_subquery_exception.q.out | 1 + .../clientnegative/insert_into_with_schema2.q.out | 2 +- .../clientnegative/subquery_all_equal.q.out| 2 +- .../clientnegative/subquery_any_aggregate.q.out| 2 +- .../clientnegative/subquery_any_notequal.q.out | 2 +- .../clientnegative/subquery_corr_from.q.out| 2 +- .../clientnegative/subquery_corr_grandparent.q.out | 2 +- .../clientnegative/subquery_corr_select.q.out | 2 +- .../subquery_correlated_grand_parent.q.out | 2 +- .../subquery_exists_implicit_gby.q.out | 2 +- .../clientnegative/subquery_in_groupby.q.out | 2 +- .../results/clientnegative/subquery_in_lhs.q.out | 2 +- .../results/clientnegative/subquery_in_on.q.out| 2 +- .../subquery_multiple_cols_in_select.q.out | 2 +- .../subquery_non_exisiting_column.q.out| 2 +- .../subquery_notexists_implicit_gby.q.out | 2 +- .../subquery_scalar_multi_columns.q.out| 2 +- .../clientnegative/subquery_select_aggregate.q.out | 2 +- .../clientnegative/subquery_select_distinct.q.out | 2 +- .../clientnegative/subquery_select_distinct2.q.out | 2 +- .../subquery_subquery_chain_exists.q.out | 2 +- .../clientnegative/subquery_windowing_corr.q.out | 2 +- .../cbo_fallback_always_semantic_exception.q.out | 31 + .../cbo_fallback_always_subquery_exception.q.out | 46 + ..._fallback_conservative_semantic_exception.q.out | 31 + .../cbo_fallback_test_semantic_exception.q.out | 31 + 47 files changed, 321 insertions(+), 37 deletions(-) create mode 100644 ql/src/java/org/apache/hadoop/hive/ql/parse/CBOFallbackStrategy.java create mode 100644 ql/src/test/queries/clientnegative/cbo_fallback_conservative_subquery_exception.q create mode 100644 ql/src/test/queries/clientnegative/cbo_fallback_never_semantic_exception.q create mode 100644 ql/src/test/queries/clientnegative/cbo_fallback_never_subquery_exception.q create mode 100644 ql/src/test/queries/clientnegative/cbo_fallback_strategy_illegal_00.q create mode 100644 ql/src/test/queries/clientnegative/cbo_fallback_strategy_illegal_01.q create mode 100644 ql/src/test/queries/clientnegative/cbo_fallback_test_subquery_exception.q create mode 100644 ql/src/test/queries/clientpositive/cbo_fallback_always_semantic_exception.q create mode 100644 ql/src/test/queries/clientpositive/cbo_fallback_always_subquery_exception.q create mode 100644 ql/src/test/queries/clientpositive/cbo_fallback_conservative_semantic_exception.q create mode 100644 ql/src/test/queries/clientpositive/cbo_fallback_test_semantic_exception.q create mode 100644 ql/src/test/results/clientnegative/cbo_fallback_conservative_subquery_exception.q.out create mode 100644 ql/src/test/results/clientnegative/cbo_fallback_never_semantic_exception.q.out create mode 100644 ql/src/test/results/clientnegative/cbo_fallback_never_subquery_exception.q.out create mode 100644 ql/src/test/results/clientnegative/cbo_fallback_strategy_illegal_00.q.out create mode 100644 ql/src/test/results/clientnegative/cbo_fallback_strategy_illegal_01.q.out create mode 100644 ql/src
[hive] branch master updated (edefb96 -> 323ee0a)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from edefb96 HIVE-24564: Extend PPD filter transitivity to be able to discover new opportunities (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) add 323ee0a HIVE-24678: Add feature toggle to control SWO parallel edge support (#1912) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hive/conf/HiveConf.java | 2 ++ .../hive/ql/optimizer/SharedWorkOptimizer.java | 34 +- 2 files changed, 22 insertions(+), 14 deletions(-)
[hive] branch master updated (1c0969f -> 338f38d)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 1c0969f HIVE-24675: Handle external table replication for HA with same NS and… (Arko Sharma, reviewed by Aasha ) add 338f38d HIVE-24345: Re-enable flaky test: TestCleanupService#testEventualCleanupService_finishesCleanupBeforeExit (#1908) (Mustafa Iman reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../test/org/apache/hadoop/hive/ql/cleanup/TestCleanupService.java | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-)
[hive] branch master updated (fa987bd -> ee4e1f3)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.
from fa987bd HIVE-24669: Improve FileSystem usage in
Hive::loadPartitionInternal (Peter Varga, reviewed by Denys Kuzmenko)
add ee4e1f3 HIVE-24569: LLAP daemon leaks file descriptors/log4j
appenders (#1858) (Stamatis Zampetakis reviewed by Prasanth Jayachandran)
No new revisions were added by this update.
Summary of changes:
llap-server/pom.xml| 6 +
.../hive/llap/daemon/impl/LlapConstants.java | 7 -
.../hadoop/hive/llap/daemon/impl/QueryTracker.java | 7 +-
.../llap/log/LlapRandomAccessFileAppender.java | 185 +
.../llap/log/LlapRoutingAppenderPurgePolicy.java | 128
.../hadoop/hive/llap/log/LlapWrappedAppender.java | 221 -
.../hive/llap/log/Log4jQueryCompleteMarker.java| 2 +-
.../main/resources/llap-daemon-log4j2.properties | 20 +-
.../hive/llap/daemon/LlapDaemonExtension.java | 98 +
.../hive/llap/daemon/LlapDaemonTestUtils.java | 62 ++
.../llap/daemon/impl/TestLlapDaemonLogging.java| 151 ++
.../llap-daemon-routing-log4j2.properties} | 38 ++--
.../junit/extensions/DoNothingTCPServer.java | 58 ++
.../extensions/DoNothingTCPServerExtension.java| 67 +++
.../TestDoNothingTCPServerExtension.java | 58 ++
15 files changed, 715 insertions(+), 393 deletions(-)
create mode 100644
llap-server/src/java/org/apache/hadoop/hive/llap/log/LlapRandomAccessFileAppender.java
delete mode 100644
llap-server/src/java/org/apache/hadoop/hive/llap/log/LlapRoutingAppenderPurgePolicy.java
delete mode 100644
llap-server/src/java/org/apache/hadoop/hive/llap/log/LlapWrappedAppender.java
create mode 100644
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/LlapDaemonExtension.java
create mode 100644
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TestLlapDaemonLogging.java
copy llap-server/src/{main/resources/llap-daemon-log4j2.properties =>
test/resources/llap-daemon-routing-log4j2.properties} (81%)
create mode 100644
testutils/src/java/org/apache/hive/testutils/junit/extensions/DoNothingTCPServer.java
create mode 100644
testutils/src/java/org/apache/hive/testutils/junit/extensions/DoNothingTCPServerExtension.java
create mode 100644
testutils/src/test/java/org/apache/hive/testutils/junit/extensions/TestDoNothingTCPServerExtension.java
[hive] branch master updated (bd4e53c -> b3803a3)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from bd4e53c HIVE-24670: DeleteReaderValue should not allocate empty vectors for delete delta files (Adam Szita, reviewed by Peter Vary) add b3803a3 HIVE-24637 make Tez progress log interval configurable (#1870) (Johan Gustavsson reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java | 10 -- .../hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java | 4 2 files changed, 12 insertions(+), 2 deletions(-)
[hive] branch master updated (0e72217 -> d8034c8)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 0e72217 HIVE-24671: Semijoinremoval should not run into an NPE in case the SJ filter contains an UDF (Zoltan Haindrich, reviewed by Krisztian Kasa) add d8034c8 HIVE-24411: Make ThreadPoolExecutorWithOomHook more awareness of OutOfMemoryError (#1695) (Zhihua Deng reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hive/ql/hooks/HiveHooks.java | 2 +- .../hive/service/cli/operation/SQLOperation.java | 4 +- .../cli/thrift/ThreadPoolExecutorWithOomHook.java | 12 +++- .../thrift/TestThreadPoolExecutorWithOomHook.java | 70 ++ 4 files changed, 84 insertions(+), 4 deletions(-) create mode 100644 service/src/test/org/apache/hive/service/cli/thrift/TestThreadPoolExecutorWithOomHook.java
[hive] branch branch-3 updated: HIVE-22981: DataFileReader is not closed in AvroGenericRecordReader#extractWriterTimezoneFromMetadata (Karen Coppage via Marta Kuczora) (ported to branch-3 by Łukasz Os
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch branch-3
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3 by this push:
new 7d1e70d HIVE-22981: DataFileReader is not closed in
AvroGenericRecordReader#extractWriterTimezoneFromMetadata (Karen Coppage via
Marta Kuczora) (ported to branch-3 by Łukasz Osipiuk) (#1686)
7d1e70d is described below
commit 7d1e70d66d083ccafb54c9c9c09361dea54503c7
Author: Łukasz Osipiuk
AuthorDate: Mon Jan 25 13:27:37 2021 +0100
HIVE-22981: DataFileReader is not closed in
AvroGenericRecordReader#extractWriterTimezoneFromMetadata (Karen Coppage via
Marta Kuczora) (ported to branch-3 by Łukasz Osipiuk) (#1686)
---
.../apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java| 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
b/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
index 3159c5b..485337e 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
@@ -152,13 +152,12 @@ public class AvroGenericRecordReader implements
}
private ZoneId extractWriterTimezoneFromMetadata(JobConf job, FileSplit
split,
- GenericDatumReader gdr) throws IOException {
+ GenericDatumReader gdr) {
if (job == null || gdr == null || split == null || split.getPath() ==
null) {
return null;
}
-try {
- DataFileReader dataFileReader =
- new DataFileReader(new FsInput(split.getPath(), job),
gdr);
+try (DataFileReader dataFileReader = new
DataFileReader(
+new FsInput(split.getPath(), job), gdr)) {
if (dataFileReader.getMeta(AvroSerDe.WRITER_TIME_ZONE) != null) {
try {
return ZoneId.of(new
String(dataFileReader.getMeta(AvroSerDe.WRITER_TIME_ZONE),
@@ -169,6 +168,7 @@ public class AvroGenericRecordReader implements
}
} catch (IOException e) {
// Can't access metadata, carry on.
+ LOG.debug(e.getMessage(), e);
}
return null;
}
[hive] branch branch-3.1 updated: HIVE-22981: DataFileReader is not closed in AvroGenericRecordReader#extractWriterTimezoneFromMetadata (Karen Coppage via Marta Kuczora) (ported to branch-3 by Łukasz
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-3.1 by this push:
new cf72bd8 HIVE-22981: DataFileReader is not closed in
AvroGenericRecordReader#extractWriterTimezoneFromMetadata (Karen Coppage via
Marta Kuczora) (ported to branch-3 by Łukasz Osipiuk) (#1686)
cf72bd8 is described below
commit cf72bd8d86598b3256cb6d985168fe690dbd63bc
Author: Łukasz Osipiuk
AuthorDate: Mon Jan 25 13:27:37 2021 +0100
HIVE-22981: DataFileReader is not closed in
AvroGenericRecordReader#extractWriterTimezoneFromMetadata (Karen Coppage via
Marta Kuczora) (ported to branch-3 by Łukasz Osipiuk) (#1686)
---
.../apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java| 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
b/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
index 3159c5b..485337e 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java
@@ -152,13 +152,12 @@ public class AvroGenericRecordReader implements
}
private ZoneId extractWriterTimezoneFromMetadata(JobConf job, FileSplit
split,
- GenericDatumReader gdr) throws IOException {
+ GenericDatumReader gdr) {
if (job == null || gdr == null || split == null || split.getPath() ==
null) {
return null;
}
-try {
- DataFileReader dataFileReader =
- new DataFileReader(new FsInput(split.getPath(), job),
gdr);
+try (DataFileReader dataFileReader = new
DataFileReader(
+new FsInput(split.getPath(), job), gdr)) {
if (dataFileReader.getMeta(AvroSerDe.WRITER_TIME_ZONE) != null) {
try {
return ZoneId.of(new
String(dataFileReader.getMeta(AvroSerDe.WRITER_TIME_ZONE),
@@ -169,6 +168,7 @@ public class AvroGenericRecordReader implements
}
} catch (IOException e) {
// Can't access metadata, carry on.
+ LOG.debug(e.getMessage(), e);
}
return null;
}
[hive] branch master updated (66ec17c -> 5338805)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 66ec17c HIVE-24680: Disable flaky unionDistinct_1.q test (#1904) add 5338805 HIVE-23546: Skip authorization when user is a superuser (#1033) (Zhihua Deng reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hive/conf/HiveConf.java | 3 +++ .../org/apache/hive/jdbc/TestRestrictedList.java | 1 + .../hive/service/cli/session/TestQueryDisplay.java | 3 +-- .../java/org/apache/hadoop/hive/ql/Compiler.java | 4 +-- .../hadoop/hive/ql/parse/BaseSemanticAnalyzer.java | 14 ++ .../hive/ql/parse/ExplainSemanticAnalyzer.java | 6 - .../hadoop/hive/ql/parse/SemanticAnalyzer.java | 4 +-- .../hadoop/hive/ql/parse/TestSemanticAnalyzer.java | 31 ++ 8 files changed, 57 insertions(+), 9 deletions(-)
[hive] branch master updated (5d735c2 -> 57f76df)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 5d735c2 HIVE-18284: Fix NPE when inserting data with 'distribute by' clause with dynpart sort optimization (#1400) (Syed Shameerur Rahman reviewed by Zoltan Haindrich) add 57f76df HIVE-24677: Fix typoed vectorization package declaration (#1900) (Zoltan Haindrich reviewed by Miklos Gergely and Panos Garefalakis) No new revisions were added by this update. Summary of changes: .../gen/vectorization/UDAFTemplates/VectorUDAFComputeBitVector.txt | 3 ++- .../hadoop/hive/ql/udf/generic/GenericUDAFComputeBitVectorHLL.java | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-)
[hive] branch master updated (baabe8d -> 5d735c2)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from baabe8d HIVE-24660: Remove Commons Logger from jdbc-handler Package (David Mollitor reviewed by Miklos Gergely) add 5d735c2 HIVE-18284: Fix NPE when inserting data with 'distribute by' clause with dynpart sort optimization (#1400) (Syed Shameerur Rahman reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../test/resources/testconfiguration.properties| 1 + .../correlation/ReduceSinkDeDuplicationUtils.java | 19 ++ .../dynpart_sort_optimization_distribute_by.q | 26 ++ .../dynpart_sort_optimization_distribute_by.q.out | 279 .../dynpart_sort_optimization_distribute_by.q.out | 284 + 5 files changed, 609 insertions(+) create mode 100644 ql/src/test/queries/clientpositive/dynpart_sort_optimization_distribute_by.q create mode 100644 ql/src/test/results/clientpositive/llap/dynpart_sort_optimization_distribute_by.q.out create mode 100644 ql/src/test/results/clientpositive/tez/dynpart_sort_optimization_distribute_by.q.out
[hive] branch master updated (3508b26 -> c2e2f36)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 3508b26 HIVE-24428: Concurrent add_partitions requests may lead to data loss (Zoltan Haindrich, reviewed by Denys Kuzmenko) add c2e2f36 HIVE-24473: Make Hive buildable with HBase 2.x GA versions (#1729) (Istvan Toth reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: itests/hcatalog-unit/pom.xml | 20 .../apache/hive/hcatalog/hbase/ManyMiniCluster.java | 4 +++- itests/util/pom.xml | 18 ++ pom.xml | 20 ql/pom.xml | 4 5 files changed, 65 insertions(+), 1 deletion(-)
[hive] branch master updated (535eddc -> 8c26718)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 535eddc HIVE-14165: Remove unnecessary file listing from FetchOperator (Peter Varga via Peter Vary) add 8c26718 HIVE-24486: Enhance shared work optimizer to allow parallel edges (#1840) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez) No new revisions were added by this update. Summary of changes: pom.xml|1 + .../hive/ql/optimizer/ParallelEdgeFixer.java | 242 .../hive/ql/optimizer/SharedWorkOptimizer.java | 118 +- .../hadoop/hive/ql/optimizer/graph/DiGraph.java| 17 + .../hive/ql/optimizer/graph/DotExporter.java | 53 +- .../hive/ql/optimizer/graph/OperatorGraph.java | 72 +- .../apache/hadoop/hive/ql/parse/TezCompiler.java |4 +- .../apache/hadoop/hive/ql/plan/ReduceSinkDesc.java |2 + .../test/queries/clientpositive/swo_vertex_merge.q | 26 + .../queries/clientpositive/swo_vertex_merge0.q | 18 + .../results/clientpositive/llap/auto_join12.q.out | 50 +- .../results/clientpositive/llap/auto_join13.q.out | 53 +- .../results/clientpositive/llap/auto_join2.q.out | 30 +- .../results/clientpositive/llap/auto_join22.q.out | 34 +- .../results/clientpositive/llap/auto_join27.q.out | 44 +- .../results/clientpositive/llap/auto_join28.q.out | 79 +- .../results/clientpositive/llap/auto_join29.q.out | 171 +-- .../results/clientpositive/llap/auto_join3.q.out | 30 +- .../results/clientpositive/llap/auto_join30.q.out | 88 +- .../results/clientpositive/llap/auto_join31.q.out | 30 +- .../results/clientpositive/llap/auto_join7.q.out | 36 +- .../llap/correlationoptimizer3.q.out | 105 +- .../llap/dynamic_partition_pruning_2.q.out | 85 +- .../clientpositive/llap/explainuser_1.q.out| 58 +- .../clientpositive/llap/explainuser_2.q.out| 1247 ++-- .../clientpositive/llap/explainuser_4.q.out| 127 +- .../llap/hybridgrace_hashjoin_2.q.out | 98 +- .../clientpositive/llap/join_max_hashtable.q.out | 183 +-- .../clientpositive/llap/mapjoin_mapjoin.q.out | 29 +- ql/src/test/results/clientpositive/llap/mrr.q.out | 29 +- .../clientpositive/llap/multiMapJoin2.q.out| 324 ++--- .../llap/reduce_deduplicate_exclude_join.q.out | 29 +- .../clientpositive/llap/skewjoin_mapjoin7.q.out| 47 +- .../clientpositive/llap/smb_mapjoin_17.q.out | 394 +++ .../clientpositive/llap/subquery_in_having.q.out | 290 +++-- .../clientpositive/llap/subquery_multiinsert.q.out | 29 +- .../clientpositive/llap/swo_vertex_merge.q.out | 294 + .../clientpositive/llap/swo_vertex_merge0.q.out| 286 + .../llap/tez_dynpart_hashjoin_1.q.out | 130 +- .../clientpositive/llap/tez_nway_join.q.out| 100 +- .../results/clientpositive/llap/tez_union.q.out| 284 +++-- .../llap/tez_vector_dynpart_hashjoin_1.q.out | 97 +- .../clientpositive/llap/vector_complex_all.q.out | 63 +- .../clientpositive/llap/vector_join30.q.out| 341 +++--- .../llap/vector_left_outer_join.q.out | 42 +- .../llap/vector_mapjoin_reduce.q.out | 106 +- .../clientpositive/llap/vector_nullsafe_join.q.out | 420 --- .../clientpositive/llap/vector_outer_join1.q.out | 69 +- .../clientpositive/llap/vector_outer_join2.q.out | 69 +- .../clientpositive/llap/vectorized_mapjoin3.q.out | 133 ++- .../llap/vectorized_multi_output_select.q.out | 27 +- .../llap/vectorized_nested_mapjoin.q.out | 45 +- .../perf/tpcds30tb/tez/cbo_query23.q.out |3 +- .../perf/tpcds30tb/tez/cbo_query58.q.out |2 +- .../perf/tpcds30tb/tez/cbo_query61.q.out |2 +- .../clientpositive/perf/tpcds30tb/tez/query1.q.out | 78 +- .../perf/tpcds30tb/tez/query11.q.out | 215 ++-- .../perf/tpcds30tb/tez/query18.q.out | 62 +- .../perf/tpcds30tb/tez/query1b.q.out | 118 +- .../clientpositive/perf/tpcds30tb/tez/query2.q.out | 115 +- .../perf/tpcds30tb/tez/query23.q.out | 471 .../perf/tpcds30tb/tez/query24.q.out | 30 +- .../perf/tpcds30tb/tez/query39.q.out | 69 +- .../clientpositive/perf/tpcds30tb/tez/query4.q.out | 367 +++--- .../perf/tpcds30tb/tez/query44.q.out | 31 +- .../perf/tpcds30tb/tez/query47.q.out | 29 +- .../perf/tpcds30tb/tez/query57.q.out | 29 +- .../perf/tpcds30tb/tez/query58.q.out | 224 ++-- .../perf/tpcds30tb/tez/query59.q.out | 137 +-- .../perf/tpcds30tb/tez/query61.q.out | 222 ++-- .../perf/tpcds30tb/tez/query64.q.out | 423 --- .../perf/tpcds30tb/tez
[hive] branch master updated (8903232 -> 58552a0)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 8903232 HIVE-24635: Support Values clause as operand of set operation (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) add 58552a0 HIVE-24632: Replace with null when GenericUDFBaseCompare has a non-interpretable val (#1863) (Zhihua Deng reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hadoop/hive/ql/exec/FunctionRegistry.java | 2 +- .../hadoop/hive/ql/parse/type/ExprFactory.java | 4 +- .../ql/parse/type/ExprNodeDescExprFactory.java | 16 ++- .../hadoop/hive/ql/parse/type/FunctionHelper.java | 3 + .../hive/ql/parse/type/HiveFunctionHelper.java | 7 ++ .../hive/ql/parse/type/RexNodeExprFactory.java | 14 ++- .../hive/ql/parse/type/TypeCheckProcFactory.java | 14 ++- .../queries/clientpositive/compare_cols_null.q | 18 +++ .../clientpositive/llap/compare_cols_null.q.out| 130 + 9 files changed, 200 insertions(+), 8 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/compare_cols_null.q create mode 100644 ql/src/test/results/clientpositive/llap/compare_cols_null.q.out
[hive] branch master updated (606527e -> 70c0a04)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 606527e HIVE-24613: Support Values clause without Insert (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez, Zoltan Haindrich) add 70c0a04 HIVE-24394: enable printing explain to console at query start (#1679) (Johan Gustavsson reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java | 5 + ql/src/java/org/apache/hadoop/hive/ql/Compiler.java | 6 +- ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java | 3 ++- 3 files changed, 12 insertions(+), 2 deletions(-)
[hive] branch master updated (68a759a -> 6deef2c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 68a759a HIVE-24617: Review beeline Driver Scanning Code (David Mollitor reviewed by Miklos Gergely) add 6deef2c HIVE-24203: Implement stats annotation rule for the LateralViewJoinOperator (#1531) No new revisions were added by this update. Summary of changes: .../stats/annotation/AnnotateWithStatistics.java |3 + .../stats/annotation/StatsRulesProcFactory.java| 105 +- .../apache/hadoop/hive/ql/stats/StatsUtils.java| 12 + .../annotate_stats_lateral_view_join.q | 60 + .../results/clientnegative/udf_assert_true.q.out | 16 +- .../results/clientnegative/udf_assert_true2.q.out |8 +- .../llap/annotate_stats_lateral_view_join.q.out| 3214 .../results/clientpositive/llap/lateral_view.q.out | 28 +- .../clientpositive/llap/lateral_view_cp.q.out |4 +- .../llap/lateral_view_explode2.q.out |8 +- .../clientpositive/llap/lateral_view_noalias.q.out | 16 +- .../clientpositive/llap/lateral_view_onview.q.out | 52 +- .../results/clientpositive/llap/lvj_mapjoin.q.out | 38 +- .../llap/multi_insert_lateral_view.q.out | 174 +- .../llap/nested_column_pruning.q.out | 44 +- .../llap/orc_nested_column_pruning.q.out | 44 +- .../llap/reduce_deduplicate_null_keys.q.out| 16 +- .../llap/select_column_pruning.q.out | 30 +- .../clientpositive/llap/skewjoin_mapjoin6.q.out|8 +- .../clientpositive/llap/udtf_json_tuple.q.out | 44 +- .../clientpositive/llap/udtf_parse_url_tuple.q.out | 44 +- .../test/results/clientpositive/llap/union26.q.out | 26 +- .../clientpositive/llap/unionDistinct_1.q.out | 28 +- .../clientpositive/llap/union_lateralview.q.out| 32 +- .../llap/vector_orc_nested_column_pruning.q.out| 44 +- .../clientpositive/llap/vector_windowing.q.out | 16 +- .../test/results/clientpositive/nonmr_fetch.q.out |8 +- .../results/clientpositive/skewjoinopt10.q.out | 16 +- .../clientpositive/tez/tez_union_udtf.q.out| 20 +- 29 files changed, 3762 insertions(+), 396 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/annotate_stats_lateral_view_join.q create mode 100644 ql/src/test/results/clientpositive/llap/annotate_stats_lateral_view_join.q.out
[hive] branch master updated (5ab7312 -> 425e1ff)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 5ab7312 HIVE-20726: hive.server2.thrift.bind.host is not honored (Denys Kuzmenko, reviewed by Laszlo Pinter) add 425e1ff HIVE-15820: comment at the head of beeline -e (#1814) (Robbie Zhang reviewed by Vihang Karajgaonkar) No new revisions were added by this update. Summary of changes: .../src/java/org/apache/hive/beeline/BeeLine.java | 7 ++- .../src/java/org/apache/hive/beeline/Commands.java | 5 +- .../test/org/apache/hive/beeline/TestCommands.java | 60 +- .../org/apache/hive/beeline/cli/TestHiveCli.java | 16 ++ .../apache/hive/common/util/HiveStringUtils.java | 4 +- .../hive/common/util/TestHiveStringUtils.java | 18 +++ 6 files changed, 78 insertions(+), 32 deletions(-)
[hive] branch branch-2 updated: HIVE-24571: Redundant code in SemanticAnalyzer.java file (#1770)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 135629b HIVE-24571: Redundant code in SemanticAnalyzer.java file
(#1770)
135629b is described below
commit 135629b8d6b538fed092641537034a9fbc59c7a0
Author: zzccctv
AuthorDate: Mon Jan 4 20:43:27 2021 +0800
HIVE-24571: Redundant code in SemanticAnalyzer.java file (#1770)
Co-authored-by: zhangzc
---
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java | 1 -
1 file changed, 1 deletion(-)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
index 2a62062..ad4765f 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
@@ -10993,7 +10993,6 @@ public class SemanticAnalyzer extends
BaseSemanticAnalyzer {
ASTNode rewrittenTree;
// Parse the rewritten query string
// check if we need to ctx.setCmd(rewrittenQuery);
- ParseDriver pd = new ParseDriver();
try {
rewrittenTree = ParseUtils.parse(rewrittenQuery);
} catch (ParseException e) {
[hive] branch master updated (22ee0bc -> 3f5e01c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 22ee0bc HIVE-24264: Fix failed-to-read test results (#1574) (Zoltan Haindrich reviewed by Krisztian Kasa) add 3f5e01c disable test No new revisions were added by this update. Summary of changes: ql/src/test/queries/clientpositive/acid_vectorization_original.q | 1 + 1 file changed, 1 insertion(+)
[hive] branch master updated (314ff8d -> 22ee0bc)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 314ff8d HIVE-24474: Failed compaction always logs TxnAbortedException (again) (Karen Coppage, reviewed by Kuczora Marta) add 22ee0bc HIVE-24264: Fix failed-to-read test results (#1574) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: Jenkinsfile| 108 +++-- .../hooks/RuntimeStatsPersistenceCheckerHook.java | 2 +- .../hive/ql/optimizer/BucketVersionPopulator.java | 2 +- .../apache/hadoop/hive/ql/ppd/OpProcFactory.java | 21 ++-- 4 files changed, 85 insertions(+), 48 deletions(-)
[hive] branch master updated (d9778b9 -> 2caad3d)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from d9778b9 HIVE-24479: Introduce setting to set lower bound of hash aggregation reduction. (ADDENDUM) add 2caad3d HIVE-24488: Make tests using docker more comfortable (#1766) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: .../metastore/dbinstall/rules/DatabaseRule.java| 48 ++ .../hive/metastore/dbinstall/rules/Derby.java | 4 +- .../hive/metastore/dbinstall/rules/Mssql.java | 8 ++-- .../hive/metastore/dbinstall/rules/Mysql.java | 8 ++-- .../hive/metastore/dbinstall/rules/Oracle.java | 8 ++-- .../hive/metastore/dbinstall/rules/Postgres.java | 16 +--- .../metastore/dbinstall/rules/PostgresTPCDS.java | 4 +- 7 files changed, 59 insertions(+), 37 deletions(-)
[hive] branch master updated (8018a75 -> d9778b9)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 8018a75 HIVE-24498: add thrift facebook classes to hive-exec jar (Marton Bod reviewed by Peter Vary) add d9778b9 HIVE-24479: Introduce setting to set lower bound of hash aggregation reduction. (ADDENDUM) No new revisions were added by this update. Summary of changes: .../test/results/clientpositive/llap/acid_nullscan.q.out | 2 +- ql/src/test/results/clientpositive/llap/acid_stats3.q.out | 2 +- ql/src/test/results/clientpositive/llap/acid_stats4.q.out | 2 +- ql/src/test/results/clientpositive/llap/acid_stats5.q.out | 8 ql/src/test/results/clientpositive/llap/cte_mat_6.q.out| 8 ql/src/test/results/clientpositive/llap/cte_mat_7.q.out| 2 +- .../results/clientpositive/llap/distinct_groupby.q.out | 14 +++--- .../test/results/clientpositive/llap/distinct_stats.q.out | 2 +- 8 files changed, 20 insertions(+), 20 deletions(-)
[hive] branch master updated (41c5c79 -> 19324b0)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.
from 41c5c79 HIVE-24504: VectorFileSinkArrowOperator does not serialize
complex types correctly (Peter Vary reviewed by Panos Garefalakis and Adam
Szita)
add 19324b0 HIVE-24388: Enhance swo optimizations to merge EventOperators
(#1750) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
No new revisions were added by this update.
Summary of changes:
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 2 +
.../hive/ql/optimizer/SharedWorkOptimizer.java | 50 +-
.../{reopt_dpp.q => swo_event_merge.q} | 29 +-
.../{reopt_dpp.q.out => swo_event_merge.q.out} | 206 +++---
.../perf/tpcds30tb/tez/query11.q.out | 429 ++---
.../clientpositive/perf/tpcds30tb/tez/query2.q.out | 115 ++--
.../perf/tpcds30tb/tez/query23.q.out | 222 ---
.../clientpositive/perf/tpcds30tb/tez/query4.q.out | 475 ++
.../perf/tpcds30tb/tez/query59.q.out | 227 ---
.../perf/tpcds30tb/tez/query74.q.out | 376 ++-
.../perf/tpcds30tb/tez/query75.q.out | 693 +++--
11 files changed, 1376 insertions(+), 1448 deletions(-)
copy ql/src/test/queries/clientpositive/{reopt_dpp.q => swo_event_merge.q}
(71%)
copy ql/src/test/results/clientpositive/llap/{reopt_dpp.q.out =>
swo_event_merge.q.out} (55%)
[hive] branch branch-2 updated: set pod timeout to 24 hours
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new f101c73 set pod timeout to 24 hours
f101c73 is described below
commit f101c735e7790b6bdb8cb83369bf356f28f91d07
Author: Zoltan Haindrich
AuthorDate: Thu Dec 10 11:53:56 2020 +
set pod timeout to 24 hours
---
Jenkinsfile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Jenkinsfile b/Jenkinsfile
index 2850142..c23ca60 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -67,10 +67,12 @@ setPrLabel("PENDING");
def executorNode(run) {
hdbPodTemplate {
+timeout(time: 24, unit: 'HOURS') {
node(POD_LABEL) {
container('hdb') {
run()
}
+ }
}
}
}
[hive] branch master updated (ee8663e -> 52cf467)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from ee8663e HIVE-24403: change min_history_level to make it compatible with older versions (Peter Varga, reviewed by Denys Kuzmenko) add 52cf467 HIVE-23965: Improve plan regression tests using TPCDS30TB metastore dump and custom configs (Stamatis Zampetakis, reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: beeline/pom.xml|4 +- .../apache/hive/beeline/TestBeelineArgParsing.java | 12 +- data/conf/perf-reg/spark/hive-site.xml | 310 - data/conf/perf-reg/tez/hive-site.xml | 320 - data/conf/perf-reg/tez/tez-site.xml| 10 - data/conf/perf/tpcds30tb/spark/hive-site.xml | 214 +++ data/conf/perf/tpcds30tb/tez/hive-site.xml | 214 +++ data/conf/perf/tpcds30tb/tez/tez-site.xml | 20 + .../metastore_export/csv/TABLE_PARAMS.txt.bz2 | Bin 481 -> 0 bytes .../metastore_export/csv/TAB_COL_STATS.txt.bz2 | Bin 6216 -> 0 bytes data/scripts/q_perf_test_init.sql | 616 - data/scripts/q_perf_test_init_constraints.sql | 772 --- .../hive/cli/TestSparkTPCDS30TBPerfCliDriver.java} | 41 +- itests/qtest/pom.xml |2 +- .../hadoop/hive/cli/TestTezPerfCliDriver.java | 74 -- .../hive/cli/TestTezTPCDS30TBPerfCliDriver.java} | 38 +- .../hadoop/hive/cli/control/AbstractCliConfig.java | 11 + .../apache/hadoop/hive/cli/control/CliAdapter.java |2 +- .../apache/hadoop/hive/cli/control/CliConfigs.java | 60 +- .../hadoop/hive/cli/control/CorePerfCliDriver.java | 59 +- .../hadoop/hive/ql/MetaStoreDumpUtility.java | 235 .../hadoop/hive/ql/QTestMetaStoreHandler.java | 14 +- .../hadoop/hive/cli/control/TestDanglingQOuts.java |7 +- pom.xml|1 + .../test/queries/clientpositive/perf/cbo_query14.q |1 + .../test/queries/clientpositive/perf/cbo_query30.q |1 + .../test/queries/clientpositive/perf/cbo_query41.q | 55 + .../test/queries/clientpositive/perf/cbo_query44.q |1 - .../test/queries/clientpositive/perf/cbo_query45.q |1 - .../test/queries/clientpositive/perf/cbo_query62.q | 29 + .../test/queries/clientpositive/perf/cbo_query67.q |1 - .../test/queries/clientpositive/perf/cbo_query70.q |1 - .../test/queries/clientpositive/perf/cbo_query86.q |1 - ql/src/test/queries/clientpositive/perf/query14.q |1 + ql/src/test/queries/clientpositive/perf/query30.q |1 + ql/src/test/queries/clientpositive/perf/query62.q | 29 + .../results/clientpositive/perf/spark/query1.q.out | 384 -- .../clientpositive/perf/spark/query10.q.out| 559 .../clientpositive/perf/spark/query11.q.out| 739 --- .../clientpositive/perf/spark/query12.q.out| 268 .../clientpositive/perf/spark/query13.q.out| 374 -- .../clientpositive/perf/spark/query15.q.out| 252 .../clientpositive/perf/spark/query16.q.out| 376 -- .../clientpositive/perf/spark/query17.q.out| 460 --- .../clientpositive/perf/spark/query18.q.out| 401 -- .../clientpositive/perf/spark/query19.q.out| 345 - .../results/clientpositive/perf/spark/query2.q.out | 457 --- .../clientpositive/perf/spark/query20.q.out| 260 .../clientpositive/perf/spark/query21.q.out| 262 .../clientpositive/perf/spark/query22.q.out| 251 .../clientpositive/perf/spark/query23.q.out| 939 - .../clientpositive/perf/spark/query24.q.out| 691 -- .../clientpositive/perf/spark/query25.q.out| 458 --- .../clientpositive/perf/spark/query26.q.out| 290 .../clientpositive/perf/spark/query27.q.out| 298 - .../clientpositive/perf/spark/query28.q.out| 584 .../clientpositive/perf/spark/query29.q.out| 456 --- .../results/clientpositive/perf/spark/query3.q.out | 206 --- .../clientpositive/perf/spark/query30.q.out| 478 --- .../clientpositive/perf/spark/query31.q.out| 918 - .../clientpositive/perf/spark/query32.q.out| 311 - .../clientpositive/perf/spark/query33.q.out| 758 --- .../clientpositive/perf/spark/query34.q.out| 304 - .../clientpositive/perf/spark/query35.q.out| 560 .../clientpositive/perf/spark/query36.q.out| 311 - .../clientpositive/perf/spark/query37.q.out| 218 --- .../clientpositive/perf/spark/query38.q.out| 488 --- .../clientpositive/perf/spark/query39.q.out| 521 .../results/clientpositive/perf/spark/query4
[hive] branch master updated (b631f66 -> 91ab242)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.
from b631f66 HIVE-21843: UNION query with regular expressions for column
name does not work (#1684) (Zoltan Haindrich reviewed by Krisztian Kasa)
add 91ab242 HIVE-24230 Integrate HPL/SQL into HiveServer2 (amagyar)
(#1633) (Attila Magyar reviewed by Zoltan Haindrich)
No new revisions were added by this update.
Summary of changes:
.../src/java/org/apache/hive/beeline/BeeLine.java | 7 +-
.../org/apache/hadoop/hive/conf/Constants.java | 1 +
hplsql/pom.xml | 2 +-
.../src/main/java/org/apache/hive/hplsql/Cmp.java | 91 +--
.../src/main/java/org/apache/hive/hplsql/Conn.java | 34 +-
.../main/java/org/apache/hive/hplsql/Console.java | 44 ++
.../src/main/java/org/apache/hive/hplsql/Copy.java | 82 ++-
.../main/java/org/apache/hive/hplsql/Cursor.java | 124
.../src/main/java/org/apache/hive/hplsql/Exec.java | 228
.../java/org/apache/hive/hplsql/Expression.java| 14 +-
.../org/apache/hive/hplsql/HplSqlSessionState.java | 27 +
.../src/main/java/org/apache/hive/hplsql/Meta.java | 67 +--
.../main/java/org/apache/hive/hplsql/Query.java| 110 +---
.../org/apache/hive/hplsql/ResultListener.java | 35 ++
.../main/java/org/apache/hive/hplsql/Select.java | 93 ++-
.../src/main/java/org/apache/hive/hplsql/Stmt.java | 623 ++---
.../java/org/apache/hive/hplsql/StreamGobbler.java | 8 +-
.../src/main/java/org/apache/hive/hplsql/Var.java | 53 +-
.../apache/hive/hplsql/executor/ColumnMeta.java| 45 ++
.../hive/hplsql/executor/JdbcQueryExecutor.java| 101
.../org/apache/hive/hplsql/executor/Metadata.java | 51 ++
.../hive/hplsql/executor/QueryException.java | 41 ++
.../apache/hive/hplsql/executor/QueryExecutor.java | 27 +
.../apache/hive/hplsql/executor/QueryResult.java | 93 +++
.../org/apache/hive/hplsql/executor/RowResult.java | 27 +
.../hive/hplsql/functions/BuiltinFunctions.java| 49 +-
.../hive/hplsql/functions/FunctionDatetime.java| 5 +-
.../apache/hive/hplsql/functions/FunctionMisc.java | 48 +-
.../apache/hive/hplsql/functions/FunctionOra.java | 7 +-
.../hive/hplsql/functions/FunctionString.java | 5 +-
.../apache/hive/hplsql/functions/HmsFunction.java | 95 ++--
.../hive/hplsql/functions/InMemoryFunction.java| 18 +-
.../java/org/apache/hive/hplsql/{ => udf}/Udf.java | 102 ++--
.../java/org/apache/hive/hplsql/TestHplsqlUdf.java | 59 --
.../org/apache/hive/hplsql/udf/TestHplsqlUdf.java | 60 ++
.../src/test/results/local/create_function.out.txt | 4 +-
.../test/results/local/create_function2.out.txt| 4 +-
.../test/results/local/create_function3.out.txt| 6 +-
.../test/results/local/create_function4.out.txt| 4 +-
.../test/results/local/create_function5.out.txt| 4 +-
.../src/test/results/local/create_package.out.txt | 4 +-
.../test/results/local/create_procedure.out.txt| 4 +-
.../test/results/local/create_procedure2.out.txt | 4 +-
.../test/results/local/create_procedure3.out.txt | 8 +-
.../test/results/local/create_procedure4.out.txt | 4 +-
.../local/create_procedure_no_params.out.txt | 20 +-
hplsql/src/test/results/local/var_scope.out.txt| 8 +-
hplsql/src/test/results/local/var_scope2.out.txt | 16 +-
.../apache/hive/beeline/TestHplSqlViaBeeLine.java | 203 +++
.../java/org/apache/hive/jdbc/HiveConnection.java | 126 +++--
.../hadoop/hive/ql/session/SessionState.java | 12 +
.../org/apache/hadoop/hive/serde2/thrift/Type.java | 9 +
service-rpc/if/TCLIService.thrift | 1 +
.../src/gen/thrift/gen-cpp/TCLIService_types.cpp | 8 +-
.../src/gen/thrift/gen-cpp/TCLIService_types.h | 3 +-
.../hive/service/rpc/thrift/TOperationType.java| 5 +-
.../src/gen/thrift/gen-php/TOperationType.php | 3 +
.../src/gen/thrift/gen-py/TCLIService/ttypes.py| 3 +
.../src/gen/thrift/gen-rb/t_c_l_i_service_types.rb | 5 +-
service/pom.xml| 5 +
.../org/apache/hive/service/cli/OperationType.java | 1 +
.../cli/operation/ExecuteStatementOperation.java | 67 ++-
.../cli/operation/HiveCommandOperation.java| 2 +-
.../hive/service/cli/operation/Operation.java | 21 +-
.../hive/service/cli/operation/SQLOperation.java | 23 +-
.../cli/operation/hplsql/BeelineConsole.java | 39 ++
.../cli/operation/hplsql/HplSqlOperation.java | 240
.../cli/operation/hplsql/HplSqlQueryExecutor.java | 149 +
68 files changed, 2328 insertions(+), 1163 deletions(-)
create mode 100644 hplsql/src/main/java/org/apache/hive/hplsql/Console.java
create mode 100644 hplsql/src/main/java/org/apache/hive/hplsql/Cursor.java
create mode 100644
hplsql/src/main/java/org/apache/hive/hpl
[hive] branch master updated (1671e6a -> b631f66)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 1671e6a HIVE-24435: Vectorized unix_timestamp is inconsistent with non-vectorized counterpart (#1713) (Zoltan Haindrich reviewed by Krisztian Kasa) add b631f66 HIVE-21843: UNION query with regular expressions for column name does not work (#1684) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: .../AlterMaterializedViewRewriteOperation.java | 2 +- .../apache/hadoop/hive/ql/parse/ParseUtils.java| 27 .../hadoop/hive/ql/parse/SemanticAnalyzer.java | 3 +- .../apache/hadoop/hive/ql/tools/LineageInfo.java | 17 ++--- .../partition/show/TestShowPartitionAnalyzer.java | 31 +++-- .../hive/ql/parse/TestMacroSemanticAnalyzer.java | 2 +- .../hadoop/hive/ql/parse/TestParseUtils.java | 25 +-- .../parse/authorization/AuthorizationTestUtil.java | 8 +-- .../ql/parse/authorization/PrivilegesTestBase.java | 5 +- .../TestHiveAuthorizationTaskFactory.java | 3 +- .../hadoop/hive/ql/tool/TestLineageInfo.java | 78 -- ql/src/test/queries/clientpositive/union_regex.q | 17 + .../results/clientpositive/llap/union_regex.q.out | 62 + 13 files changed, 202 insertions(+), 78 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/union_regex.q create mode 100644 ql/src/test/results/clientpositive/llap/union_regex.q.out
[hive] branch master updated (9cc3dc4 -> 1671e6a)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 9cc3dc4 HIVE-24453: Direct SQL error when parsing create_time value for database (Jesus Camacho Rodriguez, reviewed by Krisztian Kasa) add 1671e6a HIVE-24435: Vectorized unix_timestamp is inconsistent with non-vectorized counterpart (#1713) (Zoltan Haindrich reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: .../expressions/VectorUDFDayOfWeekString.java | 2 +- .../expressions/VectorUDFTimestampFieldString.java | 9 +--- .../expressions/VectorUDFUnixTimeStampString.java | 2 +- .../expressions/VectorUDFWeekOfYearString.java | 2 +- .../queries/clientpositive/vector_unix_timestamp.q | 21 .../clientpositive/vectorized_timestamp_funcs.q| 4 +- .../llap/vector_unix_timestamp.q.out | 56 ++ .../llap/vectorized_timestamp_funcs.q.out | 10 ++-- 8 files changed, 89 insertions(+), 17 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/vector_unix_timestamp.q create mode 100644 ql/src/test/results/clientpositive/llap/vector_unix_timestamp.q.out
[hive] branch master updated: HIVE-24365: SWO should not create complex and redundant filter expressions (#1661) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 67bb821 HIVE-24365: SWO should not create complex and redundant
filter expressions (#1661) (Zoltan Haindrich reviewed by Jesus Camacho
Rodriguez)
67bb821 is described below
commit 67bb821d02273f8c9f8f45340c9445b5508dc731
Author: Zoltan Haindrich
AuthorDate: Fri Nov 13 08:56:57 2020 +0100
HIVE-24365: SWO should not create complex and redundant filter expressions
(#1661) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
---
.../apache/hadoop/hive/ql/exec/FilterOperator.java | 7
.../hadoop/hive/ql/exec/GroupByOperator.java | 7
.../org/apache/hadoop/hive/ql/exec/Operator.java | 8 +
.../hadoop/hive/ql/exec/ReduceSinkOperator.java| 8 +
.../apache/hadoop/hive/ql/exec/SelectOperator.java | 8 +
.../hive/ql/optimizer/SharedWorkOptimizer.java | 31 +---
.../hive/ql/optimizer/graph/DotExporter.java | 8 +
.../hadoop/hive/ql/plan/ExprNodeDescUtils.java | 41 ++
.../perf/tez/constraints/query23.q.out | 6 ++--
.../perf/tez/constraints/query31.q.out | 8 ++---
.../perf/tez/constraints/query88.q.out | 14
.../results/clientpositive/perf/tez/query31.q.out | 8 ++---
.../results/clientpositive/perf/tez/query88.q.out | 14
13 files changed, 138 insertions(+), 30 deletions(-)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
b/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
index bb08f24..a3af21a 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java
@@ -26,6 +26,7 @@ import org.apache.hadoop.hive.ql.CompilationOpContext;
import org.apache.hadoop.hive.ql.io.IOContext;
import org.apache.hadoop.hive.ql.io.IOContextMap;
import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDescUtils;
import org.apache.hadoop.hive.ql.plan.FilterDesc;
import org.apache.hadoop.hive.ql.plan.api.OperatorType;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
@@ -163,4 +164,10 @@ public class FilterOperator extends Operator
implements
public boolean supportUnionRemoveOptimization() {
return true;
}
+
+ @Override
+ public void replaceTabAlias(String oldAlias, String newAlias) {
+super.replaceTabAlias(oldAlias, newAlias);
+ExprNodeDescUtils.replaceTabAlias(getConf().getPredicate(), oldAlias,
newAlias);
+ }
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
b/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
index 01de9df..f118aa1 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
@@ -41,6 +41,7 @@ import org.apache.hadoop.hive.ql.parse.OpParseContext;
import org.apache.hadoop.hive.ql.plan.AggregationDesc;
import org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc;
import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDescUtils;
import org.apache.hadoop.hive.ql.plan.GroupByDesc;
import org.apache.hadoop.hive.ql.plan.GroupByDesc.Mode;
import org.apache.hadoop.hive.ql.plan.OperatorDesc;
@@ -1146,4 +1147,10 @@ public class GroupByOperator extends
Operator implements IConfigure
}
}
+ @Override
+ public void replaceTabAlias(String oldAlias, String newAlias) {
+super.replaceTabAlias(oldAlias, newAlias);
+ExprNodeDescUtils.replaceTabAlias(getConf().getKeys(), oldAlias, newAlias);
+ }
+
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
b/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
index 0c32d6d..173f903 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
@@ -47,6 +47,7 @@ import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.parse.SemanticException;
import org.apache.hadoop.hive.ql.plan.Explain;
import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDescUtils;
import org.apache.hadoop.hive.ql.plan.OpTraits;
import org.apache.hadoop.hive.ql.plan.OperatorDesc;
import org.apache.hadoop.hive.ql.plan.Statistics;
@@ -1543,4 +1544,11 @@ public abstract class Operator
implements Serializable,C
}
return true;
}
+
+ public void replaceTabAlias(String oldAlias, String newAlias) {
+ExprNodeDescUtils.replaceTabAlias(getConf().getColumnExprMap(), oldAlias,
newAlias);
+for (Operator c : getChildOperators()) {
+ c.replaceTabAlias(oldAlias, newAlias);
+}
+ }
}
diff --git a/ql/src/java/org
[hive] branch master updated (9f1715b -> 2e2fb6c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 9f1715b HIVE-24360: SharedWorkOptimizer may create incorrect plans with DPPUnion (#1654) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez) add 2e2fb6c disable TestLlapExtClientWithCloudDeploymentConfigs No new revisions were added by this update. Summary of changes: .../apache/hive/jdbc/TestLlapExtClientWithCloudDeploymentConfigs.java| 1 + 1 file changed, 1 insertion(+)
[hive] branch master updated (a107484 -> 9f1715b)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from a107484 HIVE-24173:notification cleanup interval value changes depending upon replication enabled or not.(Arko Sharma, reviewed by Aasha Medhi) add 9f1715b HIVE-24360: SharedWorkOptimizer may create incorrect plans with DPPUnion (#1654) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez) No new revisions were added by this update. Summary of changes: data/conf/hive-log4j2.properties | 6 +- .../hive/ql/optimizer/SharedWorkOptimizer.java | 50 ++ .../rules/HivePointLookupOptimizerRule.java| 106 +--- .../hadoop/hive/ql/optimizer/graph/DagGraph.java | 85 + .../hadoop/hive/ql/optimizer/graph/DiGraph.java| 187 .../hive/ql/optimizer/graph/DotExporter.java | 125 ++ .../hive/ql/optimizer/graph/OperatorGraph.java | 190 + .../llap/acid_vectorization_original.q.out | 16 +- .../llap/dynamic_partition_pruning.q.out | 88 +- .../vectorized_dynamic_partition_pruning.q.out | 97 ++- 10 files changed, 718 insertions(+), 232 deletions(-) create mode 100644 ql/src/java/org/apache/hadoop/hive/ql/optimizer/graph/DagGraph.java create mode 100644 ql/src/java/org/apache/hadoop/hive/ql/optimizer/graph/DiGraph.java create mode 100644 ql/src/java/org/apache/hadoop/hive/ql/optimizer/graph/DotExporter.java create mode 100644 ql/src/java/org/apache/hadoop/hive/ql/optimizer/graph/OperatorGraph.java
[hive] branch master updated (90ae3c8 -> f786902)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 90ae3c8 HIVE-24305: avro decimal schema is not properly populating scale/precision if value is enclosed in quote (#1601) (Naresh Panchetty Ramanaiah, reviewed by Laszlo Pinter) add f786902 HIVE-24355: Implement hashCode/equals for Partition (#1652) (Zoltan Haindrich reviewed by Miklos Gergely) No new revisions were added by this update. Summary of changes: .../apache/hadoop/hive/ql/metadata/Partition.java | 26 +- .../apache/hadoop/hive/ql/metadata/TestHive.java | 29 ++ .../test/queries/clientpositive/swo_pruned_merge.q | 21 ++ .../clientpositive/llap/swo_pruned_merge.q.out | 365 + .../hadoop/hive/metastore/MetaStoreDirectSql.java | 4 +- .../apache/hadoop/hive/metastore/ObjectStore.java | 12 +- .../hive/metastore/utils/MetaStoreServerUtils.java | 2 + 7 files changed, 452 insertions(+), 7 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/swo_pruned_merge.q create mode 100644 ql/src/test/results/clientpositive/llap/swo_pruned_merge.q.out
[hive] branch master updated (10236b5 -> e2d637e)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 10236b5 HIVE-24307:Beeline with property-file and -e parameter is failing (Ayush Saxena, reviewed by Aasha Medhi) add e2d637e HIVE-24217: HMS storage backend for HPL/SQL stored procedures (#1542) (Attila Magyar reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/hive/hplsql/Exec.java | 283 +- .../main/java/org/apache/hive/hplsql/Package.java |13 +- .../main/java/org/apache/hive/hplsql/Select.java | 6 +- .../src/main/java/org/apache/hive/hplsql/Stmt.java |18 +- .../src/main/java/org/apache/hive/hplsql/Var.java |18 +- .../hive/hplsql/functions/BuiltinFunctions.java| 435 + .../org/apache/hive/hplsql/functions/Function.java | 764 +- .../hive/hplsql/functions/FunctionDatetime.java|30 +- .../apache/hive/hplsql/functions/FunctionMisc.java |17 +- .../apache/hive/hplsql/functions/FunctionOra.java |22 +- .../hive/hplsql/functions/FunctionString.java | 8 +- .../apache/hive/hplsql/functions/HmsFunction.java | 232 + .../hive/hplsql/functions/InMemoryFunction.java| 265 + hplsql/src/main/resources/hplsql-site.xml | 8 +- .../org/apache/hive/hplsql/TestHplsqlLocal.java| 2 +- .../org/apache/hive/hplsql/TestHplsqlOffline.java | 2 +- .../src/test/results/local/create_package.out.txt | 2 +- .../local/create_procedure_no_params.out.txt |12 +- .../hcatalog/listener/DummyRawStoreFailEvent.java |22 + .../src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp | 13128 +++ .../src/gen/thrift/gen-cpp/ThriftHiveMetastore.h | 542 + .../ThriftHiveMetastore_server.skeleton.cpp|20 + .../gen/thrift/gen-cpp/hive_metastore_types.cpp| 607 +- .../src/gen/thrift/gen-cpp/hive_metastore_types.h | 167 + .../metastore/api/ListStoredProcedureRequest.java | 478 + .../hadoop/hive/metastore/api/StoredProcedure.java | 790 ++ .../hive/metastore/api/StoredProcedureRequest.java | 570 + .../hive/metastore/api/ThriftHiveMetastore.java| 5572 +++- .../metastore/ListStoredProcedureRequest.php | 118 + .../thrift/gen-php/metastore/StoredProcedure.php | 190 + .../gen-php/metastore/StoredProcedureRequest.php | 142 + .../metastore/ThriftHiveMetastoreClient.php| 251 + .../gen-php/metastore/ThriftHiveMetastoreIf.php|25 + ...tHiveMetastore_create_stored_procedure_args.php |99 + ...iveMetastore_create_stored_procedure_result.php | 122 + ...iftHiveMetastore_drop_stored_procedure_args.php |99 + ...tHiveMetastore_drop_stored_procedure_result.php | 122 + ...iveMetastore_get_all_stored_procedures_args.php |99 + ...eMetastore_get_all_stored_procedures_result.php | 140 + ...riftHiveMetastore_get_stored_procedure_args.php |99 + ...ftHiveMetastore_get_stored_procedure_result.php | 151 + .../hive_metastore/ThriftHiveMetastore-remote |28 + .../gen-py/hive_metastore/ThriftHiveMetastore.py | 1417 +- .../src/gen/thrift/gen-py/hive_metastore/ttypes.py | 278 + .../src/gen/thrift/gen-rb/hive_metastore_types.rb |72 + .../src/gen/thrift/gen-rb/thrift_hive_metastore.rb | 253 + .../hadoop/hive/metastore/HiveMetaStoreClient.java |20 + .../hadoop/hive/metastore/IMetaStoreClient.java| 7 + .../src/main/thrift/hive_metastore.thrift |24 + .../hadoop/hive/metastore/HiveMetaStore.java |75 +- .../apache/hadoop/hive/metastore/ObjectStore.java | 125 + .../org/apache/hadoop/hive/metastore/RawStore.java |10 +- .../hadoop/hive/metastore/cache/CachedStore.java |20 + .../hadoop/hive/metastore/model/MStoredProc.java |69 + .../src/main/resources/package.jdo |25 + .../src/main/sql/derby/hive-schema-4.0.0.derby.sql |13 + .../sql/derby/upgrade-3.2.0-to-4.0.0.derby.sql |14 + .../src/main/sql/mssql/hive-schema-4.0.0.mssql.sql |14 + .../sql/mssql/upgrade-3.2.0-to-4.0.0.mssql.sql |14 + .../src/main/sql/mysql/hive-schema-4.0.0.mysql.sql |13 + .../sql/mysql/upgrade-3.2.0-to-4.0.0.mysql.sql |14 + .../main/sql/oracle/hive-schema-4.0.0.oracle.sql |14 + .../sql/oracle/upgrade-3.2.0-to-4.0.0.oracle.sql |14 + .../sql/postgres/hive-schema-4.0.0.postgres.sql|13 + .../postgres/upgrade-3.2.0-to-4.0.0.postgres.sql |14 + .../metastore/DummyRawStoreControlledCommit.java |21 + .../metastore/DummyRawStoreForJdoConnection.java |22 + .../metastore/HiveMetaStoreClientPreCatalog.java |20 + .../hadoop/hive/metastore/TestObjectStore.java | 110 +- 69 files changed, 20701 insertions(+), 7722 deletions(-) create mode 100644 hplsql/src/main/java/org/apache/hive/hplsql/functions/BuiltinFunctions.java create mode 100644
[hive] branch master updated (4e19fdb -> 275e63b)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.
from 4e19fdb HIVE-24227:sys.replication_metrics table shows incorrect
status for failed policies (Arko Sharma, reviewed by Aasha Medhi)
add 275e63b HIVE-24231: Enhance shared work optimizer to merge scans with
filters on both sides (#1553) (Zoltan Haindrich reviewed by Jesus Camacho
Rodriguez)
No new revisions were added by this update.
Summary of changes:
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 3 +
.../hive/ql/optimizer/SharedWorkOptimizer.java | 417 +++
.../hadoop/hive/ql/parse/SemiJoinBranchInfo.java | 4 +
.../hadoop/hive/ql/plan/ExprNodeDescUtils.java | 78 +-
.../{reopt_semijoin.q => sharedwork_semi.q}| 64 +-
.../llap/dynamic_partition_pruning.q.out | 88 ++-
.../clientpositive/llap/sharedwork_semi.q.out | 817 +
.../llap/special_character_in_tabnames_1.q.out | 2 +-
.../special_character_in_tabnames_quotes_1.q.out | 2 +-
.../clientpositive/llap/subquery_multi.q.out | 6 +-
.../clientpositive/llap/subquery_notin.q.out | 2 +-
.../vectorized_dynamic_partition_pruning.q.out | 97 ++-
.../perf/tez/constraints/query11.q.out | 226 +++---
.../perf/tez/constraints/query14.q.out | 544 +++---
.../perf/tez/constraints/query23.q.out | 284 ---
.../perf/tez/constraints/query31.q.out | 358 +
.../perf/tez/constraints/query4.q.out | 388 +-
.../perf/tez/constraints/query64.q.out | 372 +-
.../perf/tez/constraints/query74.q.out | 216 +++---
.../perf/tez/constraints/query75.q.out | 388 +-
.../perf/tez/constraints/query88.q.out | 632
.../results/clientpositive/perf/tez/query11.q.out | 260 ---
.../results/clientpositive/perf/tez/query14.q.out | 534 +++---
.../results/clientpositive/perf/tez/query23.q.out | 260 ---
.../results/clientpositive/perf/tez/query24.q.out | 158 ++--
.../results/clientpositive/perf/tez/query31.q.out | 358 +
.../results/clientpositive/perf/tez/query4.q.out | 386 +-
.../results/clientpositive/perf/tez/query64.q.out | 296
.../results/clientpositive/perf/tez/query74.q.out | 260 ---
.../results/clientpositive/perf/tez/query75.q.out | 394 +-
.../results/clientpositive/perf/tez/query88.q.out | 632
31 files changed, 4733 insertions(+), 3793 deletions(-)
copy ql/src/test/queries/clientpositive/{reopt_semijoin.q =>
sharedwork_semi.q} (56%)
create mode 100644
ql/src/test/results/clientpositive/llap/sharedwork_semi.q.out
[hive] branch master updated: HIVE-24106: Abort polling on the operation state when the current thr… (#1456) (Zhihua Deng reviewed by Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 6392c0e HIVE-24106: Abort polling on the operation state when the
current thr… (#1456) (Zhihua Deng reviewed by Zoltan Haindrich)
6392c0e is described below
commit 6392c0e7f16f0f767aa8c1b70eafa672c0b7865f
Author: dengzh
AuthorDate: Wed Oct 14 15:51:43 2020 +0800
HIVE-24106: Abort polling on the operation state when the current thr…
(#1456) (Zhihua Deng reviewed by Zoltan Haindrich)
---
.../java/org/apache/hadoop/hive/ql/ErrorMsg.java | 1 +
.../org/apache/hive/jdbc/TestJdbcWithMiniHS2.java | 29 ++
.../java/org/apache/hive/jdbc/HiveStatement.java | 6 +
3 files changed, 36 insertions(+)
diff --git a/common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
b/common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
index 92406cd..9b50f75 100644
--- a/common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
+++ b/common/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
@@ -509,6 +509,7 @@ public enum ErrorMsg {
REPL_FILE_MISSING_FROM_SRC_AND_CM_PATH(20016, "File is missing from both
source and cm path."),
REPL_EXTERNAL_SERVICE_CONNECTION_ERROR(20017, "Failed to connect to {0}
service. Error code {1}.",true),
+ CLIENT_POLLING_OPSTATUS_INTERRUPTED(20018, "Interrupted while polling on the
operation status", "70100"),
// An exception from runtime that will show the full stack to client
UNRESOLVED_RT_EXCEPTION(2, "Runtime Error: {0}", "58004", true),
diff --git
a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java
b/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java
index b80fb58..acbc5fd 100644
---
a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java
+++
b/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java
@@ -47,6 +47,7 @@ import java.util.Set;
import java.util.Base64;
import java.util.concurrent.Callable;
import java.util.concurrent.CancellationException;
+import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@@ -1683,4 +1684,32 @@ public class TestJdbcWithMiniHS2 {
assertTrue(getDetailedTableDescription(stmt,
"emp_mm_table").contains(mndPath));
}
}
+
+ @Test
+ public void testInterruptPollingState() throws Exception {
+ExecutorService pool = Executors.newFixedThreadPool(1);
+final CountDownLatch latch = new CountDownLatch(1);
+final Object[] results = new Object[2];
+results[0] = false;
+Future future = pool.submit(new Callable() {
+ @Override
+ public Void call() {
+try (Statement stmt = conTestDb.createStatement()) {
+ stmt.execute("create temporary function sleepMsUDF as '" +
SleepMsUDF.class.getName() + "'");
+ stmt.execute("SELECT sleepMsUDF(1, 1)");
+ results[0] = true;
+} catch (Exception e) {
+ results[1] = e;
+} finally {
+ latch.countDown();
+}
+return null;
+ }
+});
+Thread.sleep(2000);
+future.cancel(true);
+latch.await();
+assertEquals(false, results[0]);
+assertEquals("Interrupted while polling on the operation status",
((Exception)results[1]).getMessage());
+ }
}
diff --git a/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
b/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
index cd1c130..0fb8496 100644
--- a/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
+++ b/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
@@ -59,6 +59,8 @@ import java.util.Map;
import java.util.Objects;
import java.util.Optional;
+import static
org.apache.hadoop.hive.ql.ErrorMsg.CLIENT_POLLING_OPSTATUS_INTERRUPTED;
+
/**
* The object used for executing a static SQL statement and returning the
* results it produces.
@@ -360,6 +362,10 @@ public class HiveStatement implements java.sql.Statement {
// Poll on the operation status, till the operation is complete
do {
try {
+if (Thread.currentThread().isInterrupted()) {
+ throw new SQLException(CLIENT_POLLING_OPSTATUS_INTERRUPTED.getMsg(),
+ CLIENT_POLLING_OPSTATUS_INTERRUPTED.getSQLState());
+}
/**
* For an async SQLOperation, GetOperationStatus will use the long
polling approach It will
* essentially return after the HIVE_SERVER2_LONG_POLLING_TIMEOUT (a
server config) expires
[hive] branch master updated (8eba401 -> b58f544)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 8eba401 HIVE-24255: StorageHandler with select-limit query is returning 0 rows (Naresh P R, reviewed by Vineet Garg) add b58f544 HIVE-23800: Add hooks when HiveServer2 stops due to OutOfMemoryError (#1205) (Zhihua Deng reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hive/conf/HiveConf.java | 3 + .../java/org/apache/hadoop/hive/ql/HookRunner.java | 98 - .../org/apache/hadoop/hive/ql/hooks/HiveHooks.java | 160 + .../apache/hadoop/hive/ql/hooks/HookContext.java | 47 +- .../org/apache/hadoop/hive/ql/hooks/HookUtils.java | 28 +--- .../apache/hadoop/hive/ql/hooks/TestHiveHooks.java | 131 + .../hive/service/cli/session/SessionManager.java | 3 +- .../apache/hive/service/server/HiveServer2.java| 8 +- .../service/server/HiveServer2OomHookRunner.java | 31 ++-- 9 files changed, 400 insertions(+), 109 deletions(-) create mode 100644 ql/src/java/org/apache/hadoop/hive/ql/hooks/HiveHooks.java create mode 100644 ql/src/test/org/apache/hadoop/hive/ql/hooks/TestHiveHooks.java copy standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/messaging/event/filters/MessageFormatFilter.java => service/src/java/org/apache/hive/service/server/HiveServer2OomHookRunner.java (57%)
[hive] branch master updated (274986a -> 1cd3fb7)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 274986a HIVE-24146: Cleanup TaskExecutionException in GenericUDTFExplode (#1483) (Zhihua Deng reviewed by Zoltan Haindrich) add 1cd3fb7 HIVE-24248: TestMiniLlapLocalCliDriver[subquery_join_rewrite] is flaky (#1565) (Zhihua Deng reviewed by Krisztian Kasa) No new revisions were added by this update. Summary of changes: ql/src/test/queries/clientpositive/subquery_join_rewrite.q | 1 - .../results/clientpositive/llap/subquery_join_rewrite.q.out | 12 ++-- 2 files changed, 6 insertions(+), 7 deletions(-)
[hive] branch master updated: HIVE-24146: Cleanup TaskExecutionException in GenericUDTFExplode (#1483) (Zhihua Deng reviewed by Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 274986a HIVE-24146: Cleanup TaskExecutionException in
GenericUDTFExplode (#1483) (Zhihua Deng reviewed by Zoltan Haindrich)
274986a is described below
commit 274986a7dca674551dac3b65079a4411feb097aa
Author: dengzh
AuthorDate: Tue Oct 13 16:12:47 2020 +0800
HIVE-24146: Cleanup TaskExecutionException in GenericUDTFExplode (#1483)
(Zhihua Deng reviewed by Zoltan Haindrich)
---
.../hive/ql/exec/TaskExecutionException.java | 40 --
.../hive/ql/udf/generic/GenericUDTFExplode.java| 3 --
2 files changed, 43 deletions(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskExecutionException.java
b/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskExecutionException.java
deleted file mode 100644
index a6a38ae..000
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskExecutionException.java
+++ /dev/null
@@ -1,40 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.hadoop.hive.ql.exec;
-
-/**
- * Exception thrown by any task processor to signal a problem with task
- * execution.
- */
-public class TaskExecutionException extends RuntimeException {
-
- private static final long serialVersionUID = 1L;
-
- public TaskExecutionException(String msg) {
-super(msg);
- }
-
- public TaskExecutionException(String msg, Throwable cause) {
-super(msg, cause);
- }
-
- public TaskExecutionException(Throwable cause) {
-super(cause);
- }
-
-}
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFExplode.java
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFExplode.java
index 948cbb3..da1d76d 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFExplode.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFExplode.java
@@ -24,7 +24,6 @@ import java.util.Map;
import java.util.Map.Entry;
import org.apache.hadoop.hive.ql.exec.Description;
-import org.apache.hadoop.hive.ql.exec.TaskExecutionException;
import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
@@ -106,8 +105,6 @@ public class GenericUDTFExplode extends GenericUDTF {
forward(forwardMapObj);
}
break;
-default:
- throw new TaskExecutionException("explode() can only operate on an array
or a map");
}
}
[hive] branch master updated: HIVE-24107: Fix typo in ReloadFunctionsOperation (#1457) (Zhihua Deng reviewed by Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 7e81ca7 HIVE-24107: Fix typo in ReloadFunctionsOperation (#1457)
(Zhihua Deng reviewed by Zoltan Haindrich)
7e81ca7 is described below
commit 7e81ca7edad212346d1d4427a1e0cd4c91f3c0df
Author: dengzh
AuthorDate: Tue Oct 13 16:11:53 2020 +0800
HIVE-24107: Fix typo in ReloadFunctionsOperation (#1457) (Zhihua Deng
reviewed by Zoltan Haindrich)
---
.../hadoop/hive/ql/ddl/function/reload/ReloadFunctionsOperation.java| 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/ddl/function/reload/ReloadFunctionsOperation.java
b/ql/src/java/org/apache/hadoop/hive/ql/ddl/function/reload/ReloadFunctionsOperation.java
index 8af68a1..7d9f144 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/ddl/function/reload/ReloadFunctionsOperation.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/ddl/function/reload/ReloadFunctionsOperation.java
@@ -34,7 +34,7 @@ public class ReloadFunctionsOperation extends
DDLOperation
@Override
public int execute() throws HiveException {
try {
- Hive.get().reloadFunctions();
+ Hive.get(false).reloadFunctions();
return 0;
} catch (Exception e) {
context.getTask().setException(e);
[hive] branch master updated: HIVE-24069: HiveHistory should log the task that ends abnormally (#1429)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new d9588c9 HIVE-24069: HiveHistory should log the task that ends
abnormally (#1429)
d9588c9 is described below
commit d9588c9731806a381f3358ea7e53bb0050c504fc
Author: dengzh
AuthorDate: Tue Oct 13 16:10:47 2020 +0800
HIVE-24069: HiveHistory should log the task that ends abnormally (#1429)
---
ql/src/java/org/apache/hadoop/hive/ql/Executor.java | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/Executor.java
b/ql/src/java/org/apache/hadoop/hive/ql/Executor.java
index 3aba618..ce47d17 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/Executor.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/Executor.java
@@ -274,6 +274,10 @@ public class Executor {
TaskResult result = taskRun.getTaskResult();
int exitVal = result.getExitVal();
+
SessionState.get().getHiveHistory().setTaskProperty(driverContext.getQueryId(),
task.getId(),
+Keys.TASK_RET_CODE, String.valueOf(exitVal));
+SessionState.get().getHiveHistory().endTask(driverContext.getQueryId(),
task);
+
DriverUtils.checkInterrupted(driverState, driverContext, "when checking
the execution result.", hookContext,
SessionState.getPerfLogger());
@@ -284,10 +288,6 @@ public class Executor {
taskQueue.finished(taskRun);
-
SessionState.get().getHiveHistory().setTaskProperty(driverContext.getQueryId(),
task.getId(),
-Keys.TASK_RET_CODE, String.valueOf(exitVal));
-SessionState.get().getHiveHistory().endTask(driverContext.getQueryId(),
task);
-
if (task.getChildTasks() != null) {
for (Task child : task.getChildTasks()) {
if (TaskQueue.isLaunchable(child)) {
[hive] branch master updated (90d2275 -> 7fd24d9)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 90d2275 HIVE-24234: Improve checkHashModeEfficiency in VectorGroupByOperator (#1560) add 7fd24d9 HIVE-24063: SqlFunctionConverter#getHiveUDF handles cast before geting FunctionInfo (#1421) (Zhihua Deng reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hive/ql/optimizer/calcite/translator/SqlFunctionConverter.java| 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[hive] branch master updated: disable acid_stats2
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/master by this push: new 48a152d disable acid_stats2 48a152d is described below commit 48a152d089f687dc7f08f1acef651c3c8bfdbfd6 Author: Zoltan Haindrich AuthorDate: Mon Oct 12 12:15:38 2020 + disable acid_stats2 --- ql/src/test/queries/clientpositive/acid_stats2.q | 1 + 1 file changed, 1 insertion(+) diff --git a/ql/src/test/queries/clientpositive/acid_stats2.q b/ql/src/test/queries/clientpositive/acid_stats2.q index 119e82b..1126959 100644 --- a/ql/src/test/queries/clientpositive/acid_stats2.q +++ b/ql/src/test/queries/clientpositive/acid_stats2.q @@ -1,3 +1,4 @@ +--! qt:disabled:HIVE-24265 set hive.stats.dbclass=fs; set hive.stats.fetch.column.stats=true; set datanucleus.cache.collections=false;
[hive] branch master updated (f9ee863 -> d61c916)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from f9ee863 disable subquery_join_rewrite add d61c916 HIVE-23851: MSCK REPAIR Command With Partition Filtering Fails While Dropping Partitions (#1271) (Syed Shameerur Rahman reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hive/ql/ddl/misc/msck/MsckOperation.java | 1 - .../ql/metadata/SessionHiveMetaStoreClient.java| 2 +- .../ppr/PartitionExpressionForMetastore.java | 18 +++- .../queries/clientpositive/msck_repair_filter.q| 15 +++ .../clientpositive/llap/msck_repair_filter.q.out | 105 + .../hive/metastore/PartitionExpressionProxy.java | 2 +- .../hadoop/hive/metastore/conf/MetastoreConf.java | 5 + .../metastore/DefaultPartitionExpressionProxy.java | 2 +- .../metastore/MsckPartitionExpressionProxy.java| 3 +- .../apache/hadoop/hive/metastore/ObjectStore.java | 6 +- .../hadoop/hive/metastore/PartFilterExprUtil.java | 5 +- .../MockPartitionExpressionForMetastore.java | 2 +- .../hadoop/hive/metastore/TestOldSchema.java | 2 +- 13 files changed, 154 insertions(+), 14 deletions(-)
[hive] branch master updated: disable subquery_join_rewrite
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/master by this push: new f9ee863 disable subquery_join_rewrite f9ee863 is described below commit f9ee86399ce5197203e84bf9b4a3768c345ecf2a Author: Zoltan Haindrich AuthorDate: Fri Oct 9 07:43:43 2020 + disable subquery_join_rewrite --- ql/src/test/queries/clientpositive/subquery_join_rewrite.q | 1 + 1 file changed, 1 insertion(+) diff --git a/ql/src/test/queries/clientpositive/subquery_join_rewrite.q b/ql/src/test/queries/clientpositive/subquery_join_rewrite.q index 1f4ba71..18ca2ec 100644 --- a/ql/src/test/queries/clientpositive/subquery_join_rewrite.q +++ b/ql/src/test/queries/clientpositive/subquery_join_rewrite.q @@ -1,3 +1,4 @@ +--! qt:disabled:HIVE-24248 -- SORT_QUERY_RESULTS create table web_sales (ws_order_number int, ws_warehouse_sk int) stored as orc;
[hive] branch master updated: HIVE-24229. DirectSql fails in case of OracleDB. (#1552)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 4d68e56 HIVE-24229. DirectSql fails in case of OracleDB. (#1552)
4d68e56 is described below
commit 4d68e566539acd6426f0ed3c57050a10e0d94819
Author: Ayush Saxena
AuthorDate: Wed Oct 7 22:02:06 2020 +0530
HIVE-24229. DirectSql fails in case of OracleDB. (#1552)
---
.../hadoop/hive/metastore/MetastoreDirectSqlUtils.java | 17 +++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
index 942d577..53641d4 100644
---
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
+++
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java
@@ -35,6 +35,7 @@ import org.slf4j.LoggerFactory;
import javax.jdo.PersistenceManager;
import javax.jdo.Query;
+import java.math.BigDecimal;
import java.sql.Blob;
import java.sql.Clob;
import java.sql.SQLException;
@@ -168,7 +169,7 @@ class MetastoreDirectSqlUtils {
loopJoinOrderedResult(pm, partitions, queryText, 0, new
ApplyFunc() {
@Override
public void apply(Partition t, Object[] fields) {
-t.putToParameters((String)fields[1], extractSqlClob(fields[2]));
+t.putToParameters(extractSqlClob(fields[1]),
extractSqlClob(fields[2]));
}});
// Perform conversion of null map values
for (Partition t : partitions.values()) {
@@ -507,7 +508,9 @@ class MetastoreDirectSqlUtils {
/**
* Convert a boolean value returned from the RDBMS to a Java Boolean object.
- * MySQL has booleans, but e.g. Derby uses 'Y'/'N' mapping.
+ * MySQL has booleans, but e.g. Derby uses 'Y'/'N' mapping and Oracle DB
+ * doesn't supports boolean, so we have used Number to store the value,
+ * which maps to BigDecimal.
*
* @param value
* column value from the database
@@ -539,6 +542,16 @@ class MetastoreDirectSqlUtils {
// NOOP
}
}
+
+if (value instanceof BigDecimal) {
+ BigDecimal bigDecimal = (BigDecimal) value;
+ if (bigDecimal.intValue() == 0) {
+return false;
+ } else {
+return true;
+ }
+}
+LOG.debug("Value is of type {}", value.getClass());
throw new MetaException("Cannot extract boolean from column value " +
value);
}
[hive] branch master updated (0416770 -> 6198e63)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 0416770 HIVE-24178: Add managed location to SHOW CREATE DATABASE (Miklos Gergely, reviewed by David Mollitor) add 6198e63 HIVE-24193 : Select query on renamed hive acid table does not produce any output (#1520) (Raj Kumar Singh reviewed by Peter Vary) No new revisions were added by this update. Summary of changes: .../main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java| 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[hive] branch master updated: HIVE-24157: Strict mode to fail on CAST timestamp <-> numeric (#1497) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new f002c10 HIVE-24157: Strict mode to fail on CAST timestamp <-> numeric
(#1497) (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
f002c10 is described below
commit f002c103da25ea9826abff90aa84c0a1967c4a50
Author: Zoltan Haindrich
AuthorDate: Fri Oct 2 15:02:11 2020 +0200
HIVE-24157: Strict mode to fail on CAST timestamp <-> numeric (#1497)
(Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
---
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 2 +
data/conf/hive-site.xml| 6 ++
data/conf/llap/hive-site.xml | 5 ++
.../ql/udf/TimestampCastRestrictorResolver.java| 71 ++
.../apache/hadoop/hive/ql/udf/UDFToBoolean.java| 6 ++
.../org/apache/hadoop/hive/ql/udf/UDFToByte.java | 6 ++
.../org/apache/hadoop/hive/ql/udf/UDFToDouble.java | 6 ++
.../org/apache/hadoop/hive/ql/udf/UDFToFloat.java | 5 ++
.../apache/hadoop/hive/ql/udf/UDFToInteger.java| 6 ++
.../org/apache/hadoop/hive/ql/udf/UDFToLong.java | 5 ++
.../org/apache/hadoop/hive/ql/udf/UDFToShort.java | 6 ++
.../hive/ql/udf/generic/GenericUDFBaseCompare.java | 33 ++
.../hive/ql/udf/generic/GenericUDFTimestamp.java | 15 -
.../clientnegative/strict_numeric_to_timestamp.q | 2 +
.../clientnegative/strict_numeric_to_timestamp2.q | 3 +
.../clientnegative/strict_timestamp_to_numeric.q | 2 +
.../clientnegative/strict_timestamp_to_numeric2.q | 3 +
.../clientnegative/strict_timestamp_to_numeric3.q | 3 +
.../clientnegative/strict_timestamp_to_numeric4.q | 3 +
.../strict_numeric_to_timestamp.q.out | 1 +
.../strict_numeric_to_timestamp2.q.out | 9 +++
.../strict_timestamp_to_numeric.q.out | 1 +
.../strict_timestamp_to_numeric2.q.out | 9 +++
.../strict_timestamp_to_numeric3.q.out | 9 +++
.../strict_timestamp_to_numeric4.q.out | 9 +++
25 files changed, 223 insertions(+), 3 deletions(-)
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index 1e6e5ca..b0488db 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -1774,6 +1774,8 @@ public class HiveConf extends Configuration {
HIVE_STRICT_CHECKS_BUCKETING("hive.strict.checks.bucketing", true,
"Enabling strict bucketing checks disallows the following:\n" +
" Load into bucketed tables."),
+HIVE_STRICT_TIMESTAMP_CONVERSION("hive.strict.timestamp.conversion", true,
+"Restricts unsafe numeric to timestamp conversions"),
HIVE_LOAD_DATA_OWNER("hive.load.data.owner", "",
"Set the owner of files loaded using load data in managed tables."),
diff --git a/data/conf/hive-site.xml b/data/conf/hive-site.xml
index 53a38c8..f0f71f6 100644
--- a/data/conf/hive-site.xml
+++ b/data/conf/hive-site.xml
@@ -381,4 +381,10 @@
hive.scheduled.queries.executor.enabled
false
+
+
+ hive.strict.timestamp.conversion
+ false
+
+
diff --git a/data/conf/llap/hive-site.xml b/data/conf/llap/hive-site.xml
index fe4337b..7bc3791 100644
--- a/data/conf/llap/hive-site.xml
+++ b/data/conf/llap/hive-site.xml
@@ -402,4 +402,9 @@
Using property defined in HiveConf.ConfVars to test System
property overriding
+
+ hive.strict.timestamp.conversion
+ false
+
+
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/udf/TimestampCastRestrictorResolver.java
b/ql/src/java/org/apache/hadoop/hive/ql/udf/TimestampCastRestrictorResolver.java
new file mode 100644
index 000..edb0736
--- /dev/null
+++
b/ql/src/java/org/apache/hadoop/hive/ql/udf/TimestampCastRestrictorResolver.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
[hive] branch master updated: HIVE-24161: Support Oracle CLOB type in beeline (#1500) (Robbie Zhang reviewed by Yonghzi Chen)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 94437e0 HIVE-24161: Support Oracle CLOB type in beeline (#1500)
(Robbie Zhang reviewed by Yonghzi Chen)
94437e0 is described below
commit 94437e0f8e1d36debb17b7b20b60a964e23f9f66
Author: Robbie Zhang
AuthorDate: Mon Sep 28 20:29:02 2020 +1000
HIVE-24161: Support Oracle CLOB type in beeline (#1500) (Robbie Zhang
reviewed by Yonghzi Chen)
---
beeline/src/java/org/apache/hive/beeline/Rows.java| 2 +-
.../src/test/org/apache/hive/beeline/TestBufferedRows.java| 11 ++-
.../src/test/org/apache/hive/beeline/TestIncrementalRows.java | 1 +
.../org/apache/hive/beeline/TestJSONFileOutputFormat.java | 9 +
.../test/org/apache/hive/beeline/TestJSONOutputFormat.java| 9 +
.../test/org/apache/hive/beeline/TestTableOutputFormat.java | 9 +
6 files changed, 39 insertions(+), 2 deletions(-)
diff --git a/beeline/src/java/org/apache/hive/beeline/Rows.java
b/beeline/src/java/org/apache/hive/beeline/Rows.java
index 3d5eaec..7180f34 100644
--- a/beeline/src/java/org/apache/hive/beeline/Rows.java
+++ b/beeline/src/java/org/apache/hive/beeline/Rows.java
@@ -166,7 +166,7 @@ abstract class Rows implements Iterator {
} else if (o instanceof byte[]) {
value = convertBinaryArrayToString ? new String((byte[])o,
StandardCharsets.UTF_8) :
Base64.getEncoder().withoutPadding().encodeToString((byte[])o);
} else {
- value = o.toString();
+ value = rs.getString(i + 1);
}
if (beeLine.getOpts().getEscapeCRLF()) {
diff --git a/beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java
b/beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java
index 1add3c0..e02c753 100644
--- a/beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java
+++ b/beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java
@@ -107,6 +107,15 @@ public class TestBufferedRows {
return mockRow.getColumn(index);
}
});
+
+when(mockResultSet.getString(anyInt())).thenAnswer(new Answer() {
+ @Override
+ public String answer(InvocationOnMock invocation) {
+Object[] args = invocation.getArguments();
+int index = ((Integer) args[0]).intValue();
+return mockRow.getColumn(index);
+ }
+});
}
static class MockRow {
@@ -120,4 +129,4 @@ public class TestBufferedRows {
return rowData[idx - 1];
}
}
-}
\ No newline at end of file
+}
diff --git a/beeline/src/test/org/apache/hive/beeline/TestIncrementalRows.java
b/beeline/src/test/org/apache/hive/beeline/TestIncrementalRows.java
index eb4e27b..62420f8 100644
--- a/beeline/src/test/org/apache/hive/beeline/TestIncrementalRows.java
+++ b/beeline/src/test/org/apache/hive/beeline/TestIncrementalRows.java
@@ -115,6 +115,7 @@ public class TestIncrementalRows {
initNrOfResultSetCalls(10);
when(mockResultSet.getObject(1)).thenReturn("Hello World");
+when(mockResultSet.getString(1)).thenReturn("Hello World");
// IncrementalRows constructor should buffer the first
"incrementalBufferRows" rows
IncrementalRowsWithNormalization incrementalRowsWithNormalization = new
IncrementalRowsWithNormalization(
diff --git
a/beeline/src/test/org/apache/hive/beeline/TestJSONFileOutputFormat.java
b/beeline/src/test/org/apache/hive/beeline/TestJSONFileOutputFormat.java
index 7cb85ab..6805100 100644
--- a/beeline/src/test/org/apache/hive/beeline/TestJSONFileOutputFormat.java
+++ b/beeline/src/test/org/apache/hive/beeline/TestJSONFileOutputFormat.java
@@ -102,5 +102,14 @@ public class TestJSONFileOutputFormat {
return mockRow.getColumn(index);
}
});
+
+when(mockResultSet.getString(anyInt())).thenAnswer(new Answer() {
+ @Override
+ public String answer(final InvocationOnMock invocation) {
+Object[] args = invocation.getArguments();
+int index = ((Integer) args[0]);
+return mockRow.getColumn(index);
+ }
+});
}
}
diff --git a/beeline/src/test/org/apache/hive/beeline/TestJSONOutputFormat.java
b/beeline/src/test/org/apache/hive/beeline/TestJSONOutputFormat.java
index ac216d7..1ea6cd6 100644
--- a/beeline/src/test/org/apache/hive/beeline/TestJSONOutputFormat.java
+++ b/beeline/src/test/org/apache/hive/beeline/TestJSONOutputFormat.java
@@ -104,5 +104,14 @@ public class TestJSONOutputFormat {
return mockRow.getColumn(index);
}
});
+
+when(mockResultSet.getString(anyInt())).thenAnswer(new Answer() {
+ @Override
+ public String answer(final InvocationOnMock invocation) {
+Object[] args = invocation.getArguments();
+int index = ((Integer) args[0]);
+return mockRow.getColumn(index);
+
[hive] branch master updated (4bf3c2d -> bce5c32)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 4bf3c2d HIVE-24162: Query based compaction looses bloom filter (Peter Varga, reviewed by Laszlo Pinter and Karen Coppage) add bce5c32 ignore test No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hive/ql/txn/compactor/TestMmCompactorOnMr.java | 1 + 1 file changed, 1 insertion(+)
[hive] branch master updated (cf12df5 -> d69aa36)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from cf12df5 HIVE-24160: Scheduled executions must allow state transition EXECUTING->TIMED_OUT #1496 add d69aa36 HIVE-24084: Push Aggregates thru joins in case it re-groups previously unique columns (#1439) No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hive/conf/HiveConf.java | 2 + .../java/org/apache/hadoop/hive/ql/QTestUtil.java | 4 + .../rules/HiveAggregateJoinTransposeRule.java | 97 +- .../hadoop/hive/ql/parse/CalcitePlanner.java | 6 +- ql/src/test/queries/clientpositive/tpch18.q| 136 .../llap/constraints_optimization.q.out| 173 - .../llap/materialized_view_rewrite_2.q.out | 73 ++-- .../llap/materialized_view_rewrite_part_2.q.out| 75 ++-- .../clientpositive/llap/subquery_notin.q.out | 208 +-- .../test/results/clientpositive/llap/tpch18.q.out | 360 +++ .../perf/tez/constraints/cbo_query23.q.out | 54 +-- .../perf/tez/constraints/cbo_query54.q.out | 66 ++-- .../perf/tez/constraints/query23.q.out | 358 +-- .../perf/tez/constraints/query54.q.out | 386 ++--- 14 files changed, 1272 insertions(+), 726 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/tpch18.q create mode 100644 ql/src/test/results/clientpositive/llap/tpch18.q.out
[hive] branch master updated (3a8edc0 -> cf12df5)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 3a8edc0 HIVE-22290: ObjectStore.cleanWriteNotificationEvents and ObjectStore.cleanupEvents OutOfMemory on large number of pending events (#1484) add cf12df5 HIVE-24160: Scheduled executions must allow state transition EXECUTING->TIMED_OUT #1496 No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java| 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
[hive] branch master updated: HIVE-24039 : Update jquery version to mitigate CVE-2020-11023 (#1403)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 4b31ede HIVE-24039 : Update jquery version to mitigate CVE-2020-11023
(#1403)
4b31ede is described below
commit 4b31ede11189a76627469acc20298cde2676f4dd
Author: Rajkumar Singh
AuthorDate: Mon Sep 14 23:58:58 2020 -0700
HIVE-24039 : Update jquery version to mitigate CVE-2020-11023 (#1403)
Co-authored-by: rajkrrsingh
---
service/src/resources/hive-webapps/static/js/jquery.min.js | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/service/src/resources/hive-webapps/static/js/jquery.min.js
b/service/src/resources/hive-webapps/static/js/jquery.min.js
index 4d9b3a2..47b6397 100644
--- a/service/src/resources/hive-webapps/static/js/jquery.min.js
+++ b/service/src/resources/hive-webapps/static/js/jquery.min.js
@@ -1,2 +1,2 @@
-/*! jQuery v3.3.1 | (c) JS Foundation and other contributors |
jquery.org/license */
-!function(e,t){"use strict";"object"==typeof module&&"object"==typeof
module.exports?module.exports=e.document?t(e,!0):function(e){if(!e.document)throw
new Error("jQuery requires a window with a document");return
t(e)}:t(e)}("undefined"!=typeof window?window:this,function(e,t){"use
strict";var
n=[],r=e.document,i=Object.getPrototypeOf,o=n.slice,a=n.concat,s=n.push,u=n.indexOf,l={},c=l.toString,f=l.hasOwnProperty,p=f.toString,d=p.call(Object),h={},g=function
e(t){return"function"==typeof [...]
+/*! jQuery v3.5.0 | (c) JS Foundation and other contributors |
jquery.org/license */
+!function(e,t){"use strict";"object"==typeof module&&"object"==typeof
module.exports?module.exports=e.document?t(e,!0):function(e){if(!e.document)throw
new Error("jQuery requires a window with a document");return
t(e)}:t(e)}("undefined"!=typeof window?window:this,function(C,e){"use
strict";var t=[],r=Object.getPrototypeOf,s=t.slice,g=t.flat?function(e){return
t.flat.call(e)}:function(e){return
t.concat.apply([],e)},u=t.push,i=t.indexOf,n={},o=n.toString,v=n.hasOwnProperty,a=v.toString,l=
[...]
[hive] branch master updated: HIVE-24072: HiveAggregateJoinTransposeRule may try to create an invalid transformation (#1432) (Zoltan Haindrich via Jesus Camacho Rodriguez)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new d078d86 HIVE-24072: HiveAggregateJoinTransposeRule may try to create
an invalid transformation (#1432) (Zoltan Haindrich via Jesus Camacho Rodriguez)
d078d86 is described below
commit d078d86bad1e891524fa82d69a6895188438c008
Author: Zoltan Haindrich
AuthorDate: Wed Sep 9 10:36:56 2020 +0200
HIVE-24072: HiveAggregateJoinTransposeRule may try to create an invalid
transformation (#1432) (Zoltan Haindrich via Jesus Camacho Rodriguez)
---
.../ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java | 6 +-
.../authorization/plugin/metastore/events/ReadTableEvent.java | 2 +-
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java
b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java
index 76ca54b..197fcd5 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java
@@ -25,6 +25,8 @@ import java.util.SortedMap;
import java.util.TreeMap;
import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
+
import org.apache.calcite.linq4j.Ord;
import org.apache.calcite.plan.RelOptCost;
import org.apache.calcite.plan.RelOptRuleCall;
@@ -175,7 +177,9 @@ public class HiveAggregateJoinTransposeRule extends
AggregateJoinTransposeRule {
}
if (unique) {
++uniqueCount;
- side.newInput = joinInput;
+ relBuilder.push(joinInput);
+
relBuilder.project(belowAggregateKey.asList().stream().map(relBuilder::field).collect(Collectors.toList()));
+ side.newInput = relBuilder.build();
} else {
List belowAggCalls = new ArrayList<>();
final SqlSplittableAggFunction.Registry
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/ReadTableEvent.java
b/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/ReadTableEvent.java
index 21152c3..083e2b9 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/ReadTableEvent.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/ReadTableEvent.java
@@ -20,7 +20,7 @@
package
org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.events;
import org.apache.commons.logging.Log;
-import org.apache.commons.logging.LogFactory;;
+import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.hive.metastore.api.Table;
import org.apache.hadoop.hive.metastore.events.PreEventContext;
import org.apache.hadoop.hive.metastore.events.PreReadTableEvent;
[hive] branch master updated: HIVE-24031: Infinite planning time on syntactically big queries (#1424) (Stamatis Zampetakis reviewed by Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 587b402 HIVE-24031: Infinite planning time on syntactically big
queries (#1424) (Stamatis Zampetakis reviewed by Zoltan Haindrich)
587b402 is described below
commit 587b402aa6f5357cf6bd16893606b48b3406e9e5
Author: Stamatis Zampetakis
AuthorDate: Tue Sep 8 18:45:23 2020 +0200
HIVE-24031: Infinite planning time on syntactically big queries (#1424)
(Stamatis Zampetakis reviewed by Zoltan Haindrich)
1. Drop the defensive copy of children inside ASTNode#getChildren.
2. Protect clients by accidentally modifying the list via an
unmodifiable collection.
Profiling shows the vast majority of time spend on creating defensive
copies of the node expression list inside ASTNode#getChildren.
The method is called extensively from various places in the code
especially those walking over the expression tree so it needs to be
efficient.
Most of the time creating defensive copies is not necessary. For those
cases (if any) that the list needs to be modified clients should perform
a copy themselves.
---
parser/src/java/org/apache/hadoop/hive/ql/parse/ASTNode.java | 9 +++--
.../apache/hadoop/hive/ql/parse/TestParseDriverIntervals.java| 2 +-
.../storage/skewed/AlterTableSetSkewedLocationAnalyzer.java | 2 +-
.../org/apache/hadoop/hive/ql/parse/MergeSemanticAnalyzer.java | 2 +-
.../java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java | 2 +-
5 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/parser/src/java/org/apache/hadoop/hive/ql/parse/ASTNode.java
b/parser/src/java/org/apache/hadoop/hive/ql/parse/ASTNode.java
index 2d611fb..f51de08 100644
--- a/parser/src/java/org/apache/hadoop/hive/ql/parse/ASTNode.java
+++ b/parser/src/java/org/apache/hadoop/hive/ql/parse/ASTNode.java
@@ -21,6 +21,7 @@ package org.apache.hadoop.hive.ql.parse;
import java.io.Serializable;
import java.util.ArrayDeque;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.Deque;
import java.util.List;
@@ -74,11 +75,15 @@ public class ASTNode extends CommonTree implements
Node,Serializable {
* @see org.apache.hadoop.hive.ql.lib.Node#getChildren()
*/
@Override
- public ArrayList getChildren() {
+ public List getChildren() {
if (super.getChildCount() == 0) {
return null;
}
-return new ArrayList<>((List) super.getChildren());
+// We know that children always contains Node instances so the cast is
safe.
+assert this.children.get(0) instanceof Node;
+@SuppressWarnings("unchecked") // Cast safe by design of the class
+List nodeList = (List) (List) this.children;
+return Collections.unmodifiableList(nodeList);
}
/*
diff --git
a/parser/src/test/org/apache/hadoop/hive/ql/parse/TestParseDriverIntervals.java
b/parser/src/test/org/apache/hadoop/hive/ql/parse/TestParseDriverIntervals.java
index 3abaf24..fd3cc76 100644
---
a/parser/src/test/org/apache/hadoop/hive/ql/parse/TestParseDriverIntervals.java
+++
b/parser/src/test/org/apache/hadoop/hive/ql/parse/TestParseDriverIntervals.java
@@ -70,7 +70,7 @@ public class TestParseDriverIntervals {
return n;
}
}
-ArrayList children = n.getChildren();
+List children = n.getChildren();
if (children != null) {
for (Node c : children) {
ASTNode r = findFunctionNode((ASTNode) c);
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/skewed/AlterTableSetSkewedLocationAnalyzer.java
b/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/skewed/AlterTableSetSkewedLocationAnalyzer.java
index 488f76e..d9fe352 100644
---
a/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/skewed/AlterTableSetSkewedLocationAnalyzer.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/skewed/AlterTableSetSkewedLocationAnalyzer.java
@@ -56,7 +56,7 @@ public class AlterTableSetSkewedLocationAnalyzer extends
AbstractAlterTableAnaly
@Override
protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
throws SemanticException {
-ArrayList locationNodes = command.getChildren();
+List locationNodes = command.getChildren();
if (locationNodes == null) {
throw new
SemanticException(ErrorMsg.ALTER_TBL_SKEWED_LOC_NO_LOC.getMsg());
}
diff --git
a/ql/src/java/org/apache/hadoop/hive/ql/parse/MergeSemanticAnalyzer.java
b/ql/src/java/org/apache/hadoop/hive/ql/parse/MergeSemanticAnalyzer.java
index cea7b11..bdcbf62 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/MergeSemanticAnalyzer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/MergeSemanticAnalyzer.java
@@ -593,7 +593,7 @@ public class MergeSemanticAnalyzer extends
Rewrite
[hive] branch master updated (bba73fb -> 4a90b9d)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from bba73fb HIVE-23408: Hive on Tez : Kafka storage handler broken in secure environment (#1379) (Laszlo Bodor reviewed by Ashutosh Chauhan) add 4a90b9d HIVE-24104: NPE due to null key columns in ReduceSink after deduplication (#1453) (Stamatis Zampetakis reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../correlation/ReduceSinkDeDuplicationUtils.java | 8 +- .../apache/hadoop/hive/ql/plan/ReduceSinkDesc.java | 12 +- .../dynamic_semijoin_reduction_on_aggcol.q | 1 + .../clientpositive/reduce_deduplicate_null_keys.q | 40 + .../llap/reduce_deduplicate_null_keys.q.out| 168 + 5 files changed, 221 insertions(+), 8 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/reduce_deduplicate_null_keys.q create mode 100644 ql/src/test/results/clientpositive/llap/reduce_deduplicate_null_keys.q.out
[hive] branch master updated (f9668a2 -> 5656763)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from f9668a2 HIVE-23302: Create HiveJdbcDatabaseAccessor for JDBC storage handler (Jesus Camacho Rodriguez, reviewed by Krisztian Kasa) add 5656763 HIVE-23649: Spotbugs should ignore generated thrift files (#1426) No new revisions were added by this update. Summary of changes: Jenkinsfile | 3 ++- spotbugs/spotbugs-exclude.xml | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-)
[hive] branch master updated (5bc50f6 -> e800aff)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 5bc50f6 HIVE-23961:Enable external table replication by default (Aasha Medhi, reviewed by Pravin Kumar Sinha) add e800aff HIVE-23959: Provide an option to wipe out column stats for partitioned tables in case of column removal (#1341) (Zoltan Haindrich reviewed by Peter Vary)) No new revisions were added by this update. Summary of changes: .../hcatalog/listener/DummyRawStoreFailEvent.java | 5 + .../colstats_remove_on_col_replace.q | 25 +++ .../colstats_remove_on_col_replace2.q | 13 ++ .../llap/colstats_remove_on_col_replace.q.out | 232 + .../llap/colstats_remove_on_col_replace2.q.out | 117 +++ .../hadoop/hive/metastore/conf/MetastoreConf.java | 6 + .../hadoop/hive/metastore/HiveAlterHandler.java| 39 ++-- .../hadoop/hive/metastore/MetaStoreDirectSql.java | 57 +++-- .../apache/hadoop/hive/metastore/ObjectStore.java | 73 +++ .../org/apache/hadoop/hive/metastore/RawStore.java | 2 + .../hadoop/hive/metastore/cache/CachedStore.java | 9 +- .../metastore/DummyRawStoreControlledCommit.java | 6 + .../metastore/DummyRawStoreForJdoConnection.java | 6 + 13 files changed, 560 insertions(+), 30 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/colstats_remove_on_col_replace.q create mode 100644 ql/src/test/queries/clientpositive/colstats_remove_on_col_replace2.q create mode 100644 ql/src/test/results/clientpositive/llap/colstats_remove_on_col_replace.q.out create mode 100644 ql/src/test/results/clientpositive/llap/colstats_remove_on_col_replace2.q.out
[hive] branch master updated: disable unstable tests
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new fdfd136 disable unstable tests
fdfd136 is described below
commit fdfd136083de39493ea36f64ce34ca22c348c537
Author: Zoltan Haindrich
AuthorDate: Tue Aug 4 09:57:42 2020 +
disable unstable tests
---
.../hcatalog/templeton/TestConcurrentJobRequestsThreadsAndTimeout.java | 1 +
.../hadoop/hive/ql/parse/TestStatsReplicationScenariosMigration.java | 1 +
.../ql/parse/TestStatsReplicationScenariosMigrationNoAutogather.java | 1 +
ql/src/test/queries/clientpositive/druidmini_dynamic_partition.q | 3 ++-
ql/src/test/queries/clientpositive/druidmini_expressions.q | 1 +
ql/src/test/queries/clientpositive/kafka_storage_handler.q | 2 ++
6 files changed, 8 insertions(+), 1 deletion(-)
diff --git
a/hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestConcurrentJobRequestsThreadsAndTimeout.java
b/hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestConcurrentJobRequestsThreadsAndTimeout.java
index d351a98..abb81a0 100644
---
a/hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestConcurrentJobRequestsThreadsAndTimeout.java
+++
b/hcatalog/webhcat/svr/src/test/java/org/apache/hive/hcatalog/templeton/TestConcurrentJobRequestsThreadsAndTimeout.java
@@ -33,6 +33,7 @@ import static org.junit.Assert.assertTrue;
* Requests and job request execution time outs. Verify that we get
appropriate exceptions
* and exception message.
*/
+@org.junit.Ignore("HIVE-23983")
public class TestConcurrentJobRequestsThreadsAndTimeout extends
ConcurrentJobRequestsTestBase {
private static AppConfig config;
diff --git
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigration.java
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigration.java
index 4b3bf4f..8544d32 100644
---
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigration.java
+++
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigration.java
@@ -29,6 +29,7 @@ import java.util.Map;
/**
* Tests statistics replication for ACID tables.
*/
+@org.junit.Ignore("HIVE-23986")
public class TestStatsReplicationScenariosMigration extends
TestStatsReplicationScenarios {
@Rule
public final TestName testName = new TestName();
diff --git
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigrationNoAutogather.java
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigrationNoAutogather.java
index 7fc3bc5..21790c5 100644
---
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigrationNoAutogather.java
+++
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMigrationNoAutogather.java
@@ -29,6 +29,7 @@ import java.util.Map;
/**
* Tests statistics replication for ACID tables.
*/
+@org.junit.Ignore("HIVE-23982")
public class TestStatsReplicationScenariosMigrationNoAutogather extends
TestStatsReplicationScenarios {
@Rule
public final TestName testName = new TestName();
diff --git a/ql/src/test/queries/clientpositive/druidmini_dynamic_partition.q
b/ql/src/test/queries/clientpositive/druidmini_dynamic_partition.q
index 36a68d0..eacdcfe 100644
--- a/ql/src/test/queries/clientpositive/druidmini_dynamic_partition.q
+++ b/ql/src/test/queries/clientpositive/druidmini_dynamic_partition.q
@@ -1,3 +1,4 @@
+--! qt:disabled:HIVE-23984
--! qt:dataset:alltypesorc
SET hive.vectorized.execution.enabled=true ;
SET hive.ctas.external.tables=true;
@@ -186,4 +187,4 @@ SELECT sum(cint), sum(cbigint) FROM
druid_max_size_partition ;
DROP TABLE druid_partitioned_table_0;
DROP TABLE druid_partitioned_table;
-DROP TABLE druid_max_size_partition;
\ No newline at end of file
+DROP TABLE druid_max_size_partition;
diff --git a/ql/src/test/queries/clientpositive/druidmini_expressions.q
b/ql/src/test/queries/clientpositive/druidmini_expressions.q
index 59ffb5b..2347d62 100644
--- a/ql/src/test/queries/clientpositive/druidmini_expressions.q
+++ b/ql/src/test/queries/clientpositive/druidmini_expressions.q
@@ -1,3 +1,4 @@
+--! qt:disabled:HIVE-23984
--! qt:dataset:druid_table_alltypesorc
SET hive.ctas.external.tables=true;
diff --git a/ql/src/test/queries/clientpositive/kafka_storage_handler.q
b/ql/src/test/queries/clientpositive/kafka_storage_handler.q
index f4670f5..b8283e7 100644
--- a/ql/src/test/queries/clientpositive/kafka_storage_handler.q
+++ b/ql/src/test/queries/clientpositive/kafka_storage_handler.q
@@ -1,3 +1,5 @@
+--!
[hive] branch master updated: HIVE-23847: Extracting hive-parser module broke exec jar upload in tez (#1252)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/master by this push: new 28b6384 HIVE-23847: Extracting hive-parser module broke exec jar upload in tez (#1252) 28b6384 is described below commit 28b6384e9ba287188015418b4c38c85dfdde8133 Author: Antal Sinkovits AuthorDate: Sun Aug 2 21:41:06 2020 +0200 HIVE-23847: Extracting hive-parser module broke exec jar upload in tez (#1252) --- ql/pom.xml | 1 + 1 file changed, 1 insertion(+) diff --git a/ql/pom.xml b/ql/pom.xml index 12b08ec..acd9bc1 100644 --- a/ql/pom.xml +++ b/ql/pom.xml @@ -1040,6 +1040,7 @@ org.apache.hive:hive-common org.apache.hive:hive-udf + org.apache.hive:hive-parser org.apache.hive:hive-exec org.apache.hive:hive-serde org.apache.hive:hive-llap-common
[hive] branch master updated (cf5c4f2 -> abb5d7e)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from cf5c4f2 HIVE-23911: CBO fails when query has distinct in function and having clause (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) add abb5d7e HIVE-23892 HIVE-23750: addendum No new revisions were added by this update. Summary of changes: .../clientpositive/llap/cardinality_preserving_join_opt.q.out | 8 1 file changed, 4 insertions(+), 4 deletions(-)
[hive] branch master updated (7be80c3 -> 7229a91)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 7be80c3 HIVE-23915: Improve Github PR Template (#1303) (Hunter Logan reviewed by Zoltan Haindrich) add 7229a91 disable remaining external_jdbc tests No new revisions were added by this update. Summary of changes: ql/src/test/queries/clientpositive/external_jdbc_table.q| 1 + ql/src/test/queries/clientpositive/external_jdbc_table2.q | 1 + ql/src/test/queries/clientpositive/external_jdbc_table4.q | 1 + ql/src/test/queries/clientpositive/external_jdbc_table_perf.q | 1 + ql/src/test/queries/clientpositive/external_jdbc_table_typeconversion.q | 1 + 5 files changed, 5 insertions(+)
[hive] branch master updated (ee27dd3 -> 7be80c3)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from ee27dd3 HIVE-23894 - SubmitDag should not be retried incase of query cancel (#1293) add 7be80c3 HIVE-23915: Improve Github PR Template (#1303) (Hunter Logan reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .github/pull_request_template.md | 50 1 file changed, 45 insertions(+), 5 deletions(-)
[hive] branch master updated: HIVE-23727: Improve SQLOperation log handling when canceling background (#1272) (Zhihua Deng reviewed by Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new ffbbd84 HIVE-23727: Improve SQLOperation log handling when canceling
background (#1272) (Zhihua Deng reviewed by Zoltan Haindrich)
ffbbd84 is described below
commit ffbbd84c2c6bd1d56989f2fd56608f75278ae546
Author: dengzh
AuthorDate: Thu Jul 30 22:31:14 2020 +0800
HIVE-23727: Improve SQLOperation log handling when canceling background
(#1272) (Zhihua Deng reviewed by Zoltan Haindrich)
---
.../hive/service/cli/operation/SQLOperation.java | 28 --
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git
a/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
b/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
index a76878a..0021828 100644
--- a/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
+++ b/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
@@ -172,10 +172,10 @@ public class SQLOperation extends
ExecuteStatementOperation {
timeoutExecutor.schedule(() -> {
try {
final String queryId = queryState.getQueryId();
-log.info("Query timed out after: " + queryTimeout + " seconds.
Cancelling the execution now: " + queryId);
+log.info("Query timed out after: {} seconds. Cancelling the
execution now: {}", queryTimeout, queryId);
SQLOperation.this.cancel(OperationState.TIMEDOUT);
} catch (HiveSQLException e) {
-log.error("Error cancelling the query after timeout: " +
queryTimeout + " seconds", e);
+log.error("Error cancelling the query after timeout: {} seconds",
queryTimeout, e);
}
return null;
}, queryTimeout, TimeUnit.SECONDS);
@@ -206,6 +206,9 @@ public class SQLOperation extends ExecuteStatementOperation
{
throw toSQLException("Error while compiling statement", e);
} catch (Throwable e) {
setState(OperationState.ERROR);
+ if (e instanceof OutOfMemoryError) {
+throw e;
+ }
throw new HiveSQLException("Error running query", e);
}
}
@@ -229,11 +232,8 @@ public class SQLOperation extends
ExecuteStatementOperation {
* may return a non-zero response code. We will simply return if the
operation state is
* CANCELED, TIMEDOUT, CLOSED or FINISHED, otherwise throw an exception
*/
- if ((getStatus().getState() == OperationState.CANCELED)
- || (getStatus().getState() == OperationState.TIMEDOUT)
- || (getStatus().getState() == OperationState.CLOSED)
- || (getStatus().getState() == OperationState.FINISHED)) {
-log.warn("Ignore exception in terminal state", e);
+ if (getStatus().getState().isTerminal()) {
+log.warn("Ignore exception in terminal state: {}",
getStatus().getState(), e);
return;
}
setState(OperationState.ERROR);
@@ -241,6 +241,8 @@ public class SQLOperation extends ExecuteStatementOperation
{
throw toSQLException("Error while compiling statement",
(CommandProcessorException)e);
} else if (e instanceof HiveSQLException) {
throw (HiveSQLException) e;
+ } else if (e instanceof OutOfMemoryError) {
+throw (OutOfMemoryError) e;
} else {
throw new HiveSQLException("Error running query", e);
}
@@ -341,7 +343,7 @@ public class SQLOperation extends ExecuteStatementOperation
{
currentUGI.doAs(doAsAction);
} catch (Exception e) {
setOperationException(new HiveSQLException(e));
-log.error("Error running hive query as user : " +
currentUGI.getShortUserName(), e);
+log.error("Error running hive query as user : {}",
currentUGI.getShortUserName(), e);
} finally {
/**
* We'll cache the ThreadLocal RawStore object for this background
thread for an orderly cleanup
@@ -383,9 +385,9 @@ public class SQLOperation extends ExecuteStatementOperation
{
boolean success = backgroundHandle.cancel(true);
String queryId = queryState.getQueryId();
if (success) {
- log.info("The running operation has been successfully interrupted: "
+ queryId);
-} else if (state == OperationState.CANCELED) {
- log.info("The running operation could not be cancelled, typically
because it has already completed normally: " + queryId);
+ log.info("The running operation has been successfully interrupted:
{}", queryId);
+} else {
+ log.info("The running operation could not be cancelled, typically
because it has alr
[hive] branch master updated: HIVE-23806: Avoid clearing column stat states in all partition in case schema is extended (#1215) (Zoltan Haindrich reviewed by Peter Vary)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 11cbf17 HIVE-23806: Avoid clearing column stat states in all
partition in case schema is extended (#1215) (Zoltan Haindrich reviewed by
Peter Vary)
11cbf17 is described below
commit 11cbf17e57dc13b800aa9ef4f987582b9219e3dd
Author: Zoltan Haindrich
AuthorDate: Wed Jul 29 10:02:26 2020 +0200
HIVE-23806: Avoid clearing column stat states in all partition in case
schema is extended (#1215) (Zoltan Haindrich reviewed by Peter Vary)
---
.../org/apache/hadoop/hive/metastore/HiveAlterHandler.java | 8 ++--
.../hadoop/hive/metastore/utils/MetaStoreServerUtils.java | 14 ++
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
index 2beeab4..0cfb329 100644
---
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
+++
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
@@ -59,7 +59,6 @@ import java.io.IOException;
import java.net.URI;
import java.util.ArrayList;
import java.util.Collections;
-import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
@@ -362,7 +361,12 @@ public class HiveAlterHandler implements AlterHandler {
if (isPartitionedTable) {
//Currently only column related changes can be cascaded in alter
table
- if(!MetaStoreServerUtils.areSameColumns(oldt.getSd().getCols(),
newt.getSd().getCols())) {
+ boolean runPartitionMetadataUpdate =
+ (cascade &&
!MetaStoreServerUtils.areSameColumns(oldt.getSd().getCols(),
newt.getSd().getCols()));
+ // we may skip the update entirely if there are only new columns
added
+ runPartitionMetadataUpdate |=
+ !cascade &&
!MetaStoreServerUtils.arePrefixColumns(oldt.getSd().getCols(),
newt.getSd().getCols());
+ if (runPartitionMetadataUpdate) {
parts = msdb.getPartitions(catName, dbname, name, -1);
for (Partition part : parts) {
Partition oldPart = new Partition(part);
diff --git
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
index d6cf43c..1969ddd 100644
---
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
+++
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java
@@ -32,6 +32,7 @@ import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Objects;
@@ -501,6 +502,19 @@ public class MetaStoreServerUtils {
return ListUtils.isEqualList(oldCols, newCols);
}
+ /**
+ * Returns true if p is a prefix of s.
+ */
+ public static boolean arePrefixColumns(List p,
List s) {
+if (p == s) {
+ return true;
+}
+if (p.size() > s.size()) {
+ return false;
+}
+return ListUtils.isEqualList(p, s.subList(0, p.size()));
+ }
+
public static void updateBasicState(EnvironmentContext environmentContext,
Map
params) {
if (params == null) {
[hive] branch master updated: disable TestStatsReplicationScenariosMMNoAutogather, TestCodahaleReportersConf
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 5684401 disable TestStatsReplicationScenariosMMNoAutogather,
TestCodahaleReportersConf
5684401 is described below
commit 56844015ffac23c10bfb898f27960eb5acb923d5
Author: Zoltan Haindrich
AuthorDate: Wed Jul 29 07:24:17 2020 +
disable TestStatsReplicationScenariosMMNoAutogather,
TestCodahaleReportersConf
---
.../hadoop/hive/common/metrics/metrics2/TestCodahaleReportersConf.java | 1 +
.../hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java | 1 +
2 files changed, 2 insertions(+)
diff --git
a/common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleReportersConf.java
b/common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleReportersConf.java
index e89a605..80db730 100644
---
a/common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleReportersConf.java
+++
b/common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleReportersConf.java
@@ -33,6 +33,7 @@ import java.io.File;
/**
* Unit tests for Codahale reporter config backward compatibility
*/
+@org.junit.Ignore("HIVE-23945")
public class TestCodahaleReportersConf {
private static File workDir = new File(System.getProperty("test.tmp.dir"));
diff --git
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java
index 1d77796..9df81b3 100644
---
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java
+++
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java
@@ -30,6 +30,7 @@ import java.util.Map;
/**
* Tests statistics replication for ACID tables.
*/
+@org.junit.Ignore("HIVE-23944")
public class TestStatsReplicationScenariosMMNoAutogather extends
TestStatsReplicationScenarios {
@Rule
public final TestName testName = new TestName();
[hive] branch master updated (e34acf5 -> 287707c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from e34acf5 HIVE-23835:Repl Dump should dump function binaries to staging directory( Pravin Kumar Sinha, reviewed by Aasha Medhi) add 287707c ignore TestSSL No new revisions were added by this update. Summary of changes: itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSSL.java | 1 + 1 file changed, 1 insertion(+)
[hive] branch master updated (1497c8b -> 7dd4eb1)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 1497c8b HIVE-23736: Disable topn in ReduceSinkOp if a TNK is introduced (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez) add 2cde4ee disable unstable tests add 7dd4eb1 HIVE-23736: addendum No new revisions were added by this update. Summary of changes: .../apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosACID.java | 1 + .../apache/hadoop/hive/ql/parse/TestTableLevelReplicationScenarios.java | 1 + .../src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java | 1 + .../queries/clientpositive/schema_evol_text_vec_part_all_primitive.q | 1 + .../results/clientpositive/llap/special_character_in_tabnames_1.q.out| 1 - .../clientpositive/llap/special_character_in_tabnames_quotes_1.q.out | 1 - 6 files changed, 4 insertions(+), 2 deletions(-)
[hive] branch master updated: disable flaky tests
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/master by this push: new d35fc7c disable flaky tests d35fc7c is described below commit d35fc7c125446cf2f0578f75f9263aeafa728699 Author: Zoltan Haindrich AuthorDate: Thu Jul 23 13:06:47 2020 + disable flaky tests --- ql/src/test/queries/clientpositive/external_jdbc_auth.q| 1 + ql/src/test/queries/clientpositive/external_jdbc_table3.q | 1 + ql/src/test/queries/clientpositive/external_jdbc_table_partition.q | 1 + 3 files changed, 3 insertions(+) diff --git a/ql/src/test/queries/clientpositive/external_jdbc_auth.q b/ql/src/test/queries/clientpositive/external_jdbc_auth.q index f4cbe94..27587e6 100644 --- a/ql/src/test/queries/clientpositive/external_jdbc_auth.q +++ b/ql/src/test/queries/clientpositive/external_jdbc_auth.q @@ -1,3 +1,4 @@ +--! qt:disabled:HIVE-23910 --! qt:dataset:src CREATE TEMPORARY FUNCTION dboutput AS 'org.apache.hadoop.hive.contrib.genericudf.example.GenericUDFDBOutput'; diff --git a/ql/src/test/queries/clientpositive/external_jdbc_table3.q b/ql/src/test/queries/clientpositive/external_jdbc_table3.q index 3b6092a..4c8f7bb 100644 --- a/ql/src/test/queries/clientpositive/external_jdbc_table3.q +++ b/ql/src/test/queries/clientpositive/external_jdbc_table3.q @@ -1,3 +1,4 @@ +--! qt:disabled:HIVE-23910 --! qt:dataset:src -- SORT_QUERY_RESULTS diff --git a/ql/src/test/queries/clientpositive/external_jdbc_table_partition.q b/ql/src/test/queries/clientpositive/external_jdbc_table_partition.q index 4759af8..8f1ec42 100644 --- a/ql/src/test/queries/clientpositive/external_jdbc_table_partition.q +++ b/ql/src/test/queries/clientpositive/external_jdbc_table_partition.q @@ -1,3 +1,4 @@ +--! qt:disabled:HIVE-23910 --! qt:dataset:src CREATE TEMPORARY FUNCTION dboutput AS 'org.apache.hadoop.hive.contrib.genericudf.example.GenericUDFDBOutput';
[hive] branch master updated (9418c08 -> e90b7c7)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 9418c08 Clean up Driver (Miklos Gergely, reviewed by Peter Vary) (#1222) add e90b7c7 ignore flaky tests No new revisions were added by this update. Summary of changes: .../src/test/org/apache/hadoop/hive/kafka/KafkaRecordIteratorTest.java | 1 + .../org/apache/hive/service/cli/operation/TestQueryShutdownHooks.java| 1 + 2 files changed, 2 insertions(+)
[hive] branch master updated (6720320 -> 5dfbe71)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 6720320 HIVE-23830: Remove shutdownRunner runner after query is committed (Mustaga Iman reviewed by Panagiotis Garefalakis and Peter Vary) add 5dfbe71 HIVE-22957: Support Partition Filtering In MSCK REPAIR TABLE Command (#1105) (Syed Shameerur Rahman reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hive/ql/parse/HiveParser.g | 11 +- .../hadoop/hive/ql/parse/IdentifiersParser.g | 17 +- .../hadoop/hive/ql/ddl/misc/msck/MsckAnalyzer.java | 24 +- .../hadoop/hive/ql/ddl/misc/msck/MsckDesc.java | 16 +- .../hive/ql/ddl/misc/msck/MsckOperation.java | 4 +- .../drop/AbstractDropPartitionAnalyzer.java| 116 +-- .../hive/ql/exec/SerializationUtilities.java | 2 +- .../hadoop/hive/ql/parse/BaseSemanticAnalyzer.java | 16 +- .../apache/hadoop/hive/ql/parse/ParseUtils.java| 133 .../hive/ql/metadata/TestHiveMetaStoreChecker.java | 12 +- .../queries/clientpositive/msck_repair_filter.q| 141 .../clientpositive/llap/msck_repair_filter.q.out | 378 + .../hive/metastore/HiveMetaStoreChecker.java | 105 +++--- .../org/apache/hadoop/hive/metastore/Msck.java | 5 +- .../org/apache/hadoop/hive/metastore/MsckInfo.java | 16 +- .../hive/metastore/utils/MetaStoreServerUtils.java | 11 + 16 files changed, 791 insertions(+), 216 deletions(-) create mode 100644 ql/src/test/queries/clientpositive/msck_repair_filter.q create mode 100644 ql/src/test/results/clientpositive/llap/msck_repair_filter.q.out
[hive] branch master updated: HIVE-22412: StatsUtils throw NPE when explain (#1209) (xiepengjie reviewed by David Mollitor and Zoltan Haindrich)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 5a54752 HIVE-22412: StatsUtils throw NPE when explain (#1209)
(xiepengjie reviewed by David Mollitor and Zoltan Haindrich)
5a54752 is described below
commit 5a54752f8f1964b266a25a47f3d05d894806dc78
Author: Jeff.r
AuthorDate: Mon Jul 13 16:46:09 2020 +0800
HIVE-22412: StatsUtils throw NPE when explain (#1209) (xiepengjie reviewed
by David Mollitor and Zoltan Haindrich)
---
.../apache/hadoop/hive/ql/stats/StatsUtils.java| 10 +-
ql/src/test/queries/clientpositive/analyze_npe.q | 26 +++
.../results/clientpositive/llap/analyze_npe.q.out | 215 +
.../StandardConstantMapObjectInspector.java| 4 +-
4 files changed, 252 insertions(+), 3 deletions(-)
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
b/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
index 9b160dd..962b131 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java
@@ -1182,7 +1182,12 @@ public class StatsUtils {
// constant list projection of known length
StandardConstantListObjectInspector scloi =
(StandardConstantListObjectInspector) oi;
-length = scloi.getWritableConstantValue().size();
+List value = scloi.getWritableConstantValue();
+if (null == value) {
+ length = 0;
+} else {
+ length = value.size();
+}
// check if list elements are primitive or Objects
ObjectInspector leoi = scloi.getListElementObjectInspector();
@@ -1336,6 +1341,9 @@ public class StatsUtils {
*/
public static long getSizeOfMap(StandardConstantMapObjectInspector scmoi) {
Map map = scmoi.getWritableConstantValue();
+if (null == map) {
+ return 0L;
+}
ObjectInspector koi = scmoi.getMapKeyObjectInspector();
ObjectInspector voi = scmoi.getMapValueObjectInspector();
long result = 0;
diff --git a/ql/src/test/queries/clientpositive/analyze_npe.q
b/ql/src/test/queries/clientpositive/analyze_npe.q
new file mode 100644
index 000..e7cfea0
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/analyze_npe.q
@@ -0,0 +1,26 @@
+drop table if exists explain_npe_map;
+drop table if exists explain_npe_array;
+drop table if exists explain_npe_struct;
+
+create table explain_npe_map( c1 map );
+create table explain_npe_array ( c1 array );
+create table explain_npe_struct ( c1 struct );
+
+-- error
+set hive.cbo.enable=false;
+explain select c1 from explain_npe_map where c1 is null;
+explain select c1 from explain_npe_array where c1 is null;
+-- maybe correct in branch of master,
+-- but i think it is necessary to initialize the value of
StandardConstantStructObjectInspector
+explain select c1 from explain_npe_struct where c1 is null;
+
+-- correct
+set hive.cbo.enable=true;
+explain select c1 from explain_npe_map where c1 is null;
+explain select c1 from explain_npe_array where c1 is null;
+explain select c1 from explain_npe_struct where c1 is null;
+
+-- drop test table
+drop table if exists explain_npe_map;
+drop table if exists explain_npe_array;
+drop table if exists explain_npe_struct;
\ No newline at end of file
diff --git a/ql/src/test/results/clientpositive/llap/analyze_npe.q.out
b/ql/src/test/results/clientpositive/llap/analyze_npe.q.out
new file mode 100644
index 000..f532295
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/analyze_npe.q.out
@@ -0,0 +1,215 @@
+PREHOOK: query: drop table if exists explain_npe_map
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: drop table if exists explain_npe_map
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: drop table if exists explain_npe_array
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: drop table if exists explain_npe_array
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: drop table if exists explain_npe_struct
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: drop table if exists explain_npe_struct
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: create table explain_npe_map( c1 map )
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@explain_npe_map
+POSTHOOK: query: create table explain_npe_map( c1 map )
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@explain_npe_map
+PREHOOK: query: create table explain_npe_array ( c1 array )
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@explain_npe_array
+POSTHOOK: query: create table explain_npe_array ( c1 array )
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@explain_npe_array
+PREHOOK: query: create table explain_npe_struct ( c1 struct )
+PREHOOK: type: CREATETABLE
+PREHOOK
[hive] branch master updated (f7ec294 -> 65cf695)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from f7ec294 HIVE-23760: Upgrading to Kafka 2.5 Clients (Karen Coppage, reviewed by Peter Vary) add 65cf695 HIVE-23638: Spotbugs issues in hive-common (#1161) (Panagiotis Garefalakis reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: Jenkinsfile| 3 +- .../hadoop/hive/common/CompressionUtils.java | 102 +++-- .../hive/common/CopyOnFirstWriteProperties.java| 7 +- .../org/apache/hadoop/hive/common/FileUtils.java | 19 ++-- .../hadoop/hive/common/StringInternUtils.java | 48 ++ .../hadoop/hive/common/cli/CommonCliOptions.java | 2 + .../hadoop/hive/common/io/DigestPrintStream.java | 3 +- .../common/io/NonSyncByteArrayInputStream.java | 3 + .../common/io/NonSyncByteArrayOutputStream.java| 3 + .../hive/common/io/SortAndDigestPrintStream.java | 3 +- .../hadoop/hive/common/jsonexplain/Connection.java | 16 .../common/jsonexplain/DagJsonParserUtils.java | 8 +- .../apache/hadoop/hive/common/jsonexplain/Op.java | 8 +- .../hadoop/hive/common/jsonexplain/Vertex.java | 17 +++- .../hadoop/hive/common/log/InPlaceUpdate.java | 3 +- .../hadoop/hive/common/log/LogRedirector.java | 5 +- .../hive/common/metrics/MetricsMBeanImpl.java | 7 +- .../metrics/metrics2/JsonFileMetricsReporter.java | 3 +- .../org/apache/hadoop/hive/common/type/Date.java | 4 + .../hadoop/hive/common/type/HiveVarchar.java | 5 +- .../hadoop/hive/common/type/SqlMathUtil.java | 23 +++-- .../apache/hadoop/hive/common/type/Timestamp.java | 4 + .../hadoop/hive/common/type/UnsignedInt128.java| 8 +- .../java/org/apache/hadoop/hive/conf/HiveConf.java | 36 .../hive/conf/LoopingByteArrayInputStream.java | 3 + .../apache/hadoop/hive/conf/SystemVariables.java | 2 +- .../org/apache/hadoop/hive/conf/Validator.java | 22 +++-- .../org/apache/hadoop/hive/ql/log/PerfLogger.java | 4 + .../hive/common/util/FixedSizedObjectPool.java | 2 +- .../org/apache/hive/common/util/HiveTestUtils.java | 7 +- .../src/java/org/apache/hive/common/util/Ref.java | 1 + .../hive/common/util/ShutdownHookManager.java | 1 + .../org/apache/hive/common/util/StreamPrinter.java | 3 +- .../hive/common/util/SuppressFBWarnings.java | 20 ++-- .../java/org/apache/hive/http/ProfileServlet.java | 2 +- .../apache/hive/http/security/PamConstraint.java | 2 + pom.xml| 6 -- ql/pom.xml | 6 -- .../parse/repl/dump/log/state/AtlasDumpBegin.java | 2 +- .../ql/parse/repl/dump/log/state/AtlasDumpEnd.java | 2 +- .../parse/repl/dump/log/state/RangerDumpBegin.java | 2 +- .../parse/repl/dump/log/state/RangerDumpEnd.java | 2 +- .../parse/repl/load/log/state/AtlasLoadBegin.java | 2 +- .../ql/parse/repl/load/log/state/AtlasLoadEnd.java | 2 +- .../parse/repl/load/log/state/RangerLoadBegin.java | 2 +- .../parse/repl/load/log/state/RangerLoadEnd.java | 2 +- .../ql/parse/repl/metric/event/StageMapper.java| 2 +- .../apache/hadoop/hive/shims/Hadoop23Shims.java| 2 +- .../org/apache/hadoop/util/SuppressFBWarnings.java | 20 ++-- .../hadoop/hive/metastore/conf/MetastoreConf.java | 2 +- standalone-metastore/pom.xml | 9 -- .../hive/common/util/SuppressFBWarnings.java | 18 52 files changed, 300 insertions(+), 190 deletions(-) copy ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedUDAFs.java => common/src/java/org/apache/hive/common/util/SuppressFBWarnings.java (67%) copy ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedExpressions.java => shims/0.23/src/main/java/org/apache/hadoop/util/SuppressFBWarnings.java (67%)
[hive] branch master updated: HIVE-23827: Upgrade to DataSketches 1.1.0 (#1233) (Zoltan Haindrich via Denys Kuzmenko)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hive.git The following commit(s) were added to refs/heads/master by this push: new ca892e0 HIVE-23827: Upgrade to DataSketches 1.1.0 (#1233) (Zoltan Haindrich via Denys Kuzmenko) ca892e0 is described below commit ca892e0e0841b7eaeb03797e2a920af68a15f1dc Author: Zoltan Haindrich AuthorDate: Fri Jul 10 10:22:20 2020 +0200 HIVE-23827: Upgrade to DataSketches 1.1.0 (#1233) (Zoltan Haindrich via Denys Kuzmenko) --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 0513fe7..d709b94 100644 --- a/pom.xml +++ b/pom.xml @@ -215,7 +215,7 @@ 2.0.1 2.4.0 3.0.11 -1.0.0-incubating +1.1.0-incubating 4.0.3
[hive] branch master updated (f98e136 -> 784ec0c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from f98e136 HIVE-23525: TestAcidTxnCleanerService is unstable (Peter Varga reviewed by Peter Vary) add 784ec0c HIVE-23509 Fixing MapJoin Capacity Assertion Error (#1026) (Shashank Pedamallu reviewed by Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java| 4 ++-- .../hive/ql/exec/persistence/TestBytesBytesMultiHashMap.java | 7 +++ 2 files changed, 9 insertions(+), 2 deletions(-)
[hive] branch master updated (b5c4ba9 -> 8e6930e)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from b5c4ba9 HIVE-23813: Fix MetricsMaintTask run frequency (Aasha Medhi via Zoltan Haindrich) add 8e6930e HIVE-22952: Use LinkedHashMap in TestStandardObjectInspectors (cpugputpu reviewed by David Mollitor) No new revisions were added by this update. Summary of changes: .../hive/serde2/objectinspector/TestStandardObjectInspectors.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
[hive] branch master updated (8e6930e -> 6f9c7bc)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 8e6930e HIVE-22952: Use LinkedHashMap in TestStandardObjectInspectors (cpugputpu reviewed by David Mollitor) add 6f9c7bc HIVE-21825: Improve client error msg when Active/Passive HA is enabled (#682) (Richard Zhang reviewed by Prashanth J and David Mollitor) No new revisions were added by this update. Summary of changes:
[hive] branch master updated (b5c4ba9 -> 8e6930e)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from b5c4ba9 HIVE-23813: Fix MetricsMaintTask run frequency (Aasha Medhi via Zoltan Haindrich) add 8e6930e HIVE-22952: Use LinkedHashMap in TestStandardObjectInspectors (cpugputpu reviewed by David Mollitor) No new revisions were added by this update. Summary of changes: .../hive/serde2/objectinspector/TestStandardObjectInspectors.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
[hive] branch master updated (aa407da -> b5c4ba9)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from aa407da HIVE-23363: Upgrade DataNucleus dependency to 5.2 (David Mollitor via Ashutosh Chauhan) Signed-off-by: Ashutosh Chauhan add b5c4ba9 HIVE-23813: Fix MetricsMaintTask run frequency (Aasha Medhi via Zoltan Haindrich) No new revisions were added by this update. Summary of changes: .../hive/hcatalog/hbase/TestPigHBaseStorageHandler.java | 5 - .../apache/hadoop/hive/ql/parse/repl/metric/MetricSink.java | 3 ++- .../hadoop/hive/metastore/ReplicationMetricsMaintTask.java | 12 +--- 3 files changed, 7 insertions(+), 13 deletions(-)
[hive] branch master updated (661c792 -> ac254b8)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 661c792 HIVE-23790: The error message length of 2000 is exceeded for scheduled query (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez) add ac254b8 HIVE-22301: Hive lineage is not generated for insert overwrite queries on partitioned tables (Zoltan Haindrich reviewed by Denys Kuzmenko and Jesus Camacho Rodriguez) No new revisions were added by this update. Summary of changes: .../src/test/results/clientpositive/buckets.q.out | 2 ++ .../insert_into_dynamic_partitions.q.out | 4 +++ .../insert_overwrite_dynamic_partitions.q.out | 5 +++ ...t_overwrite_dynamic_partitions_merge_move.q.out | 2 ++ ...t_overwrite_dynamic_partitions_merge_only.q.out | 2 ++ ...rt_overwrite_dynamic_partitions_move_only.q.out | 2 ++ .../test/results/clientpositive/orc_buckets.q.out | 2 ++ .../results/clientpositive/orc_format_part.q.out | 1 + .../clientpositive/orc_nonstd_partitions_loc.q.out | 3 ++ .../results/clientpositive/parquet_buckets.q.out | 2 ++ .../clientpositive/parquet_format_part.q.out | 1 + .../parquet_nonstd_partitions_loc.q.out| 3 ++ .../results/clientpositive/rcfile_buckets.q.out| 2 ++ .../clientpositive/rcfile_format_part.q.out| 1 + .../rcfile_nonstd_partitions_loc.q.out | 3 ++ .../hadoop/hive/ql/parse/SemanticAnalyzer.java | 2 +- ql/src/test/queries/clientpositive/lineage4.q | 3 ++ ql/src/test/results/clientnegative/mm_delete.q.out | 1 + ql/src/test/results/clientnegative/mm_update.q.out | 1 + .../encrypted/encryption_drop_partition.q.out | 1 + .../encryption_insert_partition_dynamic.q.out | 3 ++ .../llap/acid_direct_insert_insert_overwrite.q.out | 10 ++ .../llap/acid_insert_overwrite.q.out | 2 ++ .../llap/acid_insert_overwrite_update.q.out| 1 + .../llap/acid_multiinsert_dyn_part.q.out | 6 .../clientpositive/llap/acid_no_buckets.q.out | 10 ++ .../results/clientpositive/llap/acid_stats2.q.out | 1 + .../clientpositive/llap/acid_subquery.q.out| 2 ++ .../llap/acid_table_directories_test.q.out | 2 ++ .../clientpositive/llap/acid_table_stats.q.out | 3 ++ .../clientpositive/llap/alterColumnStatsPart.q.out | 2 ++ .../llap/alter_partition_change_col.q.out | 1 + .../llap/alter_partition_coltype.q.out | 3 ++ .../clientpositive/llap/alter_table_cascade.q.out | 2 ++ .../llap/alter_table_partition_drop.q.out | 1 + .../llap/analyze_table_null_partition.q.out| 1 + .../clientpositive/llap/annotate_stats_part.q.out | 1 + .../clientpositive/llap/autoColumnStats_1.q.out| 3 ++ .../clientpositive/llap/autoColumnStats_2.q.out| 3 ++ .../clientpositive/llap/autoColumnStats_3.q.out| 2 ++ .../clientpositive/llap/autoColumnStats_8.q.out| 2 ++ .../llap/auto_sortmerge_join_16.q.out | 4 +++ .../llap/avro_extschema_insert.q.out | 1 + .../clientpositive/llap/avro_partitioned.q.out | 2 ++ .../llap/avro_partitioned_native.q.out | 1 + .../llap/avro_schema_evolution_native.q.out| 1 + .../results/clientpositive/llap/combine2.q.out | 1 + .../results/clientpositive/llap/constprog_dp.q.out | 2 ++ .../test/results/clientpositive/llap/cp_sel.q.out | 2 ++ .../llap/create_transactional_full_acid.q.out | 3 ++ .../llap/create_transactional_insert_only.q.out| 1 + ql/src/test/results/clientpositive/llap/ctas.q.out | 1 + .../clientpositive/llap/dbtxnmgr_query4.q.out | 1 + .../llap/drop_partitions_filter4.q.out | 8 + .../llap/dynamic_partition_insert.q.out| 12 +++ .../llap/dynamic_partition_pruning.q.out | 1 + .../llap/dynamic_partition_skip_default.q.out | 4 +++ .../llap/dynpart_sort_opt_bucketing.q.out | 8 + .../llap/dynpart_sort_opt_vectorization.q.out | 14 .../llap/dynpart_sort_optimization.q.out | 37 ++ .../llap/dynpart_sort_optimization2.q.out | 16 ++ .../llap/dynpart_sort_optimization_acid.q.out | 5 +++ .../llap/dynpart_sort_optimization_acid2.q.out | 1 + .../llap/enforce_constraint_notnull.q.out | 3 ++ .../clientpositive/llap/explain_locks.q.out| 3 ++ .../llap/extrapolate_part_stats_full.q.out | 2 ++ .../llap/extrapolate_part_stats_partial.q.out | 2 ++ .../llap/extrapolate_part_stats_partial_ndv.q.out | 2 ++ .../clientpositive/llap/filter_numeric.q.out | 1 + .../llap/implicit_cast_during_insert.q.out | 2 ++ .../llap/infer_bucket_sort_dyn_part.q.out | 3 ++ .../test/results/clientpositive/llap/insert0.q.out | 1 + .../llap/insert_acid_dynamic_partition.q.out | 1
[hive] branch master updated (661c792 -> ac254b8)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from 661c792 HIVE-23790: The error message length of 2000 is exceeded for scheduled query (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez) add ac254b8 HIVE-22301: Hive lineage is not generated for insert overwrite queries on partitioned tables (Zoltan Haindrich reviewed by Denys Kuzmenko and Jesus Camacho Rodriguez) No new revisions were added by this update. Summary of changes: .../src/test/results/clientpositive/buckets.q.out | 2 ++ .../insert_into_dynamic_partitions.q.out | 4 +++ .../insert_overwrite_dynamic_partitions.q.out | 5 +++ ...t_overwrite_dynamic_partitions_merge_move.q.out | 2 ++ ...t_overwrite_dynamic_partitions_merge_only.q.out | 2 ++ ...rt_overwrite_dynamic_partitions_move_only.q.out | 2 ++ .../test/results/clientpositive/orc_buckets.q.out | 2 ++ .../results/clientpositive/orc_format_part.q.out | 1 + .../clientpositive/orc_nonstd_partitions_loc.q.out | 3 ++ .../results/clientpositive/parquet_buckets.q.out | 2 ++ .../clientpositive/parquet_format_part.q.out | 1 + .../parquet_nonstd_partitions_loc.q.out| 3 ++ .../results/clientpositive/rcfile_buckets.q.out| 2 ++ .../clientpositive/rcfile_format_part.q.out| 1 + .../rcfile_nonstd_partitions_loc.q.out | 3 ++ .../hadoop/hive/ql/parse/SemanticAnalyzer.java | 2 +- ql/src/test/queries/clientpositive/lineage4.q | 3 ++ ql/src/test/results/clientnegative/mm_delete.q.out | 1 + ql/src/test/results/clientnegative/mm_update.q.out | 1 + .../encrypted/encryption_drop_partition.q.out | 1 + .../encryption_insert_partition_dynamic.q.out | 3 ++ .../llap/acid_direct_insert_insert_overwrite.q.out | 10 ++ .../llap/acid_insert_overwrite.q.out | 2 ++ .../llap/acid_insert_overwrite_update.q.out| 1 + .../llap/acid_multiinsert_dyn_part.q.out | 6 .../clientpositive/llap/acid_no_buckets.q.out | 10 ++ .../results/clientpositive/llap/acid_stats2.q.out | 1 + .../clientpositive/llap/acid_subquery.q.out| 2 ++ .../llap/acid_table_directories_test.q.out | 2 ++ .../clientpositive/llap/acid_table_stats.q.out | 3 ++ .../clientpositive/llap/alterColumnStatsPart.q.out | 2 ++ .../llap/alter_partition_change_col.q.out | 1 + .../llap/alter_partition_coltype.q.out | 3 ++ .../clientpositive/llap/alter_table_cascade.q.out | 2 ++ .../llap/alter_table_partition_drop.q.out | 1 + .../llap/analyze_table_null_partition.q.out| 1 + .../clientpositive/llap/annotate_stats_part.q.out | 1 + .../clientpositive/llap/autoColumnStats_1.q.out| 3 ++ .../clientpositive/llap/autoColumnStats_2.q.out| 3 ++ .../clientpositive/llap/autoColumnStats_3.q.out| 2 ++ .../clientpositive/llap/autoColumnStats_8.q.out| 2 ++ .../llap/auto_sortmerge_join_16.q.out | 4 +++ .../llap/avro_extschema_insert.q.out | 1 + .../clientpositive/llap/avro_partitioned.q.out | 2 ++ .../llap/avro_partitioned_native.q.out | 1 + .../llap/avro_schema_evolution_native.q.out| 1 + .../results/clientpositive/llap/combine2.q.out | 1 + .../results/clientpositive/llap/constprog_dp.q.out | 2 ++ .../test/results/clientpositive/llap/cp_sel.q.out | 2 ++ .../llap/create_transactional_full_acid.q.out | 3 ++ .../llap/create_transactional_insert_only.q.out| 1 + ql/src/test/results/clientpositive/llap/ctas.q.out | 1 + .../clientpositive/llap/dbtxnmgr_query4.q.out | 1 + .../llap/drop_partitions_filter4.q.out | 8 + .../llap/dynamic_partition_insert.q.out| 12 +++ .../llap/dynamic_partition_pruning.q.out | 1 + .../llap/dynamic_partition_skip_default.q.out | 4 +++ .../llap/dynpart_sort_opt_bucketing.q.out | 8 + .../llap/dynpart_sort_opt_vectorization.q.out | 14 .../llap/dynpart_sort_optimization.q.out | 37 ++ .../llap/dynpart_sort_optimization2.q.out | 16 ++ .../llap/dynpart_sort_optimization_acid.q.out | 5 +++ .../llap/dynpart_sort_optimization_acid2.q.out | 1 + .../llap/enforce_constraint_notnull.q.out | 3 ++ .../clientpositive/llap/explain_locks.q.out| 3 ++ .../llap/extrapolate_part_stats_full.q.out | 2 ++ .../llap/extrapolate_part_stats_partial.q.out | 2 ++ .../llap/extrapolate_part_stats_partial_ndv.q.out | 2 ++ .../clientpositive/llap/filter_numeric.q.out | 1 + .../llap/implicit_cast_during_insert.q.out | 2 ++ .../llap/infer_bucket_sort_dyn_part.q.out | 3 ++ .../test/results/clientpositive/llap/insert0.q.out | 1 + .../llap/insert_acid_dynamic_partition.q.out | 1
[hive] branch master updated: HIVE-23790: The error message length of 2000 is exceeded for scheduled query (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 661c792 HIVE-23790: The error message length of 2000 is exceeded for
scheduled query (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
661c792 is described below
commit 661c792c77196070656229b1571dc23a17c85ec2
Author: Zoltan Haindrich
AuthorDate: Wed Jul 8 15:55:49 2020 +0200
HIVE-23790: The error message length of 2000 is exceeded for scheduled
query (Zoltan Haindrich reviewed by Jesus Camacho Rodriguez)
---
.../org/apache/hadoop/hive/metastore/ObjectStore.java | 15 ++-
.../metastore/client/TestMetastoreScheduledQueries.java | 14 ++
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
index c472e93..57a5f02 100644
---
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
+++
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java
@@ -13061,7 +13061,7 @@ public class ObjectStore implements RawStore,
Configurable {
execution.setExecutorQueryId(info.getExecutorQueryId());
}
if (info.isSetErrorMessage()) {
-execution.setErrorMessage(info.getErrorMessage());
+
execution.setErrorMessage(abbreviateErrorMessage(info.getErrorMessage(), 1000));
}
switch (info.getState()) {
@@ -13088,6 +13088,19 @@ public class ObjectStore implements RawStore,
Configurable {
}
}
+ /**
+ * Abbreviates the error message to the given size.
+ *
+ * There might be error messages which may also contain a stack trace.
+ */
+ private String abbreviateErrorMessage(String errorMessage, int maxLength) {
+if (errorMessage.length() < maxLength) {
+ return errorMessage;
+}
+String[] parts = errorMessage.split("\n", 2);
+return StringUtils.abbreviate(parts[0], maxLength);
+ }
+
@Override
public void addReplicationMetrics(ReplicationMetricList
replicationMetricList) {
boolean commited = false;
diff --git
a/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestMetastoreScheduledQueries.java
b/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestMetastoreScheduledQueries.java
index d84cc04..0852e98 100644
---
a/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestMetastoreScheduledQueries.java
+++
b/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestMetastoreScheduledQueries.java
@@ -339,6 +339,7 @@ public class TestMetastoreScheduledQueries extends
MetaStoreClientTest {
info = new ScheduledQueryProgressInfo(
pollResult.getExecutionId(), QueryState.FAILED, "executor-query-id");
+info.setErrorMessage(generateLongErrorMessage());
//info.set
client.scheduledQueryProgress(info);
@@ -349,6 +350,8 @@ public class TestMetastoreScheduledQueries extends
MetaStoreClientTest {
assertNull(q.getLastUpdateTime());
assertTrue(q.getEndTime() <= getEpochSeconds());
assertTrue(q.getEndTime() >= getEpochSeconds() - 1);
+ assertTrue(q.getErrorMessage().length() < 2000);
+ assertFalse(q.getErrorMessage().contains("x"));
}
// clustername is taken into account; this should be empty
@@ -357,6 +360,17 @@ public class TestMetastoreScheduledQueries extends
MetaStoreClientTest {
assertFalse(pollResult.isSetQuery());
}
+ private String generateLongErrorMessage() {
+StringBuilder sb = new StringBuilder();
+for (int i = 0; i < 2500; i++) {
+ sb.append("e");
+}
+for (int j = 0; j < 10; j++) {
+ sb.append("\nx");
+}
+return sb.toString();
+ }
+
@Test
public void testCleanup() throws Exception {
String namespace = "cleanup";
[hive] branch master updated (d1839f8 -> 4942a7c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from d1839f8 HIVE-23751: QTest: Override #mkdirs() method in ProxyFileSystem To Align After HADOOP-16582 (#1167) add 4942a7c HIVE-23722: Show the operation's drilldown link to client (#1145) No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hive/conf/HiveConf.java | 2 + .../TestThriftCliServiceWithInfoMessage.java | 139 + .../java/org/apache/hive/jdbc/HiveStatement.java | 6 + .../service/cli/operation/OperationManager.java| 23 +++- .../hive/service/cli/thrift/ThriftCLIService.java | 17 ++- 5 files changed, 182 insertions(+), 5 deletions(-) create mode 100644 itests/hive-unit/src/test/java/org/apache/hive/service/cli/thrift/TestThriftCliServiceWithInfoMessage.java
[hive] branch master updated (c1f811e -> d1839f8)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from c1f811e HIVE-23789: Merge ValidTxnManager into DriverTxnHandler (Miklos Gergely, reviewed by Peter Vary) add d1839f8 HIVE-23751: QTest: Override #mkdirs() method in ProxyFileSystem To Align After HADOOP-16582 (#1167) No new revisions were added by this update. Summary of changes: shims/common/src/main/java/org/apache/hadoop/fs/ProxyFileSystem.java | 5 + 1 file changed, 5 insertions(+)
[hive] branch master updated: HIVE-23593: Schemainit fails with NoSuchFieldError (#1137)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new fd52677 HIVE-23593: Schemainit fails with NoSuchFieldError (#1137)
fd52677 is described below
commit fd526774d9e35a67ef2c0f23d6e41d3f9877c282
Author: Zoltan Haindrich
AuthorDate: Fri Jun 19 12:21:16 2020 +0200
HIVE-23593: Schemainit fails with NoSuchFieldError (#1137)
---
Jenkinsfile| 49 +--
itests/hive-jmh/pom.xml| 97 --
packaging/src/main/assembly/bin.xml| 1 +
.../metastore-tools/metastore-benchmarks/pom.xml | 2 +-
4 files changed, 96 insertions(+), 53 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index ad4c95b..304d001 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -67,10 +67,12 @@ setPrLabel("PENDING");
def executorNode(run) {
hdbPodTemplate {
+timeout(time: 6, unit: 'HOURS') {
node(POD_LABEL) {
container('hdb') {
run()
}
+ }
}
}
}
@@ -85,11 +87,13 @@ set -x
export USER="`whoami`"
export MAVEN_OPTS="-Xmx2g"
export -n HIVE_CONF_DIR
-OPTS=" -s $SETTINGS -B -Dmaven.test.failure.ignore -Dtest.groups= "
-OPTS+=" -Pitests,qsplits"
+cp $SETTINGS .git/settings.xml
+OPTS=" -s $PWD/.git/settings.xml -B -Dtest.groups= "
+OPTS+=" -Pitests,qsplits,dist"
OPTS+="
-Dorg.slf4j.simpleLogger.log.org.apache.maven.plugin.surefire.SurefirePlugin=INFO"
OPTS+=" -Dmaven.repo.local=$PWD/.git/m2"
-OPTS+=" $M_OPTS "
+git config extra.mavenOpts "$OPTS"
+OPTS=" $M_OPTS -Dmaven.test.failure.ignore "
if [ -s inclusions.txt ]; then OPTS+="
-Dsurefire.includesFile=$PWD/inclusions.txt";fi
if [ -s exclusions.txt ]; then OPTS+="
-Dsurefire.excludesFile=$PWD/exclusions.txt";fi
mvn $OPTS '''+args+'''
@@ -103,13 +107,23 @@ df -h
def hdbPodTemplate(closure) {
podTemplate(
containers: [
-containerTemplate(name: 'hdb', image: 'kgyrtkirk/hive-dev-box:executor',
ttyEnabled: true, command: 'cat',
+containerTemplate(name: 'hdb', image: 'kgyrtkirk/hive-dev-box:executor',
ttyEnabled: true, command: 'tini -- cat',
alwaysPullImage: true,
resourceRequestCpu: '1800m',
resourceLimitCpu: '8000m',
resourceRequestMemory: '6400Mi',
-resourceLimitMemory: '12000Mi'
+resourceLimitMemory: '12000Mi',
+envVars: [
+envVar(key: 'DOCKER_HOST', value: 'tcp://localhost:2375')
+]
),
+containerTemplate(name: 'dind', image: 'docker:18.05-dind',
+alwaysPullImage: true,
+privileged: true,
+),
+ ],
+ volumes: [
+emptyDirVolume(mountPath: '/var/lib/docker', memory: false),
], yaml:'''
spec:
securityContext:
@@ -192,8 +206,31 @@ jobWrappers {
}
stage('Testing') {
-
def branches = [:]
+for (def d in ['derby','postgres']) {
+ def dbType=d
+ def splitName = "init@$dbType"
+ branches[splitName] = {
+executorNode {
+ stage('Prepare') {
+ loadWS();
+ }
+ stage('init-metastore') {
+ withEnv(["dbType=$dbType"]) {
+ sh '''#!/bin/bash -e
+set -x
+echo 127.0.0.1 dev_$dbType | sudo tee -a /etc/hosts
+. /etc/profile.d/confs.sh
+sw hive-dev $PWD
+ping -c2 dev_$dbType
+export DOCKER_NETWORK=host
+reinit_metastore $dbType
+'''
+}
+ }
+}
+ }
+}
for (int i = 0; i < splits.size(); i++) {
def num = i
def split = splits[num]
diff --git a/itests/hive-jmh/pom.xml b/itests/hive-jmh/pom.xml
index 097a6f9..b289da7 100644
--- a/itests/hive-jmh/pom.xml
+++ b/itests/hive-jmh/pom.xml
@@ -82,51 +82,56 @@
-
-
-
-org.apache.maven.plugins
-maven-shade-plugin
-
-
-package
-
- shade
-
-
- ${uberjar.name}
-
-
-
- org.openjdk.jmh.Main
-
-
-
-
-
- *:*
-
-META-INF/*.SF
-META-INF/*.DSA
-META-INF/*.RSA
-
-
-
-
-
-
-
-
-com.github.edwgiz
-
maven-shade-plugin.log4j2-cachefile-transformer
-2.1
-
-
-
-
-
+
+
+ perf
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
[hive] branch revert-1108-HIVE-23689-bump-tez0.9.2 created (now b09ab8c)
This is an automated email from the ASF dual-hosted git repository. kgyrtkirk pushed a change to branch revert-1108-HIVE-23689-bump-tez0.9.2 in repository https://gitbox.apache.org/repos/asf/hive.git. at b09ab8c Revert "HIVE-23689: Bump Tez version to 0.9.2 (#1108)" No new revisions were added by this update.
