This is an automated email from the ASF dual-hosted git repository.
dengzh pushed a change to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/hive.git
from 522b0bb7c18 HIVE-28368: Iceberg: Unable to read PARTITIONS Metadata
table. (#5339). (Ayush Saxena, reviewed by Sourabh Badhya)
add 587aed21612 HIVE-28315: Missing classes while using hive jdbc
standalone jar (#5313) (Zhihua Deng, reviewed by Denys Kuzmenko)
add 1a893bbe09d HIVE-28327: Missing null-check in TruncDateFromTimestamp
(Seonggon Namgung, reviewed by Denys Kuzmenko, Shohei Okumiya)
add 62e3dd2e09a HIVE-28347: Make a UDAF 'collect_set' work with complex
types, even when map-side aggregation is disabled (Jeongdae Kim, reviewed by
Shohei Okumiya, Denys Kuzmenko)
add bf2df8d31e5 HIVE-28350: Drop remote database should succeed deleting
data under db location (#5322) (Sai Hemanth Gantasala, reviewed by Butao Zhang,
Zhihua Deng)
add e87502a22d8 HIVE-27357: Map-side SMB Join returns incorrect result
when tables have different bucket size (Seonggon Namgung, reviewed by Attila
Turoczy, Denys Kuzmenko)
add c9864fc2dfe HIVE-28042: DigestMD5 token expired or does not exist
error while opening a new connection to HMS (Vikram Ahuja, Reviewed by Butao
Zhang, Chinna Rao Lalam)
add db9eee7295d HIVE-28264: OOM/slow compilation when query contains
SELECT clauses with nested expressions (Krisztian Kasa, reviewed by Stamatis
Zampetakis)
add d7e264de7cc HIVE-28310: Disable
hive.optimize.join.disjunctive.transitive.predicates.pushdown by default
(Stamatis Zampetakis reviewed by Alessandro Solimando, Krisztian Kasa)
add afa8791d441 HIVE-28326: Fix StageIDsRearranger (Seonggon Namgung,
reviewed by Denys Kuzmenko, Shohei Okumiya)
add af2734b5955 HIVE-28428: Performance regression in map-hash aggregation
(Ryu Kobayashi, reviewed by Denys Kuzmenko)
add 4ac3f4de6df HIVE-28431 Fix RexLiteral to ExprNode conversion if the
literal is an empty string (Ramesh Kumar reviewed by Soumyakanti Das)
add 6f46ddf40ad HIVE-28484: SharedWorkOptimizer leaves residual unused
operator tree that send DPP events to unknown operators (Ramesh Kumar, reviewed
by Seonggon Namgung, Denys Kuzmenko)
add 202a7c308ad HIVE-28487: Fix outdated MetastoreSchemaTool class in
schemaTool.sh (#5419). (Sebastian Bernauer, Reviewed by Ayush Saxena,
Indhumathi Muthumurugesh)
add 57623992df0 HIVE-28399: Improve the fetch size in HiveConnection
(#5362) (Zhihua Deng, reviewed by Butao Zhang)
add 856e736b96e HIVE-28515: Iceberg: Concurrent queries fail during commit
with ValidationException. (#5438). (Ayush Saxena, reviewed by Simhadri
Govindappa)
add 8cf5fdcfb19 HIVE-28480: Disable SMB on partition hash generator
mismatch across join branches in previous RS (Himanshu Mishra, reviewed by
Krisztian Kasa, Shohei Okumiya)
add 12abf2cda90 HIVE-28482: Iceberg: CTAS, CTLT query failure while
fetching URI for authorization (#5414) (Sourabh Badhya reviewed by Denys
Kuzmenko)
add cd5391743d3 HIVE-28434: Upgrade to tez 0.10.4 (#5382) (Laszlo Bodor
reviewed by Ayush Saxena)
add e92a9829394 HIVE-28349: SHOW TABLES with invalid connector, giving 0
results, instead of failing (#5321)(Sai Hemanth Gantasala, reviewed by Butao
Zhang)
add a914a364be3 HIVE-28249: Parquet legacy timezone conversion converts
march 1st to 29th feb and fails with not a leap year exception .(#5241)
(Simhadri Govindappa , reviewed by Denys Kuzmenko and Stamatis Zampetakis )
add 69c89956be3 HIVE-28207: NullPointerException is thrown when checking
column uniqueness (Shohei Okumiya reviewed by Stamatis Zampetakis, Ayush Saxena)
add d4328254820 HIVE-28167: Full table deletion fails when converting to
truncate for Iceberg and ACID tables (#5175) (Sourabh Badhya reviewed by Denys
Kuzmenko)
add 11a2e3e43ca HIVE-23964: SemanticException in query 30 while generating
logical plan (Krisztian Kasa, reviewed by Stamatis Zampetakis)
add 30ea5ced90b HIVE-28360: Upgrade jersey to 1.19.4 (#5333)
add b1c7eb38cd1 HIVE-28436: Fix Incorrect syntax in Hive schema file for
table MIN_HISTORY_LEVEL (#5388). (Indhumathi Muthumurugesh, reviewed by Ayush
Saxena, Shohei Okumiya, Kokila N)
add 538286c1634 HIVE-28456: ObjectStore
updatePartitionColumnStatisticsInBatch can cause connection starvation (#5398)
(Zhihua Deng, reviewed by Denys Kuzmenko)
add 16af516da53 HIVE-28087: Iceberg: Timestamp partition columns with
transforms are not correctly sorted during insert (#5134) (Sourabh Badhya
reviewed by Ayush Saxena, Simhadri Govindappa)
add 9928754f917 HIVE-28266: Iceberg: Select count(*) from data_files
metadata tables gives wrong result (Dmitriy Fingerman, reviewed by Butao Zhang,
Denys Kuzmenko)
No new revisions were added by this update.
Summary of changes:
common/pom.xml | 8 +
.../apache/hadoop/hive/common/type/Timestamp.java | 7 +-
.../hadoop/hive/common/type/TimestampTZUtil.java | 22 +-
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 8 +-
data/files/legacyLeapYearInSingaporeTZ | Bin 0 -> 365 bytes
data/files/query-hive-28087.csv | 47 +
.../iceberg/mr/hive/HiveIcebergStorageHandler.java | 6 +-
.../apache/iceberg/mr/hive/IcebergAcidUtil.java | 1 +
.../apache/iceberg/mr/hive/IcebergTableUtil.java | 2 +-
.../iceberg/mr/hive/udf/GenericUDFIcebergDay.java | 4 +-
.../iceberg/mr/hive/udf/GenericUDFIcebergHour.java | 8 +-
.../mr/hive/udf/GenericUDFIcebergMonth.java | 4 +-
.../iceberg/mr/hive/udf/GenericUDFIcebergYear.java | 4 +-
.../iceberg/mr/hive/TestHiveIcebergCTAS.java | 21 +
.../iceberg/mr/hive/TestOptimisticRetry.java | 23 +
.../src/test/queries/positive/delete_all_iceberg.q | 40 +-
.../src/test/queries/positive/iceberg_clustered.q | 57 +
.../iceberg_major_compaction_query_metadata.q | 44 +
.../test/results/positive/delete_all_iceberg.q.out | 299 ++
.../results/positive/llap/iceberg_clustered.q.out | 215 +
.../iceberg_major_compaction_query_metadata.q.out | 142 +
itests/hive-blobstore/pom.xml | 10 +
itests/hive-jmh/pom.xml | 10 +
.../apache/hive/minikdc/TestAuthWithDigestMD5.java | 98 +
.../metastore/security/TestHadoopAuthBridge23.java | 15 +-
itests/hive-unit/pom.xml | 10 +
itests/qtest-accumulo/pom.xml | 10 +
itests/qtest-iceberg/pom.xml | 10 +
itests/qtest-kudu/pom.xml | 10 +
itests/qtest/pom.xml | 10 +
.../test/resources/testconfiguration.properties | 5 +-
itests/util/pom.xml | 10 +
jdbc/pom.xml | 104 +-
.../java/org/apache/hive/jdbc/HiveConnection.java | 30 +-
.../java/org/apache/hive/jdbc/HiveStatement.java | 14 +-
.../apache/hive/jdbc/saml/IJdbcBrowserClient.java | 2 -
.../hive/jdbc/TestHivePreparedStatement.java | 4 +-
.../org/apache/hive/jdbc/TestHiveStatement.java | 19 +-
llap-client/pom.xml | 8 +
llap-common/pom.xml | 8 +
llap-ext-client/pom.xml | 13 +-
llap-tez/pom.xml | 8 +
pom.xml | 14 +-
ql/pom.xml | 12 +
.../hadoop/hive/ql/exec/GroupByOperator.java | 12 +-
.../apache/hadoop/hive/ql/exec/OperatorUtils.java | 17 +
.../hive/ql/exec/tez/CustomPartitionVertex.java | 100 +-
.../vector/expressions/TruncDateFromTimestamp.java | 4 +-
.../ql/io/parquet/timestamp/NanoTimeUtils.java | 27 +-
.../hive/ql/optimizer/ConvertJoinMapJoin.java | 32 +-
.../DynamicPartitionPruningOptimization.java | 13 +-
.../hive/ql/optimizer/SemiJoinReductionMerge.java | 3 +-
.../hive/ql/optimizer/SharedWorkOptimizer.java | 17 +-
.../hadoop/hive/ql/optimizer/calcite/Bug.java | 5 +
.../hive/ql/optimizer/calcite/HiveRelOptUtil.java | 14 +
.../calcite/rules/HiveAggregateSplitRule.java | 4 +-
.../rules/HiveFilterProjectTSTransposeRule.java | 17 +-
.../rules/HiveFilterProjectTransposeRule.java | 63 +-
.../calcite/rules/HiveJoinConstraintsRule.java | 3 +-
.../optimizer/calcite/stats/HiveRelMdRowCount.java | 3 +-
.../calcite/translator/ExprNodeConverter.java | 3 +
.../translator/opconventer/HiveGBOpConvUtil.java | 9 +-
.../ql/optimizer/physical/StageIDsRearranger.java | 2 +-
.../hive/ql/parse/DeleteSemanticAnalyzer.java | 9 +-
.../hadoop/hive/ql/parse/SemanticAnalyzer.java | 25 +-
.../apache/hadoop/hive/ql/plan/GroupByDesc.java | 16 +-
.../ql/reexec/ReExecuteOnWriteConflictPlugin.java | 4 +-
.../generic/GenericUDAFMkCollectionEvaluator.java | 21 +-
.../TestParquetTimestampsHive2Compatibility.java | 120 +
.../clientpositive/auto_sortmerge_join_18.q | 39 +
.../queries/clientpositive/cbo_join_constraints.q | 11 +
.../cbo_row_count_non_simple_filter.q | 26 +
.../queries/clientpositive/filter_emptyliteral.q | 4 +
.../clientpositive/parquet_historical_timestamp.q | 14 +-
.../test/queries/clientpositive/perf/cbo_query30.q | 5 +-
ql/src/test/queries/clientpositive/perf/query30.q | 5 +-
.../clientpositive/sharedworkresidual_dpp.q | 29 +
.../smb_join_with_different_bucket_size.q | 23 +
.../clientpositive/subquery_nested_expressions.q | 10 +
.../queries/clientpositive/udaf_collect_set_2.q | 191 +
.../llap/auto_sortmerge_join_18.q.out | 325 ++
.../clientpositive/llap/cbo_join_constraints.q.out | 78 +
.../llap/cbo_row_count_non_simple_filter.q.out | 159 +
.../llap/correlationoptimizer8.q.out | 134 +-
.../llap/delete_all_non_partitioned.q.out | 6 +-
.../llap/delete_all_partitioned.q.out | 10 +-
.../clientpositive/llap/filter_emptyliteral.q.out | 34 +
.../clientpositive/llap/insert_update_delete.q.out | 6 +-
.../test/results/clientpositive/llap/join34.q.out | 60 +-
.../test/results/clientpositive/llap/join35.q.out | 66 +-
.../clientpositive/{ => llap}/manyViewJoin.q.out | 5122 ++++++++++----------
.../llap/materialized_view_rewrite_7.q.out | 10 +-
.../llap/materialized_view_rewrite_by_text_9.q.out | 2 +-
.../llap/parquet_historical_timestamp.q.out | 37 +-
.../llap/sharedworkresidual_dpp.q.out | 149 +
.../llap/smb_join_with_different_bucket_size.q.out | 134 +
.../llap/subquery_nested_expressions.q.out | 36 +
.../clientpositive/llap/udaf_collect_set_2.q.out | 598 ++-
.../{ => llap}/vector_udf_trunc.q.out | 990 ++--
.../perf/tpcds30tb/tez/cbo_query30.q.out | 44 +
.../perf/tpcds30tb/tez/query30.q.out | 421 ++
.../hive/serde2/io/TimestampLocalTZWritable.java | 16 +
.../hadoop/hive/serde2/io/TimestampWritableV2.java | 16 +
service/pom.xml | 10 +
shims/0.23/pom.xml | 10 +
.../hadoop/hive/metastore/DirectSqlUpdatePart.java | 193 +-
.../apache/hadoop/hive/metastore/HMSHandler.java | 11 +-
.../apache/hadoop/hive/metastore/ObjectStore.java | 4 +-
.../TokenStoreDelegationTokenSecretManager.java | 54 +-
.../src/main/scripts/ext/schemaTool.sh | 2 +-
.../sql/hive/hive-schema-4.0.0-alpha-2.hive.sql | 2 +-
.../sql/hive/hive-schema-4.0.0-beta-1.hive.sql | 2 +-
.../src/main/sql/hive/hive-schema-4.0.0.hive.sql | 2 +-
...upgrade-4.0.0-alpha-1-to-4.0.0-alpha-2.hive.sql | 2 +-
...TestTokenStoreDelegationTokenSecretManager.java | 43 +-
115 files changed, 7643 insertions(+), 3441 deletions(-)
create mode 100644 data/files/legacyLeapYearInSingaporeTZ
create mode 100644 data/files/query-hive-28087.csv
create mode 100644
iceberg/iceberg-handler/src/test/queries/positive/iceberg_clustered.q
create mode 100644
iceberg/iceberg-handler/src/test/queries/positive/iceberg_major_compaction_query_metadata.q
create mode 100644
iceberg/iceberg-handler/src/test/results/positive/llap/iceberg_clustered.q.out
create mode 100644
iceberg/iceberg-handler/src/test/results/positive/llap/iceberg_major_compaction_query_metadata.q.out
create mode 100644
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestAuthWithDigestMD5.java
create mode 100644 ql/src/test/queries/clientpositive/auto_sortmerge_join_18.q
create mode 100644 ql/src/test/queries/clientpositive/cbo_join_constraints.q
create mode 100644
ql/src/test/queries/clientpositive/cbo_row_count_non_simple_filter.q
create mode 100644 ql/src/test/queries/clientpositive/filter_emptyliteral.q
create mode 100644 ql/src/test/queries/clientpositive/sharedworkresidual_dpp.q
create mode 100644
ql/src/test/queries/clientpositive/smb_join_with_different_bucket_size.q
create mode 100644
ql/src/test/queries/clientpositive/subquery_nested_expressions.q
create mode 100644
ql/src/test/results/clientpositive/llap/auto_sortmerge_join_18.q.out
create mode 100644
ql/src/test/results/clientpositive/llap/cbo_join_constraints.q.out
create mode 100644
ql/src/test/results/clientpositive/llap/cbo_row_count_non_simple_filter.q.out
create mode 100644
ql/src/test/results/clientpositive/llap/filter_emptyliteral.q.out
rename ql/src/test/results/clientpositive/{ => llap}/manyViewJoin.q.out (86%)
create mode 100644
ql/src/test/results/clientpositive/llap/sharedworkresidual_dpp.q.out
create mode 100644
ql/src/test/results/clientpositive/llap/smb_join_with_different_bucket_size.q.out
create mode 100644
ql/src/test/results/clientpositive/llap/subquery_nested_expressions.q.out
rename ql/src/test/results/clientpositive/{ => llap}/vector_udf_trunc.q.out
(52%)
create mode 100644
ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query30.q.out
create mode 100644
ql/src/test/results/clientpositive/perf/tpcds30tb/tez/query30.q.out