[jira] [Created] (HIVE-22643) Unable to execute method public java.lang.String udf.IpCityName.evaluate(java.lang.String) on object udf.IpCityName@2edadd75 of class udf.IpCityName with arguments {113.
FengWu created HIVE-22643:
-
Summary: Unable to execute method public java.lang.String
udf.IpCityName.evaluate(java.lang.String) on object udf.IpCityName@2edadd75 of
class udf.IpCityName with arguments {113.***..151:java.lang.String} of size
1
Key: HIVE-22643
URL: https://issues.apache.org/jira/browse/HIVE-22643
Project: Hive
Issue Type: Task
Components: UDF
Affects Versions: 2.1.1
Reporter: FengWu
Attachments: IpProvinceCode.java
hello,i want to use udf to Resolve IP address，a local file is need.i want to
create a new static object,but its error when run the sql.
if you cant understand,please check the javafilethanks
//我想用udf函数来解析ip地址，需要引用一个本体文件来创建一个实例对象City，但是定义成全局变量会出错，请问该怎么写。
create temporary function IpCityName as 'udf.IpCityName' 可以成功，但是运行时候会报错。
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 71820: HIVE-20150
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71820/ --- (Updated Dec. 13, 2019, 6:28 a.m.) Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich. Bugs: HIVE-20150 https://issues.apache.org/jira/browse/HIVE-20150 Repository: hive-git Description --- TopNKey pushdown 1. Apply patch: https://issues.apache.org/jira/secure/attachment/12941630/HIVE-20150.11.patch 2. TopNKey introduction depends only from Reduce Sink with topn property >= 0 3. Implement TopNKey operator pushdown through: projection, group by, redeuce sink, left outer join, other topnkey 4. Add sort order and null sort order direction check when determining if the topnkey op can be pushed 5. Implement handling cases when topnkey op and the parent op has a common key prefix only. 6. fix Key object inspectors non-vectorized mode 7. fix decimal64 to decimal cast issues when creating VectorExpressions of keyExpression during TopnKey vectorization Diffs (updated) - kudu-handler/src/test/results/positive/kudu_complex_queries.q.out 1324b27f8e ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java bbbde7978b ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java 0d6cf3c755 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 6876787e11 ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/CommonKeyPrefix.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyPushdownProcessor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 5c7a64c950 ql/src/test/org/apache/hadoop/hive/ql/optimizer/topnkey/TestCommonKeyPrefix.java PRE-CREATION ql/src/test/queries/clientpositive/topnkey.q 057b6a45ba ql/src/test/queries/clientpositive/vector_topnkey.q 85c5880cd6 ql/src/test/results/clientpositive/druid/druidmini_expressions.q.out 31862546b0 ql/src/test/results/clientpositive/druid/druidmini_test1.q.out fe75ddd894 ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 0c051c926b ql/src/test/results/clientpositive/llap/bucketmapjoin7.q.out 53e57b892f ql/src/test/results/clientpositive/llap/check_constraint.q.out 9f2c9a1cd0 ql/src/test/results/clientpositive/llap/constraints_optimization.q.out b6d210becf ql/src/test/results/clientpositive/llap/ctas.q.out c10d75d081 ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 4e62779446 ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out 6688944b78 ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 9343e078b7 ql/src/test/results/clientpositive/llap/explainuser_1.q.out 283a665a20 ql/src/test/results/clientpositive/llap/explainuser_2.q.out 0219af8833 ql/src/test/results/clientpositive/llap/external_jdbc_table_perf.q.out 545cce75a9 ql/src/test/results/clientpositive/llap/filter_union.q.out 0df77762a0 ql/src/test/results/clientpositive/llap/groupby_groupingset_bug.q.out 63d56172b7 ql/src/test/results/clientpositive/llap/insert1_overwrite_partitions.q.out d51654bc5b ql/src/test/results/clientpositive/llap/insert_into1.q.out dd9140d3b6 ql/src/test/results/clientpositive/llap/insert_into2.q.out 161318b720 ql/src/test/results/clientpositive/llap/lateral_view.q.out b742230a71 ql/src/test/results/clientpositive/llap/limit_join_transpose.q.out 6258f4fe1e ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 3fdd77d802 ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out efa8c38d7c ql/src/test/results/clientpositive/llap/materialized_view_rewrite_8.q.out 303111a7d4 ql/src/test/results/clientpositive/llap/offset_limit.q.out 23f2de46e5 ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 4ecb7bc46d ql/src/test/results/clientpositive/llap/orc_predicate_pushdown.q.out 2fd8faa7b5 ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out 0eac389eb7 ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out 4362fb6f2e ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out 24468c9a1b ql/src/test/results/clientpositive/llap/parquet_predicate_pushdown.q.out 8c900157c5 ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out 45890a1890 ql/src/test/results/clientpositive/llap/q93_with_constraints.q.out 5bba57633b ql/src/test/results/clientpositive/llap/smb_mapjoin_15.q.out 8ebc139faa ql/src/test/results/clientpositive/llap/subquery_ALL.q.out d910c1a79d ql/src/test/results/clientpositive/llap/subquery_ANY.q.out 91472d631e ql/src/test/results/clientpositive/llap/subquery_in.q.out b24ff5ca1e ql/src/test/results/clientpositive/llap/subquery_notin.q.out b08e8f3e2f
[jira] [Created] (HIVE-22642) Fix the TCLIService.thrift warning
Yuming Wang created HIVE-22642:
--
Summary: Fix the TCLIService.thrift warning
Key: HIVE-22642
URL: https://issues.apache.org/jira/browse/HIVE-22642
Project: Hive
Issue Type: Improvement
Reporter: Yuming Wang
{noformat}
TCLIService.thrift:361] Consider using the more efficient "binary" type instead
of "list"
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Review Request 71905: HIVE-22573:HMS filter metadata using Ranger plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71905/ --- Review request for hive, Naveen Gangam and Thejas Nair. Bugs: HIVE-22573 https://issues.apache.org/jira/browse/HIVE-22573 Repository: hive-git Description --- HIVE-22573:HMS filter metadata using Ranger plugin Diffs - ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java 434d1c9 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthzInfo.java a372c78 ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/ReadDatabaseEvent.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/events/ReadTableEvent.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/DatabaseFilterContext.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/filtercontext/TableFilterContext.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/TestHiveMetaStoreAuthorizer.java b9c0dcc Diff: https://reviews.apache.org/r/71905/diff/1/ Testing --- - Testing done in local VM with spark-shell for SHOW DATABASE, SHOW TABLES and Select commands. Thanks, Ramesh Mani
[jira] [Created] (HIVE-22641) Columns returned in sorted order when show columns query is run with no search pattern.
Chiran Ravani created HIVE-22641:
Summary: Columns returned in sorted order when show columns query
is run with no search pattern.
Key: HIVE-22641
URL: https://issues.apache.org/jira/browse/HIVE-22641
Project: Hive
Issue Type: Improvement
Components: Hive, HiveServer2
Affects Versions: 3.0.0
Reporter: Chiran Ravani
In Hive 1.2.1 and 2.0 while displaying columns for a table, it used to return
in same order as it was created. for example
{code}
create table col_order_test(server_name string, task_name string,
partition_name string, start_time string, end_time string, table_owner string,
table_name string) stored as orc;
show columns in col_order_test;
+-+--+
| field |
+-+--+
| server_name |
| task_name |
| partition_name |
| start_time |
| end_time|
| table_owner |
| table_name |
+-+--+
{code}
For Hive 3 columns are returned in sorted order for the same query, below is
output.
{code}
create table col_order_test(server_name string, task_name string,
partition_name string, start_time string, end_time string, table_owner string,
table_name string) stored as orc;
show columns in col_order_test;
+-+
| field |
+-+
| end_time|
| partition_name |
| server_name |
| start_time |
| table_name |
| table_owner |
| task_name |
+-+
{code}
Above behaviour looks to be changed with the introduction of search column
feature as part of Jira [HIVE-18373
|https://issues.apache.org/jira/browse/HIVE-18373]
This behaviour change can cause code to generate the INSERT OVERWRITE in a
different manner, which may result in query failure.
Would like to request community if we can improve the Jira [HIVE-18373
|https://issues.apache.org/jira/browse/HIVE-18373] by returning column order
same as it was created if search pattern provided by the user is null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22640) Decimal64ColumnVector: ClassCastException when partition column type is Decimal
Denys Kuzmenko created HIVE-22640: - Summary: Decimal64ColumnVector: ClassCastException when partition column type is Decimal Key: HIVE-22640 URL: https://issues.apache.org/jira/browse/HIVE-22640 Project: Hive Issue Type: Bug Reporter: Denys Kuzmenko -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71761: HIVE-22489
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71761/ --- (Updated Dec. 12, 2019, 2:59 p.m.) Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich. Bugs: HIVE-22489 https://issues.apache.org/jira/browse/HIVE-22489 Repository: hive-git Description (updated) --- Reduce Sink operator orders nulls first === 1. Set the default null sort order by hive config when creating Reduce Sink Desc. 2. Hash join uses `org.apache.hadoop.hive.serde2.binarysortable.fast.BinarySortableSerializeWrite` for selializing keys. For bigtable keys always ascending and nulls first ordering was hardcoded. This patch changes this behaviour to use the `Operator.getConf().TableDesc.getProperties()` (in this case `MapJoinOperator`) to setup ordering in `BinarySortableSerializeWrite` 3. Use the default null ordering when comparing keys in `CommonMergeJoinOperator` (This is the null ordering of the children Reduce Sink operators) Diffs (updated) - accumulo-handler/src/test/results/positive/accumulo_queries.q.out 7c552621f2 contrib/src/test/results/clientpositive/udaf_example_group_concat.q.out 6846720d95 hbase-handler/src/test/results/positive/hbase_queries.q.out a32ef81a7b itests/hive-blobstore/src/test/results/clientpositive/write_final_output_blobstore.q.out e997fa65cf kudu-handler/src/test/results/positive/kudu_complex_queries.q.out 1324b27f8e ql/src/java/org/apache/hadoop/hive/ql/exec/CommonMergeJoinOperator.java 3974627a24 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerBigOnlyMultiKeyOperator.java f587517b08 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinInnerMultiKeyOperator.java cdee3fd957 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinLeftSemiMultiKeyOperator.java e5d9fdae19 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinOuterMultiKeyOperator.java 29c531bd51 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedCreateHashTable.java 21c355cb42 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedLongCommon.java de1ee15c3b ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedLongHashMap.java 42573f0898 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedLongHashMultiSet.java 829a03737d ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedLongHashSet.java 18e1435019 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedStringCommon.java da0e8365b1 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedStringHashMap.java 6c4d8a81d1 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedStringHashMultiSet.java a6b754c7eb ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedStringHashSet.java fdcd83dde7 ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java 5c409e4573 ql/src/java/org/apache/hadoop/hive/ql/optimizer/CountDistinctRewriteProc.java a50ad78e8f ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 0f95d7788c ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java 89b55001f0 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/opconventer/HiveGBOpConvUtil.java 46ddffd4fa ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/opconventer/HiveOpConverterUtils.java 9cc1712f45 ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkReduceSinkMapJoinProc.java ac5caa6135 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 90549f9f3a ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 2314f49631 ql/src/java/org/apache/hadoop/hive/ql/util/NullOrdering.java 3bd25cd075 ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java dd70524948 ql/src/test/queries/clientpositive/hashjoin.q PRE-CREATION ql/src/test/results/clientnegative/bucket_mapjoin_mismatch1.q.out 62a36ac7e5 ql/src/test/results/clientnegative/spark/spark_job_max_tasks.q.out 9369fbd318 ql/src/test/results/clientnegative/spark/spark_stage_max_tasks.q.out 9369fbd318 ql/src/test/results/clientpositive/autoColumnStats_5a.q.out 9e2606f7d9 ql/src/test/results/clientpositive/autoColumnStats_8.q.out 90039f828e ql/src/test/results/clientpositive/auto_join_reordering_values.q.out d9c7720da5 ql/src/test/results/clientpositive/beeline/explain_outputs.q.out 4631cce5a4
[jira] [Created] (HIVE-22639) Bucket file name does not match bucket id after query based major compaction
Aron Hamvas created HIVE-22639:
--
Summary: Bucket file name does not match bucket id after query
based major compaction
Key: HIVE-22639
URL: https://issues.apache.org/jira/browse/HIVE-22639
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 3.1.0, 3.0.0
Reporter: Aron Hamvas
Assignee: Aron Hamvas
While debugging
{{TestCrudCompactorOnTez#testCompactionWithSchemaEvolutionAndBuckets()}}, it
has come to my attention, that even though before compaction, the file name of
the single bucket in the delta directories is {{bucket_1}}, in the new
base, the name of the new single bucket file is {{bucket_0}}. At the same
time, the bucket value in the ROW__ID of the records remain the same and
suggest that the bucket id is 1.
So the bucket id and the file name do not match. This could lead to problems.
The test itself does not reveal this issue, although I think that the tests
should check this, too. At the same time, the tests assume the exact bucket id
value in cases where it cannot be predicted and fail, even though the bucket it
does not change after the compaction, so the check should really pass.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22638) Fix insert statement issue with return path
Miklos Gergely created HIVE-22638: - Summary: Fix insert statement issue with return path Key: HIVE-22638 URL: https://issues.apache.org/jira/browse/HIVE-22638 Project: Hive Issue Type: Bug Components: Hive Reporter: Miklos Gergely Assignee: Miklos Gergely Insert statements were not handled properly with return path. It was revealed during examining why TestUpgradeTool is not working with return path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
HIVE-22611 Sub Tasks
Hello Gang, Do anyone know what HIVE-22611 sub-tasks are not being picked up by YETUS? Thanks!
[jira] [Created] (HIVE-22637) Avoid cost based rules during generating expressions from AST
Vineet Garg created HIVE-22637: -- Summary: Avoid cost based rules during generating expressions from AST Key: HIVE-22637 URL: https://issues.apache.org/jira/browse/HIVE-22637 Project: Hive Issue Type: Improvement Components: Query Planning Reporter: Vineet Garg Assignee: Vineet Garg genExprNode uses default dispatcher which fire rules based on cost, computation of cost is expensive and looks un-necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71903: When filter is optimized to False on a partition table, it will throw improperly SemanticException reporting that there is no partition predicate found.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71903/ --- (Updated 十二月 12, 2019, 12:36 p.m.) Review request for hive and Peter Vary. Bugs: HIVE-22634 https://issues.apache.org/jira/browse/HIVE-22634 Repository: hive-git Description --- HIVE-22634 Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 673d8580d5 Diff: https://reviews.apache.org/r/71903/diff/1/ Testing --- File Attachments HIVE-22634.patch https://reviews.apache.org/media/uploaded/files/2019/12/12/97ceb2dd-5427-457e-956a-7ad90d3e2cf3__HIVE-22634.patch Thanks, edison wang
Re: Review Request 71871: StringIndexOutOfBoundsException when getting sessionId from worker node name
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71871/ --- (Updated Dec. 12, 2019, 12:22 p.m.) Review request for hive, Laszlo Bodor, prasanthj, and Slim Bouguerra. Changes --- warning log if there is no suffix after worker- Bugs: HIVE-22577 https://issues.apache.org/jira/browse/HIVE-22577 Repository: hive-git Description --- The sequence number from the worker node name might be missing under some circumstances (the root cause is not fully clear it might be a zookeeper bug). In this case the following exception occurs: Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -1Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -1 at java.lang.String.substring(String.java:1931) at org.apache.hadoop.hive.registry.impl.ZkRegistryBase.extractSeqNum(ZkRegistryBase.java:781) at org.apache.hadoop.hive.registry.impl.ZkRegistryBase.populateCache(ZkRegistryBase.java:507) at org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl.access$000(LlapZookeeperRegistryImpl.java:65) at Diffs (updated) - llap-client/src/java/org/apache/hadoop/hive/registry/impl/ZkRegistryBase.java 5751b8ed939 Diff: https://reviews.apache.org/r/71871/diff/3/ Changes: https://reviews.apache.org/r/71871/diff/2-3/ Testing --- qtest Thanks, Attila Magyar
Re: Unstable tests ruin our days!
+1 for disabling flakies Or a flaky hunt? I would pay a beer for every fixed flaky test for the contributor, and another 2 if it was actually a bug... :) (3 years ago when we were actively fixing flakies 1 in every 3 were real bugs :( ) Zoltan Haindrich ezt írta (időpont: 2019. dec. 11., Sze 10:17): > Hey all! > > Recently the number of unstable tests have rised above the "usual" level... > These are usually hard to fix - and in most cases need a deep dive in the > area where the test operates. > Because of that I tend to just reattach the patch to the jira to get > another run in a day or so... > > The downside of the above approach is that reattaching on unrelated > failures has a positive hivqqa queuesize coefficient. > There is another downside which might not be obvious first: it reduces the > trust in the system and as a result there were cases when I did reattach > the patch; but it was a > genuine failure...it seemed unrelated; but actually it was. > > Instead of continuing to reattach patches every day; I would like to > propose a way to handle them: > > * check that the falling test has nothing to do with the actual patch >* it's important to be able to run test on our machines - but the most > important is to maintain that HiveQA is able to run them successfully; for > this reason I think > having 2 HiveQA runs for the same changeset where in one of them the > unstable test fails is the best >* you can search the jira for the testcase and look if other patches > have also bumped into it >* ? > * add a comment about that you are about the disable the test in > HIVE-22621 and commit it >* I think it would be ok to skip the regular code change process > * create a new subtask under HIVE-22619 with the details you know about > the falling testcase > * (resubmit your patch) > > What do you think? > > cheers, > Zoltan >
[jira] [Created] (HIVE-22636) Data loss on skewjoin for ACID tables.
Aditya Shah created HIVE-22636:
--
Summary: Data loss on skewjoin for ACID tables.
Key: HIVE-22636
URL: https://issues.apache.org/jira/browse/HIVE-22636
Project: Hive
Issue Type: Bug
Affects Versions: 4.0.0
Reporter: Aditya Shah
I am trying to do a skewjoin and writing the result into a FullAcid table. The
results are incorrect. The issue is similar to seen for MM tables in HIVE-16051
where the fix was to skip having a skewjoin for MM table.
Steps to reproduce:
Used a qtest similar to HIVE-16051:
{code:java}
--! qt:dataset:src1
--! qt:dataset:src
-- MASK_LINEAGE
set hive.mapred.mode=nonstrict;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.optimize.skewjoin=true;
set hive.skewjoin.key=2;
set hive.optimize.metadataonly=false;
CREATE TABLE skewjoin_acid(key INT, value STRING) STORED AS ORC tblproperties
("transactional"="true");
FROM src src1 JOIN src src2 ON (src1.key = src2.key) INSERT into TABLE
skewjoin_acid SELECT src1.key, src2.value;
select count(distinct key) from skewjoin_acid;
drop table skewjoin_acid;
{code}
The expected result for the count was 309 but got 173.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Review Request 71904: HIVE-21164: ACID: explore how we can avoid a move step during inserts/compaction
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71904/ --- Review request for hive, Gopal V and Peter Vary. Bugs: HIVE-21164 https://issues.apache.org/jira/browse/HIVE-21164 Repository: hive-git Description --- Extended the original patch with saving the task attempt ids in the file names and also fixed some bugs in the original patch. With this fix, inserting into an ACID table would not use move task to place the generated files into the final directory. It will inserts every files to the final directory and then clean up the files which are not needed (like written by failed task attempts). Also fixed the replication tests which failed for the original patch as well. Diffs - hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java da677c7 itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestAcidOnTez.java 2868427 itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 31d15fd itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java 445e39c itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java b7245e2 ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractFileMergeOperator.java 9a32581 ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java 9ad4e71 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 06e4ebe ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 3d30d09 ql/src/java/org/apache/hadoop/hive/ql/io/AcidInputFormat.java bba3960 ql/src/java/org/apache/hadoop/hive/ql/io/AcidOutputFormat.java 1e8bb22 ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 3c508ec ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java 8980a62 ql/src/java/org/apache/hadoop/hive/ql/io/RecordUpdater.java 737e677 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 76984ab ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.java c4c56f8 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java 2ac6232 ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 3fa61d3 ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java 2543dc6 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java f4bd0f9 ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 73ca658 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 90549f9 ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java c102a69 ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java ecc7bde ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java bed0581 ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 2b2cc1a ql/src/java/org/apache/hadoop/hive/ql/util/UpgradeTool.java 58e6289 ql/src/test/org/apache/hadoop/hive/ql/TestTxnAddPartition.java c9cb669 ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands.java 8421408 ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java 88ca683 ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands3.java 908ceb4 ql/src/test/org/apache/hadoop/hive/ql/TestTxnConcatenate.java 8676e0d ql/src/test/org/apache/hadoop/hive/ql/TestTxnExIm.java 66b2b27 ql/src/test/org/apache/hadoop/hive/ql/TestTxnLoadData.java bb55d9f ql/src/test/org/apache/hadoop/hive/ql/TestTxnNoBuckets.java ea6b1d9 ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java af14e62 ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java dd70524 ql/src/test/org/apache/hadoop/hive/ql/exec/TestFileSinkOperator.java 2c4b69b ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java c033a94 ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/CompactorTest.java cfd7290 ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestWorker.java 70ae85c ql/src/test/results/clientpositive/acid_subquery.q.out 1dc1775 ql/src/test/results/clientpositive/create_transactional_full_acid.q.out e324d5e ql/src/test/results/clientpositive/encrypted/encryption_insert_partition_dynamic.q.out 61b0057 ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 5571c53 ql/src/test/results/clientpositive/llap/insert_overwrite.q.out fbc3326 ql/src/test/results/clientpositive/llap/mm_all.q.out 7542a6a ql/src/test/results/clientpositive/mm_all.q.out 1377856 streaming/src/test/org/apache/hive/streaming/TestStreaming.java 58b3ae2 Diff: https://reviews.apache.org/r/71904/diff/1/ Testing --- Had to modify some tests because of the file name changes. Also added some specific tests. In the pre-commit run all tests passed successfully. Thanks, Marta Kuczora
[jira] [Created] (HIVE-22635) Disable scheduled query executor for unittests
Zoltan Haindrich created HIVE-22635: --- Summary: Disable scheduled query executor for unittests Key: HIVE-22635 URL: https://issues.apache.org/jira/browse/HIVE-22635 Project: Hive Issue Type: Bug Reporter: Zoltan Haindrich Assignee: Zoltan Haindrich Attachments: HIVE-22635.01.patch HIVE-21884 missed to set the default to off; so it may sometime interfere with unit tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 71903: When filter is optimized to False on a partition table, it will throw improperly SemanticException reporting that there is no partition predicate found.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71903/ --- Review request for hive and Peter Vary. Repository: hive-git Description --- HIVE-22634 Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 673d8580d5 Diff: https://reviews.apache.org/r/71903/diff/1/ Testing --- File Attachments HIVE-22634.patch https://reviews.apache.org/media/uploaded/files/2019/12/12/97ceb2dd-5427-457e-956a-7ad90d3e2cf3__HIVE-22634.patch Thanks, edison wang
[jira] [Created] (HIVE-22634) Improperly SemanticException when filter is optimized to False on a partition table
EdisonWang created HIVE-22634: - Summary: Improperly SemanticException when filter is optimized to False on a partition table Key: HIVE-22634 URL: https://issues.apache.org/jira/browse/HIVE-22634 Project: Hive Issue Type: Improvement Reporter: EdisonWang When filter is optimized to False on a partition table, it will throw improperly SemanticException reporting that there is no partition predicate found. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22633) GROUP BY query with SET hive.groupby.skewindata=true causes "java.lang.NullPointerException"
zhangbutao created HIVE-22633: - Summary: GROUP BY query with SET hive.groupby.skewindata=true causes "java.lang.NullPointerException" Key: HIVE-22633 URL: https://issues.apache.org/jira/browse/HIVE-22633 Project: Hive Issue Type: Bug Affects Versions: 3.1.1, 3.1.0 Reporter: zhangbutao -- This message was sent by Atlassian Jira (v8.3.4#803005)
