[ https://issues.apache.org/jira/browse/HIVE-20954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694762#comment-16694762 ]
Hive QA commented on HIVE-20954: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12949035/HIVE-20954.1.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 15549 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[keep_uniform] (batchId=78) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets] (batchId=176) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14] (batchId=174) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_9] (batchId=176) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_exists] (batchId=179) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_in] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_not_in] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cluster] (batchId=166) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer2] (batchId=171) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer3] (batchId=181) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer6] (batchId=171) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[keep_uniform] (batchId=177) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3] (batchId=170) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partialdhj] (batchId=168) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=175) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in_having] (batchId=174) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_multi] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin] (batchId=177) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar] (batchId=169) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select] (batchId=169) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_views] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[table_access_keys_stats] (batchId=177) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side] (batchId=172) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_union_group_by] (batchId=180) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_auto_smb_mapjoin_14] (batchId=166) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/15027/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15027/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15027/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 25 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12949035 - PreCommit-HIVE-Build > Vector RS operator is not using uniform hash function for TPC-DS query 95 > ------------------------------------------------------------------------- > > Key: HIVE-20954 > URL: https://issues.apache.org/jira/browse/HIVE-20954 > Project: Hive > Issue Type: Improvement > Reporter: Teddy Choi > Assignee: Teddy Choi > Priority: Major > Labels: pull-request-available > Attachments: HIVE-20954.1.patch > > > Distribution of rows is skewed in DHJ causing slowdown. > Same RS outputs, but the two branches use VectorReduceSinkObjectHashOperator > and VectorReduceSinkLongOperator. > {code} > | Select Operator | > | expressions: ws_warehouse_sk (type: bigint), > ws_order_number (type: bigint) | > | outputColumnNames: _col0, _col1 | > | Select Vectorization: | > | className: VectorSelectOperator | > | native: true | > | projectedOutputColumnNums: [14, 16] | > | Statistics: Num rows: 7199963324 Data size: > 115185006696 Basic stats: COMPLETE Column stats: COMPLETE | > | Reduce Output Operator | > | key expressions: _col1 (type: bigint) | > | sort order: + | > | Map-reduce partition columns: _col1 (type: bigint) | > | Reduce Sink Vectorization: | > | className: VectorReduceSinkObjectHashOperator | > | keyColumnNums: [16] | > | native: true | > | nativeConditionsMet: > hive.vectorized.execution.reducesink.new.enabled IS true, > hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No > DISTINCT columns IS true, BinarySortableSerDe for keys IS true, > LazyBinarySerDe for values IS true | > | partitionColumnNums: [16] | > | valueColumnNums: [14] | > +----------------------------------------------------+ > | Explain | > +----------------------------------------------------+ > | Statistics: Num rows: 7199963324 Data size: > 115185006696 Basic stats: COMPLETE Column stats: COMPLETE | > | value expressions: _col0 (type: bigint) | > | Reduce Output Operator | > | key expressions: _col1 (type: bigint) | > | sort order: + | > | Map-reduce partition columns: _col1 (type: bigint) | > | Reduce Sink Vectorization: | > | className: VectorReduceSinkLongOperator | > | keyColumnNums: [16] | > | native: true | > | nativeConditionsMet: > hive.vectorized.execution.reducesink.new.enabled IS true, > hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No > DISTINCT columns IS true, BinarySortableSerDe for keys IS true, > LazyBinarySerDe for values IS true | > | valueColumnNums: [14] | > | Statistics: Num rows: 7199963324 Data size: > 115185006696 Basic stats: COMPLETE Column stats: COMPLETE | > | value expressions: _col0 (type: bigint) | > | Execution mode: vectorized, llap | > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)