----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71820/#review219047 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java Lines 4341 (patched) <https://reviews.apache.org/r/71820/#comment307082> What is the bug in Decimal64 to Decimal conversion? Do we need a call to the function fixDecimalDataTypePhysicalVariations() ? Because I see the function getVectorExpressionsUpConvertDecimal64() is calling wrapWithDecimal64ToDecimalConversion() on all the child expressions. - Ramesh Kumar Thangarajan On Dec. 14, 2019, 10:31 a.m., Krisztian Kasa wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71820/ > ----------------------------------------------------------- > > (Updated Dec. 14, 2019, 10:31 a.m.) > > > Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich. > > > Bugs: HIVE-20150 > https://issues.apache.org/jira/browse/HIVE-20150 > > > Repository: hive-git > > > Description > ------- > > TopNKey pushdown > ================ > 1. Apply patch: > https://issues.apache.org/jira/secure/attachment/12941630/HIVE-20150.11.patch > 2. TopNKey introduction depends only from Reduce Sink with topn property >= 0 > 3. Implement TopNKey operator pushdown through: projection, group by, redeuce > sink, left outer join, other topnkey > 4. Add sort order and null sort order direction check when determining if the > topnkey op can be pushed > 5. Implement handling cases when topnkey op and the parent op has a common > key prefix only. > 6. fix Key object inspectors non-vectorized mode > 7. fix decimal64 to decimal cast issues when creating VectorExpressions of > keyExpression during TopnKey vectorization > > > Diffs > ----- > > kudu-handler/src/test/results/positive/kudu_complex_queries.q.out > 1324b27f8e > ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java bbbde7978b > ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java > 0d6cf3c755 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java > 6876787e11 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/CommonKeyPrefix.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/topnkey/TopNKeyPushdownProcessor.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 5c7a64c950 > > ql/src/test/org/apache/hadoop/hive/ql/optimizer/topnkey/TestCommonKeyPrefix.java > PRE-CREATION > ql/src/test/queries/clientpositive/topnkey.q 057b6a45ba > ql/src/test/queries/clientpositive/vector_topnkey.q 85c5880cd6 > ql/src/test/results/clientpositive/druid/druidmini_expressions.q.out > 31862546b0 > ql/src/test/results/clientpositive/druid/druidmini_test1.q.out fe75ddd894 > ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 0c051c926b > ql/src/test/results/clientpositive/llap/bucketmapjoin7.q.out 53e57b892f > ql/src/test/results/clientpositive/llap/check_constraint.q.out 9f2c9a1cd0 > ql/src/test/results/clientpositive/llap/constraints_optimization.q.out > b6d210becf > ql/src/test/results/clientpositive/llap/ctas.q.out c10d75d081 > > ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out > 4e62779446 > ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out > 6688944b78 > ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out > 9343e078b7 > ql/src/test/results/clientpositive/llap/explainuser_1.q.out 64fdc481eb > ql/src/test/results/clientpositive/llap/explainuser_2.q.out 0219af8833 > ql/src/test/results/clientpositive/llap/external_jdbc_table_perf.q.out > 545cce75a9 > ql/src/test/results/clientpositive/llap/filter_union.q.out 0df77762a0 > ql/src/test/results/clientpositive/llap/groupby_groupingset_bug.q.out > 63d56172b7 > ql/src/test/results/clientpositive/llap/insert1_overwrite_partitions.q.out > d51654bc5b > ql/src/test/results/clientpositive/llap/insert_into1.q.out dd9140d3b6 > ql/src/test/results/clientpositive/llap/insert_into2.q.out 161318b720 > ql/src/test/results/clientpositive/llap/lateral_view.q.out b742230a71 > ql/src/test/results/clientpositive/llap/limit_join_transpose.q.out > 6258f4fe1e > ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 3fdd77d802 > ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out efa8c38d7c > ql/src/test/results/clientpositive/llap/materialized_view_rewrite_8.q.out > 303111a7d4 > ql/src/test/results/clientpositive/llap/offset_limit.q.out 23f2de46e5 > ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out > 4ecb7bc46d > ql/src/test/results/clientpositive/llap/orc_predicate_pushdown.q.out > 2fd8faa7b5 > ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out > 0eac389eb7 > > ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out > 4362fb6f2e > > ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out > 24468c9a1b > ql/src/test/results/clientpositive/llap/parquet_predicate_pushdown.q.out > 8c900157c5 > > ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out > 45890a1890 > ql/src/test/results/clientpositive/llap/q93_with_constraints.q.out > 5bba57633b > ql/src/test/results/clientpositive/llap/smb_mapjoin_15.q.out 8ebc139faa > ql/src/test/results/clientpositive/llap/subquery_ALL.q.out d910c1a79d > ql/src/test/results/clientpositive/llap/subquery_ANY.q.out 91472d631e > ql/src/test/results/clientpositive/llap/subquery_in.q.out 479e60fda9 > ql/src/test/results/clientpositive/llap/subquery_notin.q.out 3dc247591c > ql/src/test/results/clientpositive/llap/subquery_scalar.q.out fd65f872c6 > ql/src/test/results/clientpositive/llap/subquery_select.q.out 9f40436116 > ql/src/test/results/clientpositive/llap/temp_table.q.out c972b10601 > ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out > 5367d33e0d > ql/src/test/results/clientpositive/llap/topnkey.q.out 1e77587f82 > ql/src/test/results/clientpositive/llap/union_top_level.q.out ac39cc2622 > ql/src/test/results/clientpositive/llap/vector_case_when_2.q.out 3501884ce5 > ql/src/test/results/clientpositive/llap/vector_case_when_conversion.q.out > 4da58e0c2b > ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out > cc2dc47280 > ql/src/test/results/clientpositive/llap/vector_char_2.q.out f7e76e5a8b > ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 9573b3db77 > ql/src/test/results/clientpositive/llap/vector_data_types.q.out df9c3499bf > ql/src/test/results/clientpositive/llap/vector_decimal_expressions.q.out > 8500562893 > > ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out > 6fd15e7101 > ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out > d6325982e3 > ql/src/test/results/clientpositive/llap/vector_identity_reuse.q.out > 00aa4fbf7d > ql/src/test/results/clientpositive/llap/vector_llap_text_1.q.out 1f7ed5a129 > ql/src/test/results/clientpositive/llap/vector_partitioned_date_time.q.out > 48e840584e > ql/src/test/results/clientpositive/llap/vector_string_concat.q.out > a8019be7aa > ql/src/test/results/clientpositive/llap/vector_topnkey.q.out c140bdfd37 > ql/src/test/results/clientpositive/llap/vector_varchar_simple.q.out > 6bb2fbde0c > ql/src/test/results/clientpositive/llap/vectorization_13.q.out 3f9f1b7770 > ql/src/test/results/clientpositive/llap/vectorization_7.q.out bfcd6a3568 > ql/src/test/results/clientpositive/llap/vectorization_8.q.out 1253c2bc83 > ql/src/test/results/clientpositive/llap/vectorization_div0.q.out 73ca096e22 > ql/src/test/results/clientpositive/llap/vectorization_limit.q.out > 7326adf522 > ql/src/test/results/clientpositive/llap/vectorization_part_project.q.out > 1c22230f2d > ql/src/test/results/clientpositive/llap/vectorization_short_regress.q.out > cdeb909b02 > ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out e9308cd709 > ql/src/test/results/clientpositive/perf/tez/cbo_query54.q.out e96d4c60a6 > ql/src/test/results/clientpositive/perf/tez/cbo_query58.q.out 523f02e7d7 > ql/src/test/results/clientpositive/perf/tez/cbo_query6.q.out 944b113e7b > ql/src/test/results/clientpositive/perf/tez/cbo_query77.q.out 02caf99f7d > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out > 43e1b2b5c2 > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out > ece76c3e4e > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query58.q.out > 954bdc4dd1 > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query6.q.out > eb872d7e6d > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query77.q.out > 2f75361df1 > ql/src/test/results/clientpositive/perf/tez/constraints/mv_query44.q.out > a546472908 > ql/src/test/results/clientpositive/perf/tez/constraints/query1.q.out > 3aea12dda6 > ql/src/test/results/clientpositive/perf/tez/constraints/query10.q.out > bb3b1b6660 > ql/src/test/results/clientpositive/perf/tez/constraints/query11.q.out > cc47c7e7a1 > ql/src/test/results/clientpositive/perf/tez/constraints/query12.q.out > 57c16347e5 > ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out > 228b20a8d7 > ql/src/test/results/clientpositive/perf/tez/constraints/query15.q.out > 5268ed3ecf > ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out > d96222d9e1 > ql/src/test/results/clientpositive/perf/tez/constraints/query18.q.out > c3c17a2efc > ql/src/test/results/clientpositive/perf/tez/constraints/query19.q.out > fff6c4951d > ql/src/test/results/clientpositive/perf/tez/constraints/query1b.q.out > a6276715b4 > ql/src/test/results/clientpositive/perf/tez/constraints/query20.q.out > b71831c7c6 > ql/src/test/results/clientpositive/perf/tez/constraints/query21.q.out > af2e33e785 > ql/src/test/results/clientpositive/perf/tez/constraints/query22.q.out > cd3c0cc431 > ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out > adabb76e04 > ql/src/test/results/clientpositive/perf/tez/constraints/query26.q.out > 824bbe6769 > ql/src/test/results/clientpositive/perf/tez/constraints/query27.q.out > abbd02d6c9 > ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out > c308771dfb > ql/src/test/results/clientpositive/perf/tez/constraints/query3.q.out > 4c1f168168 > ql/src/test/results/clientpositive/perf/tez/constraints/query30.q.out > ea1622883b > ql/src/test/results/clientpositive/perf/tez/constraints/query33.q.out > f00567eeb1 > ql/src/test/results/clientpositive/perf/tez/constraints/query35.q.out > 23b3399123 > ql/src/test/results/clientpositive/perf/tez/constraints/query36.q.out > f7bccf94f7 > ql/src/test/results/clientpositive/perf/tez/constraints/query37.q.out > 187ad5c5b5 > ql/src/test/results/clientpositive/perf/tez/constraints/query4.q.out > 68796c2274 > ql/src/test/results/clientpositive/perf/tez/constraints/query40.q.out > 070b5cb1f5 > ql/src/test/results/clientpositive/perf/tez/constraints/query42.q.out > b2dc7b3a8a > ql/src/test/results/clientpositive/perf/tez/constraints/query43.q.out > b5a6c746d1 > ql/src/test/results/clientpositive/perf/tez/constraints/query44.q.out > 7644f5ec61 > ql/src/test/results/clientpositive/perf/tez/constraints/query45.q.out > 3f5dbf4beb > ql/src/test/results/clientpositive/perf/tez/constraints/query46.q.out > 804013fb80 > ql/src/test/results/clientpositive/perf/tez/constraints/query47.q.out > 701861a888 > ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out > b384aea779 > ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out > d3f79820f2 > ql/src/test/results/clientpositive/perf/tez/constraints/query50.q.out > 8c9754967f > ql/src/test/results/clientpositive/perf/tez/constraints/query51.q.out > 4255d5cc33 > ql/src/test/results/clientpositive/perf/tez/constraints/query52.q.out > 56d73b8d46 > ql/src/test/results/clientpositive/perf/tez/constraints/query53.q.out > f8f495d3a8 > ql/src/test/results/clientpositive/perf/tez/constraints/query54.q.out > d85597af02 > ql/src/test/results/clientpositive/perf/tez/constraints/query55.q.out > fb52de2aba > ql/src/test/results/clientpositive/perf/tez/constraints/query56.q.out > cd438d4ae3 > ql/src/test/results/clientpositive/perf/tez/constraints/query57.q.out > 7896f97fed > ql/src/test/results/clientpositive/perf/tez/constraints/query58.q.out > 5fa126bf6e > ql/src/test/results/clientpositive/perf/tez/constraints/query59.q.out > 82148531bc > ql/src/test/results/clientpositive/perf/tez/constraints/query6.q.out > d6031153b1 > ql/src/test/results/clientpositive/perf/tez/constraints/query60.q.out > 06a5689938 > ql/src/test/results/clientpositive/perf/tez/constraints/query63.q.out > 8d45d8260d > ql/src/test/results/clientpositive/perf/tez/constraints/query65.q.out > 8b8a4146a6 > ql/src/test/results/clientpositive/perf/tez/constraints/query66.q.out > be612609cf > ql/src/test/results/clientpositive/perf/tez/constraints/query67.q.out > ee1bf12068 > ql/src/test/results/clientpositive/perf/tez/constraints/query68.q.out > 97b97ce19d > ql/src/test/results/clientpositive/perf/tez/constraints/query69.q.out > d7469ae5a9 > ql/src/test/results/clientpositive/perf/tez/constraints/query7.q.out > b2eccdbe90 > ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out > d5efae9aeb > ql/src/test/results/clientpositive/perf/tez/constraints/query72.q.out > 9d6a3f6a00 > ql/src/test/results/clientpositive/perf/tez/constraints/query74.q.out > 68ff9ebf26 > ql/src/test/results/clientpositive/perf/tez/constraints/query75.q.out > b1341a2fdb > ql/src/test/results/clientpositive/perf/tez/constraints/query76.q.out > ce4f7cb061 > ql/src/test/results/clientpositive/perf/tez/constraints/query77.q.out > 95ab61bed2 > ql/src/test/results/clientpositive/perf/tez/constraints/query78.q.out > ecb434d3cd > ql/src/test/results/clientpositive/perf/tez/constraints/query79.q.out > 9e9bbec9e2 > ql/src/test/results/clientpositive/perf/tez/constraints/query8.q.out > 170bccf406 > ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out > b18f89373c > ql/src/test/results/clientpositive/perf/tez/constraints/query81.q.out > 2b859c230c > ql/src/test/results/clientpositive/perf/tez/constraints/query82.q.out > 8dd6ae9f0f > ql/src/test/results/clientpositive/perf/tez/constraints/query83.q.out > a2163802ad > ql/src/test/results/clientpositive/perf/tez/constraints/query84.q.out > 89b11e66e2 > ql/src/test/results/clientpositive/perf/tez/constraints/query85.q.out > 6bd833fd12 > ql/src/test/results/clientpositive/perf/tez/constraints/query86.q.out > 0b6ee2c156 > ql/src/test/results/clientpositive/perf/tez/constraints/query89.q.out > 28d604a433 > ql/src/test/results/clientpositive/perf/tez/constraints/query93.q.out > 193a542b53 > ql/src/test/results/clientpositive/perf/tez/constraints/query99.q.out > c77a73f4d5 > ql/src/test/results/clientpositive/perf/tez/query1.q.out f7affa4f14 > ql/src/test/results/clientpositive/perf/tez/query10.q.out b346a5c5fb > ql/src/test/results/clientpositive/perf/tez/query11.q.out 96bdd7b504 > ql/src/test/results/clientpositive/perf/tez/query12.q.out 4183059b2d > ql/src/test/results/clientpositive/perf/tez/query14.q.out 069fad2b4a > ql/src/test/results/clientpositive/perf/tez/query15.q.out 3670a718b3 > ql/src/test/results/clientpositive/perf/tez/query17.q.out df70fbc46e > ql/src/test/results/clientpositive/perf/tez/query18.q.out d61a57d469 > ql/src/test/results/clientpositive/perf/tez/query19.q.out 5cdeab2bf0 > ql/src/test/results/clientpositive/perf/tez/query1b.q.out b7975918c3 > ql/src/test/results/clientpositive/perf/tez/query20.q.out 2553c1ed76 > ql/src/test/results/clientpositive/perf/tez/query21.q.out 5e158834d7 > ql/src/test/results/clientpositive/perf/tez/query22.q.out e721f09f82 > ql/src/test/results/clientpositive/perf/tez/query25.q.out d006795c79 > ql/src/test/results/clientpositive/perf/tez/query26.q.out a1bf3b099b > ql/src/test/results/clientpositive/perf/tez/query27.q.out 6f49de2344 > ql/src/test/results/clientpositive/perf/tez/query29.q.out 5066893829 > ql/src/test/results/clientpositive/perf/tez/query3.q.out fe97d402c4 > ql/src/test/results/clientpositive/perf/tez/query30.q.out ac683c2515 > ql/src/test/results/clientpositive/perf/tez/query33.q.out dc4f6b17c1 > ql/src/test/results/clientpositive/perf/tez/query35.q.out 265c51bb72 > ql/src/test/results/clientpositive/perf/tez/query36.q.out 544d883846 > ql/src/test/results/clientpositive/perf/tez/query37.q.out 2724fd44dc > ql/src/test/results/clientpositive/perf/tez/query4.q.out 99552295f0 > ql/src/test/results/clientpositive/perf/tez/query40.q.out 4b65c82e00 > ql/src/test/results/clientpositive/perf/tez/query42.q.out 3c059f85f6 > ql/src/test/results/clientpositive/perf/tez/query43.q.out eb19d41926 > ql/src/test/results/clientpositive/perf/tez/query44.q.out adaea8c46e > ql/src/test/results/clientpositive/perf/tez/query45.q.out 4538a6540d > ql/src/test/results/clientpositive/perf/tez/query46.q.out 3e6358d907 > ql/src/test/results/clientpositive/perf/tez/query47.q.out e54c366848 > ql/src/test/results/clientpositive/perf/tez/query49.q.out 9c34eccceb > ql/src/test/results/clientpositive/perf/tez/query5.q.out 38fba27a8e > ql/src/test/results/clientpositive/perf/tez/query50.q.out 6e34831de6 > ql/src/test/results/clientpositive/perf/tez/query51.q.out 906d702207 > ql/src/test/results/clientpositive/perf/tez/query52.q.out b579d5fbff > ql/src/test/results/clientpositive/perf/tez/query53.q.out 1bee5e099c > ql/src/test/results/clientpositive/perf/tez/query54.q.out a19bd171e5 > ql/src/test/results/clientpositive/perf/tez/query55.q.out 5cee50ed50 > ql/src/test/results/clientpositive/perf/tez/query56.q.out 2be886a3eb > ql/src/test/results/clientpositive/perf/tez/query57.q.out b732cc873f > ql/src/test/results/clientpositive/perf/tez/query58.q.out 57f3477e48 > ql/src/test/results/clientpositive/perf/tez/query59.q.out 566f888d40 > ql/src/test/results/clientpositive/perf/tez/query6.q.out 5c49ab4517 > ql/src/test/results/clientpositive/perf/tez/query60.q.out e77c89ba69 > ql/src/test/results/clientpositive/perf/tez/query63.q.out 3cdf37b734 > ql/src/test/results/clientpositive/perf/tez/query65.q.out dbc5c79a87 > ql/src/test/results/clientpositive/perf/tez/query66.q.out 7ddcc21f92 > ql/src/test/results/clientpositive/perf/tez/query67.q.out cbacaae150 > ql/src/test/results/clientpositive/perf/tez/query68.q.out b9321679a9 > ql/src/test/results/clientpositive/perf/tez/query69.q.out d11b5494e0 > ql/src/test/results/clientpositive/perf/tez/query7.q.out c17ec8aeb9 > ql/src/test/results/clientpositive/perf/tez/query70.q.out 7fe3a62399 > ql/src/test/results/clientpositive/perf/tez/query72.q.out bebc3852b7 > ql/src/test/results/clientpositive/perf/tez/query74.q.out f98b9c8815 > ql/src/test/results/clientpositive/perf/tez/query75.q.out b426bf2d1a > ql/src/test/results/clientpositive/perf/tez/query76.q.out c0d60e88cc > ql/src/test/results/clientpositive/perf/tez/query77.q.out ab2b3dc570 > ql/src/test/results/clientpositive/perf/tez/query78.q.out 4d090af463 > ql/src/test/results/clientpositive/perf/tez/query79.q.out e1ef37ad38 > ql/src/test/results/clientpositive/perf/tez/query8.q.out 0af8fdf3df > ql/src/test/results/clientpositive/perf/tez/query80.q.out 47844158fa > ql/src/test/results/clientpositive/perf/tez/query81.q.out 7feb6175fe > ql/src/test/results/clientpositive/perf/tez/query82.q.out c7721acffe > ql/src/test/results/clientpositive/perf/tez/query83.q.out 54c1ba6161 > ql/src/test/results/clientpositive/perf/tez/query84.q.out 4fd546bc07 > ql/src/test/results/clientpositive/perf/tez/query85.q.out c411a92145 > ql/src/test/results/clientpositive/perf/tez/query86.q.out 37414f697f > ql/src/test/results/clientpositive/perf/tez/query89.q.out 927da252da > ql/src/test/results/clientpositive/perf/tez/query93.q.out 0df41d120b > ql/src/test/results/clientpositive/perf/tez/query99.q.out c01122f435 > ql/src/test/results/clientpositive/tez/explainanalyze_3.q.out 7d303849d5 > ql/src/test/results/clientpositive/tez/explainuser_3.q.out 2377a6674e > ql/src/test/results/clientpositive/tez/topnkey.q.out cf2ecf7133 > ql/src/test/results/clientpositive/tez/vector_non_string_partition.q.out > d449fd3754 > ql/src/test/results/clientpositive/tez/vector_topnkey.q.out d179013e28 > ql/src/test/results/clientpositive/topnkey.q.out cecbe89b1c > > > Diff: https://reviews.apache.org/r/71820/diff/14/ > > > Testing > ------- > > Run q tests using TestMiniLlapLocalCliDriver > topnkey.q > vector_topnkey.q > > > Thanks, > > Krisztian Kasa > >