> On Aug. 6, 2018, 1:21 a.m., Ashutosh Chauhan wrote: > > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java > > Lines 1799 (patched) > > <https://reviews.apache.org/r/68202/diff/3/?file=2067607#file2067607line1799> > > > > This is not ideal. We want joinProject Transpose rule to run regardless > > of sq_count_check config. Infact, since sq_count_check generates more > > optimal plan, this especially needs to work with its value true. Looks like > > we need to enhance sq_count_check optimization so that it doesnt assume > > presence of project and can optimize the join-join plan too. > > > > On a side note default value for sq_count_check should be true. Lets do > > that too. > > Vineet Garg wrote: > sq_count_check can not be enhanced to work on JOIN-JOIN pattern since it > removes JOIN and can change the rowtype of the plan (that is why subquery > removal logic explicitly inserts Project b/w join so that removing join won't > change the rowtype). The reason sq_count_check is disabled by default is > because it adds extra project to all scalar subqueries preventing multiple > joins merging as well as preventing join reordering. Since this only > optimizes rare group by <constant> pattern but it impacts all scalar > subqueries it was decided to keep it off by default.
In latest patch I moved sq_count_check rule before JoinProjectTranspose rule. Now both rules will work together. - Vineet ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68202/#review206867 ----------------------------------------------------------- On Aug. 6, 2018, 2:58 a.m., Vineet Garg wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68202/ > ----------------------------------------------------------- > > (Updated Aug. 6, 2018, 2:58 a.m.) > > > Review request for hive, Ashutosh Chauhan and Jesús Camacho Rodríguez. > > > Bugs: HIVE-20292 > https://issues.apache.org/jira/browse/HIVE-20292 > > > Repository: hive-git > > > Description > ------- > > See JIRA description > > > Diffs > ----- > > itests/src/test/resources/testconfiguration.properties fa90750766 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java > 8214cc9a0d > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java d2d5152eff > ql/src/test/queries/clientpositive/q93_with_constraints.q PRE-CREATION > ql/src/test/results/clientpositive/correlationoptimizer5.q.out aab38e5768 > ql/src/test/results/clientpositive/join28.q.out 8a3d79da47 > ql/src/test/results/clientpositive/llap/column_access_stats.q.out > 1c873e2fe4 > ql/src/test/results/clientpositive/llap/correlationoptimizer2.q.out > cb5f357780 > ql/src/test/results/clientpositive/llap/correlationoptimizer3.q.out > c9bff2895c > ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_sw.q.out > 605686591f > ql/src/test/results/clientpositive/llap/join32_lessSize.q.out f1b4fb2ed3 > ql/src/test/results/clientpositive/llap/llap_smb_ptf.q.out 33da686fe3 > ql/src/test/results/clientpositive/llap/multiMapJoin1.q.out 261e153aba > ql/src/test/results/clientpositive/llap/q93_with_constraints.q.out > PRE-CREATION > ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 48fb6794ea > ql/src/test/results/clientpositive/llap/tez_self_join.q.out a362411d41 > ql/src/test/results/clientpositive/llap/vectorized_nested_mapjoin.q.out > 1bfe5af603 > ql/src/test/results/clientpositive/mapjoin_subquery.q.out 6116d382e8 > ql/src/test/results/clientpositive/mapjoin_subquery2.q.out 50e7ce0f3c > ql/src/test/results/clientpositive/perf/spark/query2.q.out 105f2ee1ae > ql/src/test/results/clientpositive/perf/spark/query59.q.out 73ea161fc4 > ql/src/test/results/clientpositive/perf/spark/query95.q.out 14636717af > ql/src/test/results/clientpositive/perf/tez/query2.q.out 5f908948aa > ql/src/test/results/clientpositive/perf/tez/query59.q.out 29cf1366d8 > ql/src/test/results/clientpositive/perf/tez/query95.q.out 3a8ed092fb > ql/src/test/results/clientpositive/ppd_repeated_alias.q.out b9434b107e > ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out > c1f3401f42 > ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 7366408ef9 > ql/src/test/results/clientpositive/spark/bucket_map_join_tez1.q.out > 00d5b6532a > ql/src/test/results/clientpositive/spark/column_access_stats.q.out > f372eefebb > ql/src/test/results/clientpositive/spark/join28.q.out 40d328c2bf > ql/src/test/results/clientpositive/spark/join32_lessSize.q.out 9c1e4361d8 > ql/src/test/results/clientpositive/spark/mapjoin_subquery.q.out ab25791aa6 > ql/src/test/results/clientpositive/spark/mapjoin_subquery2.q.out 112c1871e9 > > ql/src/test/results/clientpositive/spark/runtime_skewjoin_mapjoin_spark.q.out > 57ab0ecb63 > ql/src/test/results/clientpositive/spark/smb_mapjoin_25.q.out 16fa81870c > ql/src/test/results/clientpositive/spark/spark_use_op_stats.q.out > 26dd1f059f > ql/src/test/results/clientpositive/spark/subquery_scalar.q.out e2d51e85cf > ql/src/test/results/clientpositive/spark/vectorized_nested_mapjoin.q.out > 87983138aa > ql/src/test/results/clientpositive/vector_outer_join6.q.out 3c24928ffa > > > Diff: https://reviews.apache.org/r/68202/diff/4/ > > > Testing > ------- > > > Thanks, > > Vineet Garg > >
