> On Aug. 6, 2018, 1:21 a.m., Ashutosh Chauhan wrote: > > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java > > Lines 1799 (patched) > > <https://reviews.apache.org/r/68202/diff/3/?file=2067607#file2067607line1799> > > > > This is not ideal. We want joinProject Transpose rule to run regardless > > of sq_count_check config. Infact, since sq_count_check generates more > > optimal plan, this especially needs to work with its value true. Looks like > > we need to enhance sq_count_check optimization so that it doesnt assume > > presence of project and can optimize the join-join plan too. > > > > On a side note default value for sq_count_check should be true. Lets do > > that too.
sq_count_check can not be enhanced to work on JOIN-JOIN pattern since it removes JOIN and can change the rowtype of the plan (that is why subquery removal logic explicitly inserts Project b/w join so that removing join won't change the rowtype). The reason sq_count_check is disabled by default is because it adds extra project to all scalar subqueries preventing multiple joins merging as well as preventing join reordering. Since this only optimizes rare group by <constant> pattern but it impacts all scalar subqueries it was decided to keep it off by default. - Vineet ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68202/#review206867 ----------------------------------------------------------- On Aug. 4, 2018, 6:28 p.m., Vineet Garg wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68202/ > ----------------------------------------------------------- > > (Updated Aug. 4, 2018, 6:28 p.m.) > > > Review request for hive, Ashutosh Chauhan and Jesús Camacho Rodríguez. > > > Bugs: HIVE-20292 > https://issues.apache.org/jira/browse/HIVE-20292 > > > Repository: hive-git > > > Description > ------- > > See JIRA description > > > Diffs > ----- > > itests/src/test/resources/testconfiguration.properties a13ad28313 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java > 8214cc9a0d > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java e6e033066d > ql/src/test/queries/clientpositive/q93_with_constraints.q PRE-CREATION > ql/src/test/results/clientpositive/correlationoptimizer5.q.out aab38e5768 > ql/src/test/results/clientpositive/join28.q.out 8a3d79da47 > ql/src/test/results/clientpositive/llap/column_access_stats.q.out > 1c873e2fe4 > ql/src/test/results/clientpositive/llap/correlationoptimizer2.q.out > cb5f357780 > ql/src/test/results/clientpositive/llap/correlationoptimizer3.q.out > c9bff2895c > ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_sw.q.out > 605686591f > ql/src/test/results/clientpositive/llap/join32_lessSize.q.out f1b4fb2ed3 > ql/src/test/results/clientpositive/llap/llap_smb_ptf.q.out 33da686fe3 > ql/src/test/results/clientpositive/llap/multiMapJoin1.q.out 261e153aba > ql/src/test/results/clientpositive/llap/q93_with_constraints.q.out > PRE-CREATION > ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 48fb6794ea > ql/src/test/results/clientpositive/llap/tez_self_join.q.out a362411d41 > ql/src/test/results/clientpositive/llap/vectorized_nested_mapjoin.q.out > 1bfe5af603 > ql/src/test/results/clientpositive/mapjoin_subquery.q.out 6116d382e8 > ql/src/test/results/clientpositive/mapjoin_subquery2.q.out 50e7ce0f3c > ql/src/test/results/clientpositive/perf/spark/query2.q.out 105f2ee1ae > ql/src/test/results/clientpositive/perf/spark/query59.q.out 73ea161fc4 > ql/src/test/results/clientpositive/perf/spark/query95.q.out 14636717af > ql/src/test/results/clientpositive/perf/tez/query2.q.out 5f908948aa > ql/src/test/results/clientpositive/perf/tez/query59.q.out 29cf1366d8 > ql/src/test/results/clientpositive/perf/tez/query95.q.out 3a8ed092fb > ql/src/test/results/clientpositive/ppd_repeated_alias.q.out b9434b107e > ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out > c1f3401f42 > ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 7366408ef9 > ql/src/test/results/clientpositive/spark/bucket_map_join_tez1.q.out > 00d5b6532a > ql/src/test/results/clientpositive/spark/column_access_stats.q.out > f372eefebb > ql/src/test/results/clientpositive/spark/join28.q.out 40d328c2bf > ql/src/test/results/clientpositive/spark/join32_lessSize.q.out 9c1e4361d8 > ql/src/test/results/clientpositive/spark/mapjoin_subquery.q.out ab25791aa6 > ql/src/test/results/clientpositive/spark/mapjoin_subquery2.q.out 112c1871e9 > > ql/src/test/results/clientpositive/spark/runtime_skewjoin_mapjoin_spark.q.out > 57ab0ecb63 > ql/src/test/results/clientpositive/spark/smb_mapjoin_25.q.out 16fa81870c > ql/src/test/results/clientpositive/spark/spark_use_op_stats.q.out > 26dd1f059f > ql/src/test/results/clientpositive/spark/subquery_scalar.q.out e2d51e85cf > ql/src/test/results/clientpositive/spark/vectorized_nested_mapjoin.q.out > 87983138aa > ql/src/test/results/clientpositive/vector_outer_join6.q.out 3c24928ffa > > > Diff: https://reviews.apache.org/r/68202/diff/3/ > > > Testing > ------- > > > Thanks, > > Vineet Garg > >
