[ https://issues.apache.org/jira/browse/HIVE-948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-948: ----------------------------- Attachment: HIVE-948.D8463.1.patch navis requested code review of "HIVE-948 [jira] more query plan optimization rules". Reviewers: JIRA DPAL-1980 Implement cleanup precessor for logical plan Many query plans are not optimal in that they contain redundant operators. Some examples are unnecessary select operators (select followed by select, select output being the same as input etc.). Even though these operators are not very expensive, they could account for around 10% of CPU time in some simple queries. It seems they are low-hanging fruits that we should pick first. BTW, it seems these optimization rules should be added at the last stage of the physical optimization phase since some redundant operators are added to facilitate physical plan generation. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D8463 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/optimizer/CleanupProcessor.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicateTransitivePropagate.java ql/src/test/results/clientpositive/alias_casted_column.q.out ql/src/test/results/clientpositive/ambiguous_col.q.out ql/src/test/results/clientpositive/auto_join1.q.out ql/src/test/results/clientpositive/auto_join12.q.out ql/src/test/results/clientpositive/auto_join14_hadoop20.q.out ql/src/test/results/clientpositive/auto_join17.q.out ql/src/test/results/clientpositive/auto_join19.q.out ql/src/test/results/clientpositive/auto_join2.q.out ql/src/test/results/clientpositive/auto_join20.q.out ql/src/test/results/clientpositive/auto_join22.q.out ql/src/test/results/clientpositive/auto_join26.q.out ql/src/test/results/clientpositive/auto_join28.q.out ql/src/test/results/clientpositive/auto_join29.q.out ql/src/test/results/clientpositive/auto_join3.q.out ql/src/test/results/clientpositive/auto_join4.q.out ql/src/test/results/clientpositive/auto_join5.q.out ql/src/test/results/clientpositive/auto_join6.q.out ql/src/test/results/clientpositive/auto_join7.q.out ql/src/test/results/clientpositive/auto_join8.q.out ql/src/test/results/clientpositive/auto_join9.q.out ql/src/test/results/clientpositive/binarysortable_1.q.out ql/src/test/results/clientpositive/bucket_groupby.q.out ql/src/test/results/clientpositive/bucket_map_join_1.q.out ql/src/test/results/clientpositive/bucket_map_join_2.q.out ql/src/test/results/clientpositive/bucketcontext_1.q.out ql/src/test/results/clientpositive/bucketcontext_2.q.out ql/src/test/results/clientpositive/bucketcontext_3.q.out ql/src/test/results/clientpositive/bucketcontext_4.q.out ql/src/test/results/clientpositive/bucketcontext_5.q.out ql/src/test/results/clientpositive/bucketcontext_6.q.out ql/src/test/results/clientpositive/bucketcontext_7.q.out ql/src/test/results/clientpositive/bucketcontext_8.q.out ql/src/test/results/clientpositive/bucketmapjoin1.q.out ql/src/test/results/clientpositive/bucketmapjoin10.q.out ql/src/test/results/clientpositive/bucketmapjoin11.q.out ql/src/test/results/clientpositive/bucketmapjoin12.q.out ql/src/test/results/clientpositive/bucketmapjoin13.q.out ql/src/test/results/clientpositive/bucketmapjoin2.q.out ql/src/test/results/clientpositive/bucketmapjoin3.q.out ql/src/test/results/clientpositive/bucketmapjoin4.q.out ql/src/test/results/clientpositive/bucketmapjoin5.q.out ql/src/test/results/clientpositive/bucketmapjoin8.q.out ql/src/test/results/clientpositive/bucketmapjoin9.q.out ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out ql/src/test/results/clientpositive/column_access_stats.q.out ql/src/test/results/clientpositive/create_view.q.out ql/src/test/results/clientpositive/ppd1.q.out ql/src/test/results/clientpositive/ppd2.q.out ql/src/test/results/clientpositive/ppd_clusterby.q.out ql/src/test/results/clientpositive/ppd_constant_expr.q.out ql/src/test/results/clientpositive/ppd_gby.q.out ql/src/test/results/clientpositive/ppd_gby2.q.out ql/src/test/results/clientpositive/ppd_gby_join.q.out ql/src/test/results/clientpositive/ppd_join.q.out ql/src/test/results/clientpositive/ppd_join2.q.out ql/src/test/results/clientpositive/ppd_join3.q.out ql/src/test/results/clientpositive/ppd_multi_insert.q.out ql/src/test/results/clientpositive/ppd_random.q.out ql/src/test/results/clientpositive/ppd_repeated_alias.q.out ql/src/test/results/clientpositive/ppd_udf_col.q.out ql/src/test/results/clientpositive/ppd_union.q.out ql/src/test/results/clientpositive/ppd_union_view.q.out MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/20637/ To: JIRA, navis > more query plan optimization rules > ----------------------------------- > > Key: HIVE-948 > URL: https://issues.apache.org/jira/browse/HIVE-948 > Project: Hive > Issue Type: Improvement > Reporter: Ning Zhang > Attachments: HIVE-948.D8463.1.patch > > > Many query plans are not optimal in that they contain redundant operators. > Some examples are unnecessary select operators (select followed by select, > select output being the same as input etc.). Even though these operators are > not very expensive, they could account for around 10% of CPU time in some > simple queries. It seems they are low-hanging fruits that we should pick > first. > BTW, it seems these optimization rules should be added at the last stage of > the physical optimization phase since some redundant operators are added to > facilitate physical plan generation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira