[ https://issues.apache.org/jira/browse/DRILL-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007223#comment-15007223 ]
ASF GitHub Bot commented on DRILL-3765: --------------------------------------- Github user jinfengni commented on a diff in the pull request: https://github.com/apache/drill/pull/255#discussion_r44974839 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java --- @@ -116,20 +117,32 @@ public DrillSqlWorker(QueryContext context) { private RuleSet[] getRules(QueryContext context) { StoragePluginRegistry storagePluginRegistry = context.getStorage(); - RuleSet drillLogicalRules = DrillRuleSets.mergedRuleSets( + + // Ruleset for the case where VolcanoPlanner is used for everything : join, filter/project pushdown, partition pruning. + RuleSet drillLogicalVolOnlyRules = DrillRuleSets.mergedRuleSets( DrillRuleSets.getDrillBasicRules(context), + DrillRuleSets.getPruneScanRules(context), DrillRuleSets.getJoinPermRules(context), DrillRuleSets.getDrillUserConfigurableLogicalRules(context)); + + // Ruleset for the case where join planning is done in Hep-LOPT, filter/project pushdown and parttion pruning are done in VolcanoPlanner + RuleSet drillLogicalHepJoinRules = DrillRuleSets.mergedRuleSets( + DrillRuleSets.getDrillBasicRules(context), + DrillRuleSets.getPruneScanRules(context), + DrillRuleSets.getDrillUserConfigurableLogicalRules(context)); + + // Ruleset for the case where join planning and partition pruning is done in Hep, filter/project pushdown are done in VolcanoPlanner --- End diff -- Exactly. The ruleset defined here is used in Frameworks.VolcanoPlanner. In case that join / partition pruning is done in Hep, we do not put the join/partition pruning rules in the ruleset here. In stead, the join/partition pruning rules are specified when HepPlanner is built. > Partition prune rule is unnecessary fired multiple times. > ---------------------------------------------------------- > > Key: DRILL-3765 > URL: https://issues.apache.org/jira/browse/DRILL-3765 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization > Reporter: Jinfeng Ni > Assignee: Aman Sinha > > It seems that the partition prune rule may be fired multiple times, even > after the first rule execution has pushed the filter into the scan operator. > Since partition prune has to build the vectors to contain the partition /file > / directory information, to invoke the partition prune rule unnecessary may > lead to big memory overhead. > Drill planner should avoid the un-necessary partition prune rule, in order to > reduce the chance of hitting OOM exception, while the partition prune rule is > executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)