Github user amansinha100 commented on a diff in the pull request:
https://github.com/apache/drill/pull/519#discussion_r70706252
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java
---
@@ -143,6 +144,11 @@ public static final RelOptRule
getDirFilterOnScan(OptimizerRulesContext optimize
}
protected void doOnMatch(RelOptRuleCall call, Filter filterRel, Project
projectRel, TableScan scanRel) {
+ if (wasAllPartitionsPruned) {
--- End diff --
Agree that this should not be an internal state. I had added it to
overcome an issue where the prunescan rule gets re-applied even when there are
not qualifying partitions (this happens because we always populate the
newPartitions list with at least 1 entry to produce a valid schema during
execution).
I tried removing this flag but will need some associated changes to prevent
the rule from being reapplied without termination. The old code (without my
patch) terminates the pruning because FileSelection.supportDirPruning() returns
FALSE when the selection has directories. Whereas, for this optimization I
want to do the pruning on directories first. So, let me think of a way to
address that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
