[
https://issues.apache.org/jira/browse/DRILL-6374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478615#comment-16478615
]
ASF GitHub Bot commented on DRILL-6374:
---------------------------------------
amansinha100 commented on a change in pull request #1262: DRILL-6374:
Transitive Closure leads to TPCH Queries regressions and OOM when run
concurrency test
URL: https://github.com/apache/drill/pull/1262#discussion_r188851626
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java
##########
@@ -237,12 +237,8 @@ protected DrillRel convertToRawDrel(final RelNode
relNode) throws SqlUnsupported
// HEP for rules, which are failed at the LOGICAL_PLANNING stage for
Volcano planner
final RelNode setOpTransposeNode = transform(PlannerType.HEP,
PlannerPhase.PRE_LOGICAL_PLANNING, relNode);
- // HEP Join Push Transitive Predicates
- final RelNode transitiveClosureNode =
- transform(PlannerType.HEP, PlannerPhase.TRANSITIVE_CLOSURE,
setOpTransposeNode);
-
- // HEP Directory pruning .
- final RelNode pruned = transform(PlannerType.HEP_BOTTOM_UP,
PlannerPhase.DIRECTORY_PRUNING, transitiveClosureNode);
+ // HEP Directory pruning.
+ final RelNode pruned = transform(PlannerType.HEP_BOTTOM_UP,
PlannerPhase.DIRECTORY_PRUNING, setOpTransposeNode);
Review comment:
I recall that back when I had reviewed the original transitive closure PR,
one of the things I had asked was why not do the TC before the
DIRECTORY_PRUNING phase such that new predicates that are created as a result
of TC would be used for pruning. So, now it seems that doing the TC too early
is causing a problem, is that right ? Does that mean that going forward we
won't support the scenario whether an newly inferred directory predicate (e.g
dir0 = 2015) would be used for directory pruning ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Transitive Closure leads to TPCH Queries regressions and OOM when run
> concurrency test
> --------------------------------------------------------------------------------------
>
> Key: DRILL-6374
> URL: https://issues.apache.org/jira/browse/DRILL-6374
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 1.14.0
> Environment: RHEL 7
> Reporter: Dechang Gu
> Assignee: Vitalii Diravka
> Priority: Critical
> Fix For: 1.14.0
>
> Attachments: TPCH_09_2_id_2517381b-1a61-3db5-40c3-4463bd421365.json,
> TPCH_09_2_id_2517497b-d4da-dab6-6124-abde5804a25f.json
>
>
> Run TPCH regression test on Apache Drill 1.14.0 master commit
> 6fcaf4268eddcb09010b5d9c5dfb3b3be5c3f903 (DRILL-6173), most of the queries
> regressed.
> In particular, TPC-H Query 9 takes about 4x time (36 sec vs 8.6 sec),
> comparing to that when run against the parent commit
> (9173308710c3decf8ff745493ad3e85ccdaf7c37).
> Further in the concurrency test for the commit, with 48 clients each running
> 16 TPCH queries (so total 768 queries are executed) with
> planner.width.max_per_node=5, some queries hit OOM and caused 273 queries
> failed, while for the parent commit all the 768 queries completed
> successfully.
>
> Profiles for TPCH_09 in the regression tests are uploaded:
> * The failing commit file name:
> [^TPCH_09_2_id_2517381b-1a61-3db5-40c3-4463bd421365.json],
> * The parent commit file name:
> [^TPCH_09_2_id_2517497b-d4da-dab6-6124-abde5804a25f.json] ).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
