[
https://issues.apache.org/jira/browse/IMPALA-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18104173#comment-18104173
]
ASF subversion and git services commented on IMPALA-15196:
----------------------------------------------------------
Commit 6517e517ff28cfddc1e8580567b1c52d11214ce3 in impala's branch
refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6517e517f ]
IMPALA-15195: Calcite Planner performance improvements for tpcds
This commit contains changes that improve performance on a variety of
tpcds queries. This was tested on a 3T system, and the junit files
in this commit reflect the plan changes that were tested on that
machine.
The changes include:
- Avoid calling "isInvertedJoinCheaper" used by the original planner.
The Calcite join optimizer has determined what it considers to be the
best plan, and inverting the joins according to the original planner
was causing some slower join orderings. IMPALA-15196 was filed to
make this code a little cleaner in the future so that the Calcite
planner never calls invertJoins in the original planner.
- Avoid the PROJECT_TO_SEMIJOIN rule in the Calcite planner before
join optimization. For tpcds query 10, a semi-join gets created. When
this happens before join optimization, the optimizer does considers
that join separately from the other joins. This rule is still used,
but only after the join optimization occurs.
- Improve the null/is not null selectivity estimation. The is not null
selectivity was returning the default of 10%, which resulted in very
erroneous calculations. Two major changes were made:
1) The default for "is null" data is now 2% (and is not null is 98%).
2) When an outer join is detected, the percentage of nulls is calculated
by returning a percentage of the number of rows that would have been
returned if it were an inner join divided by the total number of rows
on the non-outer join side.
- Lower the epsilon value when comparing costs. The join orderings were
being considered equal when the cost was < .01. This value resulted in
way too many equivalent joins. The epsilon value is now
RelOptUtil.EPSILON.
Change-Id: I0ee6662dc6500ea6ebd9f859de4df1740c51a058
Reviewed-on: http://gerrit.cloudera.org:8080/24605
Reviewed-by: Aleksandr Efimov <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Aman Sinha <[email protected]>
> Calcite Planner: avoid invertJoins call for Calcite planner
> -----------------------------------------------------------
>
> Key: IMPALA-15196
> URL: https://issues.apache.org/jira/browse/IMPALA-15196
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Steve Carlin
> Priority: Major
>
> IMPALA-15195 fixes some of the performance issues for the Calcite planner.
> One of the changes made was to avoid calling the invertJoins to find the
> cheapest plan. It would be a little cleaner if the invertJoins call could be
> avoided altogether
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
