[
https://issues.apache.org/jira/browse/SPARK-52767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Zhu updated SPARK-52767:
----------------------------------
Description:
The maxRows of join needs to calculate the maxRows of left and right twice
respectively, the number of maxRows executions of join increases exponentially
with the number of joins, poor performance when there are many joins.
So we can make the maxRows of left and right in join only calculated once.
> Optimize the performance of maxRows in join
> -------------------------------------------
>
> Key: SPARK-52767
> URL: https://issues.apache.org/jira/browse/SPARK-52767
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Mingliang Zhu
> Priority: Major
> Labels: pull-request-available
>
> The maxRows of join needs to calculate the maxRows of left and right twice
> respectively, the number of maxRows executions of join increases
> exponentially with the number of joins, poor performance when there are many
> joins.
> So we can make the maxRows of left and right in join only calculated once.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
