[
https://issues.apache.org/jira/browse/SPARK-52769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Kupchinskiy updated SPARK-52769:
--------------------------------------
Priority: Minor (was: Major)
> InjectRuntimeFilter is not applied when hints or join type force the shuffle
> -----------------------------------------------------------------------------
>
> Key: SPARK-52769
> URL: https://issues.apache.org/jira/browse/SPARK-52769
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.5.6, 4.0.0
> Reporter: Artem Kupchinskiy
> Priority: Minor
>
> Currently, the logic of the InjectRuntimeFilter relies on shuffle detection
> in its children's plans. Basically, it tries to predict whether a join is
> gonna be broadcast or not. Since it uses only data size to determine it, the
> legit shuffle cases caused by hints or join type (i.e left/right outer joins
> when only key-preserved side fits in broadcast) are overlooked.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
