[ https://issues.apache.org/jira/browse/SPARK-32299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen updated SPARK-32299: --------------------------------- Fix Version/s: (was: 3.1.0) Target Version/s: (was: 3.1.0) > Decide SMJ Join Orientation adaptively > -------------------------------------- > > Key: SPARK-32299 > URL: https://issues.apache.org/jira/browse/SPARK-32299 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.1.0 > Reporter: Mayur Bhosale > Priority: Minor > > For SortMerge join of type EquiJoin, the left and right side of the joins are > decided on the basis of the user order. In SMJ, the left side of the join is > streamed and the right side is buffered (matching values). Because of this, B > SMJ A would perform better than A SMJ B if, sizeOf(B) > sizeOf(A) > > With adaptive query execution, once both ShuffleQueryStages corresponding to > the join have completed and if none of them have sizes lesser than the > broadcast threshold, join orientation can be changed at run time. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org