Wan Kun created SPARK-40715: ------------------------------- Summary: Support preferring shuffled hash join thought LocalMapThreshold is less than advisory partition size Key: SPARK-40715 URL: https://issues.apache.org/jira/browse/SPARK-40715 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.4.0 Reporter: Wan Kun
Now DynamicJoinSelection will prefer shuffled hash join only if *advisoryPartitionSize <= maxShuffledHashJoinLocalMapThreshold* We can also prefer shuffled hash join if the partitions in one side are smaller are smaller than advisoryPartitionSize, and the partitions in the other side are all larger enough(larger than advisoryPartitionSize) -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org