XiDuo You created SPARK-36085: --------------------------------- Summary: Make broadcast query stage executionContext isolation from AQE Key: SPARK-36085 URL: https://issues.apache.org/jira/browse/SPARK-36085 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.1.2, 3.2.0, 3.3.0 Reporter: XiDuo You
`AdaptiveSparkPlanExec` shares the execution context with `BroadcastQueryStage`, but it has a small number thread (hard code 16). Unfortunately, it's not enough in some complex queries and no thread is available to cancel the broadcast stage. In normal code path (without AQE), broadcast has it's own execution context with a config `spark.sql.broadcastExchange.maxThreadThreshold`. It should make sense to make broadcast query stage executionContext isolation from AQE and respect the existed thread threshold config. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org