Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan closed pull request #47533: [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages URL: https://github.com/apache/spark/pull/47533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on PR #47533: URL: https://github.com/apache/spark/pull/47533#issuecomment-2268312075 thanks for the review, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

yaooqinn commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1701895376 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDefa

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

yaooqinn commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1701153168 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDefa

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1701137491 ## sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala: ## @@ -61,23 +61,43 @@ trait BroadcastExchangeLike extends Exchange {

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1701137034 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDef

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

ulysses-you commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1701098965 ## sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala: ## @@ -61,23 +61,43 @@ trait BroadcastExchangeLike extends Exchange

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

yaooqinn commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1700392298 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDefa

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1700339034 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDef

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

ulysses-you commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1697849217 ## sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala: ## @@ -61,23 +61,33 @@ trait BroadcastExchangeLike extends Exchange

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1697759562 ## sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala: ## @@ -61,23 +61,33 @@ trait BroadcastExchangeLike extends Exchange {

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

ulysses-you commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1696900342 ## sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala: ## @@ -61,23 +61,33 @@ trait BroadcastExchangeLike extends Exchange

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

yaooqinn commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1696716244 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDefa

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

yaooqinn commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1696716244 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala: ## @@ -170,6 +170,16 @@ object StaticSQLConf { .intConf .createWithDefa

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on code in PR #47533: URL: https://github.com/apache/spark/pull/47533#discussion_r1696676905 ## sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala: ## @@ -61,23 +61,33 @@ trait BroadcastExchangeLike extends Exchange {

Re: [PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan commented on PR #47533: URL: https://github.com/apache/spark/pull/47533#issuecomment-2257942947 cc @ulysses-you @yaooqinn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] [SPARK-49057][SQL] Do not block the AQE loop when submitting query stages [spark]

cloud-fan opened a new pull request, #47533: URL: https://github.com/apache/spark/pull/47533 ### What changes were proposed in this pull request? We missed the fact that submitting a shuffle or broadcast query stage can be heavy, as it needs to submit subqueries and wait for t