wuyi created SPARK-34091:
----------------------------
Summary: Shuffle batch fetch can't be disabled once it's enabled
previously
Key: SPARK-34091
URL: https://issues.apache.org/jira/browse/SPARK-34091
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0, 3.1.0
Reporter: wuyi
{code:java}
if (SQLConf.get.fetchShuffleBlocksInBatch) {
dependency.rdd.context.setLocalProperty(
SortShuffleManager.FETCH_SHUFFLE_BLOCKS_IN_BATCH_ENABLED_KEY, "true")
}
{code}
The current code has a problem that once we set `fetchShuffleBlocksInBatch` to
true first, we can never disable the batch fetch even if set
`fetchShuffleBlocksInBatch` to false later.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
