[
https://issues.apache.org/jira/browse/SPARK-35917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chandni Singh updated SPARK-35917:
----------------------------------
Description:
Push-based shuffle is partially merged in apache master but some of the tasks
are still incomplete. Since 3.2 is going to cut soon, we will not be able to
get the pending tasks reviewed and merged. Few of the pending tasks make
protocol changes to the push-based shuffle protocols, so we would like to
prevent users from enabling push-based shuffle both on the client and the
server until push-based shuffle implementation is complete.
We can prevent push-based shuffle to be used by throwing
{{UnsupportedOperationException}} (or something like that) both on the client
and the server when the user tries to enable it.
> Disable push-based shuffle until the feature is complete
> --------------------------------------------------------
>
> Key: SPARK-35917
> URL: https://issues.apache.org/jira/browse/SPARK-35917
> Project: Spark
> Issue Type: Sub-task
> Components: Shuffle, Spark Core
> Affects Versions: 3.1.0
> Reporter: Chandni Singh
> Priority: Major
>
> Push-based shuffle is partially merged in apache master but some of the tasks
> are still incomplete. Since 3.2 is going to cut soon, we will not be able to
> get the pending tasks reviewed and merged. Few of the pending tasks make
> protocol changes to the push-based shuffle protocols, so we would like to
> prevent users from enabling push-based shuffle both on the client and the
> server until push-based shuffle implementation is complete.
> We can prevent push-based shuffle to be used by throwing
> {{UnsupportedOperationException}} (or something like that) both on the client
> and the server when the user tries to enable it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
