Faiz Halde created SPARK-44526:
----------------------------------
Summary: Porting k8s PVC reuse logic to spark standalone
Key: SPARK-44526
URL: https://issues.apache.org/jira/browse/SPARK-44526
Project: Spark
Issue Type: New Feature
Components: Shuffle, Spark Core
Affects Versions: 3.4.1
Reporter: Faiz Halde
Hi,
This ticket is meant to understand the work that would be involved in porting
the PVC reuse feature onto the spark standalone cluster manager
The logic in
KubernetesLocalDiskShuffleDataIO
itself is not that much. However when I tried this on the
`LocalDiskShuffleExecutorComponents` it was not a successful experiment which
suggests there is more to it
I'd like to understand what will be the work involved for this. We'll be more
than happy to contribute
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
