Discussing the idea of Shared Volume block store client

Hi Spark Community, Thank you for all the awesome features of Apache Spark 3.X like - GA of Spark On K8 - Push-based external shuffle - Easy mounting of volume claims in Spark K8s We run 1000s of spark jobs on K8s, Yarn (EMR), and Databricks. We run into shuffle block loss and RDD loss

unsubscribe

unsubscribe