Rithwik Ediga Lakhamsani created SPARK-41916: ------------------------------------------------
Summary: Address `spark.task.resource.gpu.amount > 1` Key: SPARK-41916 URL: https://issues.apache.org/jira/browse/SPARK-41916 Project: Spark Issue Type: Sub-task Components: ML, PySpark Affects Versions: 3.4.0 Reporter: Rithwik Ediga Lakhamsani We want the distributor to have the ability to run multiple torchrun processes per task if task.gpu.amount > 1. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org