[ https://issues.apache.org/jira/browse/SPARK-40524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated SPARK-40524: ---------------------------------- Summary: local mode with resource scheduling can hang (was: local mode with resource scheduling should just fail) > local mode with resource scheduling can hang > -------------------------------------------- > > Key: SPARK-40524 > URL: https://issues.apache.org/jira/browse/SPARK-40524 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.1.0 > Reporter: Thomas Graves > Priority: Major > > If you try to run spark in local mode and request custom resources like > GPU's, Spark will hang. Resource scheduling isn't supported in local mode so > just removing the request for resources fixes the issue, but its really > confusing to users since it just hangs. > > ie to reproduce: > spark-sql --conf spark.executor.resource.gpu.amount=1 --conf > spark.task.resource.gpu.amount=1 > Run: > select 1 > result: hangs > To fix run: > spark-sql > > spark-sql> select 1; > 1 > Time taken: 2.853 seconds, Fetched 1 row(s) > > It would be nice if we just fail to start or threw an exception when using > those options in local mode -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org