[
https://issues.apache.org/jira/browse/SPARK-40524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-40524:
----------------------------------
Summary: local mode with resource scheduling can hang (was: local mode
with resource scheduling should just fail)
> local mode with resource scheduling can hang
> --------------------------------------------
>
> Key: SPARK-40524
> URL: https://issues.apache.org/jira/browse/SPARK-40524
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.1.0
> Reporter: Thomas Graves
> Priority: Major
>
> If you try to run spark in local mode and request custom resources like
> GPU's, Spark will hang. Resource scheduling isn't supported in local mode so
> just removing the request for resources fixes the issue, but its really
> confusing to users since it just hangs.
>
> ie to reproduce:
> spark-sql --conf spark.executor.resource.gpu.amount=1 --conf
> spark.task.resource.gpu.amount=1
> Run:
> select 1
> result: hangs
> To fix run:
> spark-sql
>
> spark-sql> select 1;
> 1
> Time taken: 2.853 seconds, Fetched 1 row(s)
>
> It would be nice if we just fail to start or threw an exception when using
> those options in local mode
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
