Jacek Laskowski created SPARK-43912:
---------------------------------------
Summary: Incorrect SparkException for Stage-Level Scheduling in
local mode
Key: SPARK-43912
URL: https://issues.apache.org/jira/browse/SPARK-43912
Project: Spark
Issue Type: Bug
Components: Scheduler
Affects Versions: 3.4.0
Environment: ```text
scala> println(spark.version)
3.4.0
scala> println(sc.master)
local[*]
```
Reporter: Jacek Laskowski
While in `local[*]` mode, the following `SparkException` is thrown:
```text
org.apache.spark.SparkException: TaskResourceProfiles are only supported for
Standalone cluster for now when dynamic allocation is disabled.
at
org.apache.spark.resource.ResourceProfileManager.isSupported(ResourceProfileManager.scala:71)
at
org.apache.spark.resource.ResourceProfileManager.addResourceProfile(ResourceProfileManager.scala:126)
at org.apache.spark.rdd.RDD.withResources(RDD.scala:1802)
... 42 elided
```
This happens for the following snippet:
```scala
val rdd = sc.range(0, 9)
import org.apache.spark.resource.ResourceProfileBuilder
val rpb = new ResourceProfileBuilder
val rp1 = rpb.build()
rdd.withResources(rp1)
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
