[ https://issues.apache.org/jira/browse/SPARK-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen reopened SPARK-11882: ------------------------------- Just resolving as duplicate then > Allow for running Spark applications against a custom coarse grained scheduler > ------------------------------------------------------------------------------ > > Key: SPARK-11882 > URL: https://issues.apache.org/jira/browse/SPARK-11882 > Project: Spark > Issue Type: Wish > Components: Spark Core, Spark Submit > Reporter: Jacek Lewandowski > Priority: Minor > > SparkContext makes a decision which scheduler to use according to the Master > URI. How about running applications against a custom scheduler? Such a custom > scheduler would just extend {{CoarseGrainedSchedulerBackend}}. > The custom scheduler would be created by a provided factory. Factories would > be defined in the configuration like > {{spark.scheduler.factory.<name>=<factory-class>}}, where {{name}} is the > scheduler name. {{SparkContext}}, once it learns that master address is not > for standalone, Yarn, Mesos, local or any other predefined scheduler, it > would resolve scheme from the provided master URI and look for the scheduler > factory with the name equal to the resolved scheme. > For example: > {{spark.scheduler.factory.custom=org.a.b.c.CustomSchedulerFactory}} > then Master address would be {{custom://192.168.1.1}} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org