One approach would be to use a Jobserver in between, create SparkContexts
in it. Lets say you create two, one which is configured to run on
coarse-grained and another set to fine-grained. Let the high priority jobs
hit the coarse-grained SparkContext and the other jobs use the fine-grained
one.
Hello *,
We are trying to build some Batch jobs using Spark on Mesos. Mesos offer's
two main mode of deployment of Spark job.
1. Fine-grained
2. Coarse-grained
When we are running the spark jobs in fine grained mode then spark is using
max amount of offers from Mesos and running the job.
