Re: Disable queuing of spark job on Mesos cluster if sufficient resources are not found

The driver will remain in the queue indefinitely, unless you issue a kill command at /v1/submissions/kill/ https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala#L64 On Mon, May 29, 2017 at 1:15 AM, Mevada, Vatsal

RE: Disable queuing of spark job on Mesos cluster if sufficient resources are not found

Is there any configurable timeout which controls queuing of the driver in Mesos cluster mode or the driver will remain in queue for indefinite until it find resource on cluster? From: Michael Gummelt [mailto:mgumm...@mesosphere.io] Sent: Friday, May 26, 2017 11:33 PM To: Mevada, Vatsal

Re: Disable queuing of spark job on Mesos cluster if sufficient resources are not found

Nope, sorry. On Fri, May 26, 2017 at 4:38 AM, Mevada, Vatsal wrote: > Hello, > > I am using Mesos with cluster deployment mode to submit my jobs. > > When sufficient resources are not available on Mesos cluster, I can see > that my jobs are queuing up on Mesos

Disable queuing of spark job on Mesos cluster if sufficient resources are not found

Hello, I am using Mesos with cluster deployment mode to submit my jobs. When sufficient resources are not available on Mesos cluster, I can see that my jobs are queuing up on Mesos dispatcher UI. Is it possible to tweak some configuration so that my job submission fails gracefully(instead of