Re: Disable queuing of spark job on Mesos cluster if sufficient resources are not found

2017-05-30 Thread Michael Gummelt 
The driver will remain in the queue indefinitely, unless you issue a kill
command at /v1/submissions/kill/
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala#L64

On Mon, May 29, 2017 at 1:15 AM, Mevada, Vatsal 
wrote:

> Is there any configurable timeout which controls queuing of the driver in
> Mesos cluster mode or the driver will remain in queue for indefinite until
> it find resource on cluster?
>
>
>
> *From:* Michael Gummelt [mailto:mgumm...@mesosphere.io]
> *Sent:* Friday, May 26, 2017 11:33 PM
> *To:* Mevada, Vatsal 
> *Cc:* user@spark.apache.org
> *Subject:* Re: Disable queuing of spark job on Mesos cluster if
> sufficient resources are not found
>
>
>
> Nope, sorry.
>
>
>
> On Fri, May 26, 2017 at 4:38 AM, Mevada, Vatsal 
> wrote:
>
> Hello,
>
> I am using Mesos with cluster deployment mode to submit my jobs.
>
> When sufficient resources are not available on Mesos cluster, I can see
> that my jobs are queuing up on Mesos dispatcher UI.
>
> Is it possible to tweak some configuration so that my job submission fails
> gracefully(instead of queuing up) if sufficient resources are not found on
> Mesos cluster?
>
> Regards,
>
> Vatsal
>
>
>
>
> --
>
> Michael Gummelt
>
> Software Engineer
>
> Mesosphere
>



-- 
Michael Gummelt
Software Engineer
Mesosphere

RE: Disable queuing of spark job on Mesos cluster if sufficient resources are not found

2017-05-29 Thread Mevada, Vatsal 
Is there any configurable timeout which controls queuing of the driver in Mesos 
cluster mode or the driver will remain in queue for indefinite until it find 
resource on cluster?

From: Michael Gummelt [mailto:mgumm...@mesosphere.io]
Sent: Friday, May 26, 2017 11:33 PM
To: Mevada, Vatsal 
Cc: user@spark.apache.org
Subject: Re: Disable queuing of spark job on Mesos cluster if sufficient 
resources are not found

Nope, sorry.

On Fri, May 26, 2017 at 4:38 AM, Mevada, Vatsal 
> wrote:

Hello,

I am using Mesos with cluster deployment mode to submit my jobs.

When sufficient resources are not available on Mesos cluster, I can see that my 
jobs are queuing up on Mesos dispatcher UI.

Is it possible to tweak some configuration so that my job submission fails 
gracefully(instead of queuing up) if sufficient resources are not found on 
Mesos cluster?
Regards,
Vatsal



--
Michael Gummelt
Software Engineer
Mesosphere

Re: Disable queuing of spark job on Mesos cluster if sufficient resources are not found

2017-05-26 Thread Michael Gummelt 
Nope, sorry.

On Fri, May 26, 2017 at 4:38 AM, Mevada, Vatsal 
wrote:

> Hello,
>
> I am using Mesos with cluster deployment mode to submit my jobs.
>
> When sufficient resources are not available on Mesos cluster, I can see
> that my jobs are queuing up on Mesos dispatcher UI.
>
> Is it possible to tweak some configuration so that my job submission fails
> gracefully(instead of queuing up) if sufficient resources are not found on
> Mesos cluster?
>
> Regards,
>
> Vatsal
>



-- 
Michael Gummelt
Software Engineer
Mesosphere