Re: Is storage resources counted during the scheduling

[Jialin Liu]

Thanks Ted, 
but that page seems to be scheduling policy, I have no idea of what resources 
are considered in the scheduling. 

And for scheduling, I’m wondering, in case of just one application, is there 
also a scheduling process?
otherwise, why I see some launching delay in the tasks. (well, this might be 
another question). Thanks. 

Best,
Jialin
> On Apr 11, 2016, at 3:18 PM, Ted Yu  wrote:
> 
> See 
> https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application
>  
> 
> 
> On Mon, Apr 11, 2016 at 3:15 PM, Jialin Liu  > wrote:
> Hi Spark users/experts,
> 
> I’m wondering how does the Spark scheduler work?
> What kind of resources will be considered during the scheduling, does it 
> include the disk resources or I/O resources, e.g., number of IO ports.
> Is network resources considered in that?
> 
> My understanding is that only CPU is considered, right?
> 
> Best,
> Jialin
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org 
> 
> For additional commands, e-mail: user-h...@spark.apache.org 
> 
> 
> 

Re: Is storage resources counted during the scheduling

[Ted Yu]

See
https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application

On Mon, Apr 11, 2016 at 3:15 PM, Jialin Liu  wrote:

> Hi Spark users/experts,
>
> I’m wondering how does the Spark scheduler work?
> What kind of resources will be considered during the scheduling, does it
> include the disk resources or I/O resources, e.g., number of IO ports.
> Is network resources considered in that?
>
> My understanding is that only CPU is considered, right?
>
> Best,
> Jialin
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>