Re: Is storage resources counted during the scheduling

Thanks Ted, but that page seems to be scheduling policy, I have no idea of what resources are considered in the scheduling. And for scheduling, I’m wondering, in case of just one application, is there also a scheduling process? otherwise, why I see some launching delay in the tasks. (well,

Re: Is storage resources counted during the scheduling

See https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application On Mon, Apr 11, 2016 at 3:15 PM, Jialin Liu wrote: > Hi Spark users/experts, > > I’m wondering how does the Spark scheduler work? > What kind of resources will be considered during the

Is storage resources counted during the scheduling

Hi Spark users/experts, I’m wondering how does the Spark scheduler work? What kind of resources will be considered during the scheduling, does it include the disk resources or I/O resources, e.g., number of IO ports. Is network resources considered in that? My understanding is that only CPU