In that case, having pre configured pools, but using the correct pool at
code level might do.
On Tue, Mar 17, 2015 at 11:23 AM, abhi abhishek...@gmail.com wrote:
yes .
Each generated job can have a different priority it is like a recursive
function, where in each iteration generate job will
Hi
Current all the jobs in spark gets submitted using queue . i have a
requirement where submitted job will generate another set of jobs with some
priority , which should again be submitted to spark cluster based on
priority ? Means job with higher priority should be executed first,Is
it
Hi,
Maybe this is what you are looking for :
http://spark.apache.org/docs/1.2.0/job-scheduling.html#fair-scheduler-pools
Thanks,
On Mon, Mar 16, 2015 at 8:15 PM, abhi abhishek...@gmail.com wrote:
Hi
Current all the jobs in spark gets submitted using queue . i have a
requirement where
If i understand correctly , the above document creates pool for priority
which is static in nature and has to be defined before submitting the job .
.in my scenario each generated task can have different priority.
Thanks,
Abhi
On Mon, Mar 16, 2015 at 9:48 PM, twinkle sachdeva
http://apache-spark-developers-list.1001551.n3.nabble.com/Job-priority-td10076.html#a10079
On Mon, Mar 16, 2015 at 10:26 PM, abhi abhishek...@gmail.com wrote:
If i understand correctly , the above document creates pool for priority
which is static in nature and has to be defined before
yes .
Each generated job can have a different priority it is like a recursive
function, where in each iteration generate job will be submitted to the
spark cluster based on the priority. jobs will lower priority or less than
some threshold will be discarded.
Thanks,
Abhi
On Mon, Mar 16, 2015