In that case, having pre configured pools, but using the correct pool at
code level might do.
On Tue, Mar 17, 2015 at 11:23 AM, abhi abhishek...@gmail.com wrote:
yes .
Each generated job can have a different priority it is like a recursive
function, where in each iteration generate job will
Hi,
Maybe this is what you are looking for :
http://spark.apache.org/docs/1.2.0/job-scheduling.html#fair-scheduler-pools
Thanks,
On Mon, Mar 16, 2015 at 8:15 PM, abhi abhishek...@gmail.com wrote:
Hi
Current all the jobs in spark gets submitted using queue . i have a
requirement where
If i understand correctly , the above document creates pool for priority
which is static in nature and has to be defined before submitting the job .
.in my scenario each generated task can have different priority.
Thanks,
Abhi
On Mon, Mar 16, 2015 at 9:48 PM, twinkle sachdeva
http://apache-spark-developers-list.1001551.n3.nabble.com/Job-priority-td10076.html#a10079
On Mon, Mar 16, 2015 at 10:26 PM, abhi abhishek...@gmail.com wrote:
If i understand correctly , the above document creates pool for priority
which is static in nature and has to be defined before
yes .
Each generated job can have a different priority it is like a recursive
function, where in each iteration generate job will be submitted to the
spark cluster based on the priority. jobs will lower priority or less than
some threshold will be discarded.
Thanks,
Abhi
On Mon, Mar 16, 2015
