Yes, it's a recent addition to CapacityScheduler.
It's available in hadoop-2.2.0 onwards. See
https://issues.apache.org/jira/browse/YARN-569.
Arun
On Feb 21, 2014, at 8:28 AM, ricky l rickylee0...@gmail.com wrote:
Does Hadoop capacity scheduler support preemption in this scenario?
Based on
Thank Arun for the update. That's very helpful - I was going to switch
to fair scheduler for the preemption purpose.
On Mon, Feb 24, 2014 at 5:51 PM, Arun C Murthy a...@hortonworks.com wrote:
Yes, it's a recent addition to CapacityScheduler.
It's available in hadoop-2.2.0 onwards. See
Does Hadoop capacity scheduler support preemption in this scenario?
Based on what Vinod says, the preemption seems to be supported by
configuration. If so, can someone point me an instruction to do that?
The preemption would really be helpful for my use-case. thanks.
On Fri, Feb 21, 2014 at 12:39
Yes, it does take those extra resources away back to queue B. How quickly it
takes them away depends on whether preemption is enabled or not. If preemption
is not enabled, it 'takes away' as and when containers from queue A start
finishing.
+Binod
On Feb 19, 2014, at 5:35 PM, Alex Nastetsky
Please help me understand how capacity and maximum-capacity on a queue
work in the Capacity Scheduler.
My understanding is that a queue is allocated capacity amount of
resources, and if it needs more, it can stretch up to maximum-capacity
resources.
But if that's the case, why do we need
Hi Alex
You can find good explanation from here:
http://hortonworks.com/blog/understanding-apache-hadoops-capacity-scheduler/
Short term: Capacity is the soft limit that queue is guaranteed for such an
amount of resource. For the purpose of necessary elasticity, queue can go
beyond capacity limit
Thanks, makes sense now.
On Feb 19, 2014 9:35 PM, Jian He j...@hortonworks.com wrote:
Yes, in the scenario you mentioned, the scheduler will take away the 10%
from queue B and give it back to queue A
On Wed, Feb 19, 2014 at 5:35 PM, Alex Nastetsky anastet...@spryinc.comwrote:
Thanks Jian.
