Capacity Scheduler : how to use more than the queue capacity ?

, then jobs submitted to io-bound should be able to borrow up to 90 slots (50% total capacity). However, it looks like it never gets above 59 slots (33% of 180 slots). Is there something I missed ? Thanks, Sami Dalouche --- property namemapred.capacity-scheduler.queue.default.capacity/name

Re: Capacity Scheduler : how to use more than the queue capacity ?

Hi, I ended up finding another post about the exact same issue on this exact same mailing list, that was just a few days old... It looks like the setting to play with is mapred.capacity-scheduler.default-user-limit-factor Sami On Thu, Oct 20, 2011 at 1:25 PM, Sami Dalouche sa...@hopper.com

lazy-loading of Reduce's input

seems to require way too much memory. Thank You, Sami Dalouche

Re: lazy-loading of Reduce's input

Just to make sure I was clear-enough : - Is there a parameter that allows to set the size of the batch of elements that are retrieved to memory while the reduce task iterates on the input values ? Thanks, Sami dalouche On Mon, Oct 3, 2011 at 1:42 PM, Sami Dalouche sa...@hopper.com wrote: Hi