This is cool. Exactly what I was looking for ! Thanks Chris & Mahadev.
- Milind
On 10/11/11 2:33 PM, "Chris Douglas" wrote:
>On Tue, Oct 11, 2011 at 2:05 PM, wrote:
>> As part of MPI implementation in Yarn (aka Hamster), I was looking at
>> refactoring some of the functionality into node man
On Tue, Oct 11, 2011 at 2:05 PM, wrote:
> As part of MPI implementation in Yarn (aka Hamster), I was looking at
> refactoring some of the functionality into node manager services, so that
> it can be reused by other frameworks too. (Based on the discussion with
> some folks led me to believe that
Shuffle is an auxiliary service in the NM. You might want to take a look at
AuxServices.java in the NodeManager codebase.
Also, you can add config parameters to
yarn.nodemanager.aux-services
mapreduce.shuffle
(config knob in NodeManagers)
to add more configurable services in the NodeMan
Hi,
As part of MPI implementation in Yarn (aka Hamster), I was looking at
refactoring some of the functionality into node manager services, so that
it can be reused by other frameworks too. (Based on the discussion with
some folks led me to believe that suffle etc is also being planned as NM
servi
