Wow, thanks. Is this documented anywhere other than the code? I hate to waste y'alls time on things that can be RTFMed. John
-----Original Message----- From: Harsh J [mailto:ha...@cloudera.com] Sent: Wednesday, June 05, 2013 9:35 AM To: <user@hadoop.apache.org> Subject: Re: yarn-site.xml and aux-services John, The format is ID and sub-config based: First, you define an ID as a service, like the string "foo". This is the ID the applications may lookup in their container responses map we discussed over another thread (around shuffle handler). <property> <name>yarn.nodemanager.aux-services</name> <value>foo</value> </property> Then you define an actual implementation class for that ID "foo", like so: <property> <name>yarn.nodemanager.aux-services.foo.class</name> <value>com.mypack.MyAuxServiceClassForFoo</value> </property> If you have multiple services foo and bar, then it would appear like the below (comma separated IDs and individual configs): <property> <name>yarn.nodemanager.aux-services</name> <value>foo,bar</value> </property> <property> <name>yarn.nodemanager.aux-services.foo.class</name> <value>com.mypack.MyAuxServiceClassForFoo</value> </property> <property> <name>yarn.nodemanager.aux-services.bar.class</name> <value>com.mypack.MyAuxServiceClassForBar</value> </property> On Wed, Jun 5, 2013 at 8:42 PM, John Lilley <john.lil...@redpoint.net> wrote: > Good, I was hoping that would be the case. But what are the mechanics of it? > Do I just add another entry? And what exactly is "madreduce.shuffle"? A > scoped class name? Or a key string into some map elsewhere? > > e.g. like: > > <property> > <name>yarn.nodemanager.aux-services</name> > <value>mapreduce.shuffle</value> > </property> > <property> > <name>yarn.nodemanager.aux-services</name> > <value>myauxserviceclassname</value> > </property> > > Concerning auxiliary services -- do they communicate with NodeManager via > RPC? Is there an interface to implement? How are they opened and closed > with NodeManager? > > Thanks > John > > -----Original Message----- > From: Harsh J [mailto:ha...@cloudera.com] > Sent: Tuesday, June 04, 2013 11:58 PM > To: <user@hadoop.apache.org> > Subject: Re: yarn-site.xml and aux-services > > Yes, thats what this is for. You can implement, pass in and use your own > AuxService. It needs to be on the NodeManager CLASSPATH to run (and NM has to > be restarted to apply). > > On Wed, Jun 5, 2013 at 4:00 AM, John Lilley <john.lil...@redpoint.net> wrote: >> I notice the yarn-site.xml >> >> >> >> <property> >> >> <name>yarn.nodemanager.aux-services</name> >> >> <value>mapreduce.shuffle</value> >> >> <description>shuffle service that needs to be set for Map Reduce >> to run </description> >> >> </property> >> >> >> >> Is this a general-purpose hook? >> >> Can I tell yarn to run *my* per-node service? >> >> Is there some other way (within the recommended Hadoop framework) to >> run a per-node service that exists during the lifetime of the NodeManager? >> >> >> >> John Lilley >> >> Chief Architect, RedPoint Global Inc. >> >> 1515 Walnut Street | Suite 200 | Boulder, CO 80302 >> >> T: +1 303 541 1516 | M: +1 720 938 5761 | F: +1 781-705-2077 >> >> Skype: jlilley.redpoint | john.lil...@redpoint.net | www.redpoint.net >> >> > > > > -- > Harsh J -- Harsh J
