For others looking for a solution, here's one way to do it. You can add the arg directly to the code in hadoop-yarn-applications- distributedshell/src/main/java/org/apache/hadoop/yarn/ applications/distributedshell/Client.java
Rather than referencing line numbers (as there are frequent commits to this), find the variable 'vargs' inside Client.run(). The first item added to that list is the qualified path to /bin/java. vargs.add(Environment.JAVA_HOME.$() + "/bin/java"); Add this line below that: vargs.add("-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1337"); (This will be dependent on how you're building/running) >From the distributedshell directory, run mvn clean package -DskipTests; mvn install -DskipTests. You may need to copy the jar from target into hadoop-dist/... so it's properly available on Hadoop's classpath. Run the jar like normal hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami In Eclipse, set up a new debug configuration for a remote Java application. Select hadoop-yarn-applications-distributedshell as the project. Use localhost as the Host and 1337 as the port., with connection type of Standard (Socket Attach). Hit 'Debug' and Eclipse should connect to the running Application Master. Be sure to set a breakpoint inside main (or wherever) in the distributedshell project, which must be present in your Eclipse workspace. Now you can step through the AM. Let me know of any confusion or questions. Best, Curtis On Tue, Jun 18, 2013 at 3:10 PM, Curtis Ullerich <curtuller...@gmail.com>wrote: > I'm under the impression that this will launch hadoop (so, the > ResourceManager?) with those options, and not the ApplicationMaster. Is > that incorrect? > > > On Tue, Jun 18, 2013 at 2:29 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > >> If distributed shell is running as an unmanaged AM then you should set >> the debug flags for the 'hadoop jar' invocation, doing an 'export >> HADOOP_OPTS=....' with the debug flags would do. >> >> Thx >> >> >> On Tue, Jun 18, 2013 at 12:32 PM, Curtis Ullerich <curtuller...@gmail.com >> > wrote: >> >>> Update: It looks like I could add the flag at line 515 of >>> hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java >>> (package org.apache.hadoop.yarn.applications.distributedshell). >>> >>> I tried this: >>> vargs.add("-Xdebug >>> -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"); >>> >>> I recompiled the jar and launched it with >>> hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar >>> hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command >>> whoami >>> >>> I added a logging statement to make sure the change took effect. It >>> prints, but it doesn't seem to halt like the MR example (because of the >>> suspend=y option) even after launching the debugger. Why might this be? >>> >>> Thanks! >>> Curtis >>> >>> >>> >>> On Tue, Jun 18, 2013 at 8:23 AM, Curtis Ullerich <curtuller...@gmail.com >>> > wrote: >>> >>>> Hi Devaraj, >>>> >>>> That's what I'm not sure how to do. I want to debug by connecting >>>> remotely, and I'm not sure how to configure that in the code. I haven't >>>> found anything telling in the docs/source. Can you point me in the right >>>> direction? >>>> >>>> Thanks, >>>> Curtis >>>> >>>> >>>> On Mon, Jun 17, 2013 at 11:38 PM, Devaraj k <devara...@huawei.com>wrote: >>>> >>>>> Hi Curtis, **** >>>>> >>>>> ** ** >>>>> >>>>> "yarn.app.mapreduce.am.command-opts" configuration is specific to >>>>> MRAppMaster. It is not applicable for DistributedShell AM. **** >>>>> >>>>> ** ** >>>>> >>>>> If you want to dump out debug information then you can make use of the >>>>> debug option of DistributedShell application. If you want to debug by >>>>> connecting remotely, you need to update the DS application code >>>>> accordingly. >>>>> **** >>>>> >>>>> ** ** >>>>> >>>>> Thanks**** >>>>> >>>>> Devaraj K**** >>>>> >>>>> ** ** >>>>> >>>>> *From:* Curtis Ullerich [mailto:curtisuller...@gmail.com] >>>>> *Sent:* 18 June 2013 08:19 >>>>> *To:* user@hadoop.apache.org >>>>> *Subject:* Debugging YARN AM**** >>>>> >>>>> ** ** >>>>> >>>>> Hi all, **** >>>>> >>>>> ** ** >>>>> >>>>> I can successfully debug the MapReduce ApplicationMaster in standalone >>>>> mode by launching the pi estimator example with this command:**** >>>>> >>>>> ** ** >>>>> >>>>> hadoop jar hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi >>>>> "-Dyarn.app.mapreduce.am.command-opts=-Xdebug >>>>> -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000" 10 10** >>>>> ** >>>>> >>>>> ** ** >>>>> >>>>> and then attaching a debugger to port 8000 using Eclipse. This doesn't >>>>> work with the DistributedShell example, presumably because it's not >>>>> configurable with yarn.app.mapreduce.am.command as it's not MapReduce. >>>>> Looking in yarn-default.xml, I don't see an equivalent parameter. For >>>>> learning purposes, how can I debug the DistributedShell example (and other >>>>> AMs)?**** >>>>> >>>>> ** ** >>>>> >>>>> Thanks!**** >>>>> >>>>> ** ** >>>>> >>>>> Curtis**** >>>>> >>>> >>>> >>> >> >> >> -- >> Alejandro >> > >