You can also use mrunit [1] to write unit tests against your MapReduce code.
-Joey [1] http://incubator.apache.org/mrunit/ On Thu, Sep 15, 2011 at 1:18 AM, Subroto Sanyal <[email protected]>wrote: > ** > > Hi,**** > > ** ** > > MapReduce framework provide different in built approaches to debug a Job:* > *** > > **1) **LocalJobRunner (Running the Job without a cluster; check > basic functionality of the Job implementation)**** > > **2) **IsolationRunner (Running a failed TaskAttempt on cluster- > Remote debugging can be done)**** > > ** ** > > Apart from this we can use Sysouts and loggings in the Map or Reduce Tasks. > **** > > These logs will come in “sysout”,”syslog”,”syserr” accordingly. These files > will be present in the userlogs folder.**** > > ** ** > > User can pass Remote Debug argument to Child Process to enable remote > debugging of Child process.**** > > If you have access through putty, then you can use remote Debug option and > connect through eclipse as well.**** > > ** ** > > ** ** > > *Regards,* > Subroto Sanyal**** > ------------------------------ > > *From:* [email protected] [mailto:[email protected]] > *Sent:* Thursday, September 15, 2011 9:49 AM > > *To:* **[email protected]** > *Subject:* Re: Debugging Mapreduce programs**** > > ** ** > > Shreya > There are multiple ways to debug map reduce programs. I found comfortable > using the jobtracker web UI. You can your job tracker web UI at > http://<server ip>:50030/jobtracker.jsp > Open it in a browser. > There you can find the running,failed and retired jobs. You can drill down > on any job to task tracker log levels from there. > Mostly I get the root cause of my job failures from there. > > Hope it helps**** > > Regards > Bejoy K S**** > ------------------------------ > > *From: *<[email protected]> **** > > *Date: *Thu, 15 Sep 2011 09:39:23 +0530**** > > *To: *<**[email protected]**>**** > > *ReplyTo: ***[email protected]** **** > > *Subject: *Debugging Mapreduce programs**** > > ** ** > > Hi**** > > ** ** > > Can we use sysouts in java to debug MR programs.**** > > Where would those sysouts be printed?**** > > ** ** > > What would be the best way to debug a remote system where I have access to > the machine only through putty.**** > > ** ** > > Regards**** > > Shreya**** > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. > If you are not the intended recipient, please contact the sender by reply > e-mail and destroy all copies of the original message. > Any unauthorised review, use, disclosure, dissemination, forwarding, > printing or copying of this email or any action taken in reliance on this > e-mail is strictly > prohibited and may be unlawful. > -- Joseph Echeverria Cloudera, Inc. 443.305.9434
