RE: Tips for debugging pig

2011-01-26 Thread jacob
ttempt_201101261105_0002_r_06_0 0.667% reduce > reduce > 2011-01-26 11:14:38,625 INFO org.apache.hadoop.mapred.TaskTracker: > attempt_201101261105_0002_r_09_0 0.667% reduce > reduce > > > It then hangs at 66%. > > About the script, is there a way of

RE: Tips for debugging pig

2011-01-26 Thread Martin Z
-01-26 11:14:38,625 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201101261105_0002_r_09_0 0.667% reduce > reduce It then hangs at 66%. About the script, is there a way of telling in which part of it the process is hanging? Thanks! > Subject: Re: Tips for debugging pig > From: jac

Re: Tips for debugging pig

2011-01-26 Thread jacob
Martin, When you look at the task logs for the particular reducer that's stuck, what do you see? What kind of operations do you have going on in the script, possibly a GROUP ALL? --jacob On Wed, 2011-01-26 at 12:44 -0300, Martin Z wrote: > Hi all, > > I'm running a Pig script in local mo

Tips for debugging pig

2011-01-26 Thread Martin Z
Hi all, I'm running a Pig script in local mode, and it finishes successfully. When I use the same dataset and script to run pig in its distributed mode, it hangs at 90% and the hadoop processes in the node machines takes almost all the memory. It always hangs at the reduce task of the last job