Re: The reduce copier failed

2008-09-26 Thread Joe Shaw
Hi, On Fri, Sep 26, 2008 at 2:50 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote: >> Caused by: org.apache.lucene.index.MergePolicy$MergeException: segment >> "_bfu exists in external directory yet the MergeScheduler executed the >> merge in a separate thread >> at > > Hmm... could this be somet

Re: The reduce copier failed

2008-09-25 Thread Arun C Murthy
2008-09-25 17:12:18,250 INFO org.apache.hadoop.mapred.ReduceTask: attempt_200809180916_0027_r_07_2: Got 2 new map-outputs & number of known map outputs is 21 2008-09-25 17:12:18,251 WARN org.apache.hadoop.mapred.ReduceTask: attempt_200809180916_0027_r_07_2 Merge of the inmemory files thre

Re: The reduce copier failed

2008-09-25 Thread Joe Shaw
Hi again, Ugh, sorry about the butchered output. On Thu, Sep 25, 2008 at 5:42 PM, Joe Shaw <[EMAIL PROTECTED]> wrote: > Hi, > > On Thu, Sep 25, 2008 at 5:32 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote: >> What did the logs for the task-attempt >> 'attempt_200809180916_0027_r_07_2' look like?

Re: The reduce copier failed

2008-09-25 Thread Joe Shaw
Hi, On Thu, Sep 25, 2008 at 5:32 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote: > What did the logs for the task-attempt > 'attempt_200809180916_0027_r_07_2' look like? Did the TIP/Job succeed? You mean inside userlogs/attempt_blah_blah/syslog? I didn't know about this log file before, thanks!

Re: The reduce copier failed

2008-09-25 Thread Arun C Murthy
On Sep 25, 2008, at 2:26 PM, Joe Shaw wrote: Hi, I'm trying to build an index using the "index" contrib in Hadoop 0.18.0, but the reduce tasks are consistently failing. What did the logs for the task-attempt 'attempt_200809180916_0027_r_07_2' look like? Did the TIP/Job succeed? Ar

The reduce copier failed

2008-09-25 Thread Joe Shaw
Hi, I'm trying to build an index using the "index" contrib in Hadoop 0.18.0, but the reduce tasks are consistently failing. In the output from the "hadoop jar" command, I see messages like this: 08/09/25 14:12:11 INFO mapred.JobClient: map 27% reduce 4% 08/09/25 14:12:23 INFO mapred.JobClient:

Re: 0.15.3 on ec2 - using manual hadoop install no to ec2 magic: The reduce copier failed

2008-02-25 Thread Jason Venner
Venner <[EMAIL PROTECTED]> wrote: All of the test/sample jobs fail with either out of memory or The reduce copier failed. Anyone have any tips on this? Our non ec2 installations seem to work just fine. -- Jason Venner Attributor - Publish with Confidence <http://www.attri

Re: 0.15.3 on ec2 - using manual hadoop install no to ec2 magic: The reduce copier failed

2008-02-25 Thread Tom White
This might be because the tasktracker child VMs need more memory. Try setting mapred.child.java.opts to something higher like -Xmx512m. Tom On 25/02/2008, Jason Venner <[EMAIL PROTECTED]> wrote: > All of the test/sample jobs fail with either out of memory or The reduce > c

Re: 0.15.3 on ec2 - using manual hadoop install no to ec2 magic: The reduce copier failed

2008-02-25 Thread Paco NATHAN
, Feb 25, 2008 at 2:40 PM, Jason Venner <[EMAIL PROTECTED]> wrote: > All of the test/sample jobs fail with either out of memory or The reduce > copier failed. > Anyone have any tips on this? > > Our non ec2 installations seem to work just fine. > -- > Jason Venner &

0.15.3 on ec2 - using manual hadoop install no to ec2 magic: The reduce copier failed

2008-02-25 Thread Jason Venner
All of the test/sample jobs fail with either out of memory or The reduce copier failed. Anyone have any tips on this? Our non ec2 installations seem to work just fine. -- Jason Venner Attributor - Publish with Confidence <http://www.attributor.com/> Attributor is hiring Hadoop Wra