On Apr 4, 2007, at 2:57 AM, Mathijs Homminga wrote:
Your reduce task may fail because there are too many values
associated with
some key and it takes more than 10 minutes to process the key.
Please try to
let your reduce task explicitly notify the task tracker that "I am
alive" by
doing re
alive" by
doing report.setStatus(String) once, for example, every 100 or 1000 values.
Hairong
-Original Message-
From: Mathijs Homminga [mailto:[EMAIL PROTECTED]
Sent: Tuesday, April 03, 2007 3:27 AM
To: hadoop-user@lucene.apache.org
Subject: Re: Re-reduce, without re-map
Each r
ple, every 100 or 1000 values.
Hairong
-Original Message-
From: Mathijs Homminga [mailto:[EMAIL PROTECTED]
Sent: Tuesday, April 03, 2007 3:27 AM
To: hadoop-user@lucene.apache.org
Subject: Re: Re-reduce, without re-map
Each reduce task (Nutch indexing job) gets as far as 66%, and then fail
Each reduce task (Nutch indexing job) gets as far as 66%, and then fails with
the following error:
"Task failed to report status for 600 seconds. Killing."
In the end, no reduce task completes successfully.
Besides solves this issue, I was wondering if I could update code and configuration and
Hi Mathijs,
Mathijs Homminga wrote:
We have some troubles with the reduce phase of our job.
Is it possible to re-execute the reduce tasks without the need to do all
map tasks again?
That the MR-framework already does... you don't have to re-execute
the maps for the *failed* reduces. Are
Hi,
We have some troubles with the reduce phase of our job.
Is it possible to re-execute the reduce tasks without the need to do all
map tasks again?
Thanks!
Mathijs Homminga
