Hi again
Browsing the source code (Merger.class) I see that merger actually
call reporter.progress() so shouldn't this make the task be reported
as still working?
Regards
David Alves
On Nov 20, 2008, at 6:29 PM, David Alves wrote:
Hi all
I have a big map task that takes a long time to complete and
produces a lot of information.
The merge part makes the task timeout (you can see this in the end
of this email, the merge part is aborted after ten minutes, the
default time).
I've increased the mapred.tasks.timeout property to 30 min instead
of 10, in hadoop-site.xml as follows:
<property> <name>mapred.task.timeout</name> <value>1800000</value>
</property>
But stiil the task fails with:
Task attempt_200811201704_0001_m_000000_0 failed to report status
for 603 seconds. Killing!
Is there any other property I should change?.
Regards
David Alves
17:35:40,697 INFO [MapTask] Starting flush of map output
17:35:40,697 INFO [MapTask] bufstart = 17472260; bufend = 51354916;
bufvoid = 99614720
17:35:40,697 INFO [MapTask] kvstart = 39119; kvend = 39468; length
= 327680
17:35:40,950 INFO [MapTask] Index: (0, 33884416, 33884416)
17:35:40,950 INFO [MapTask] Finished spill 152
17:35:45,333 INFO [Merger] Merging 153 sorted segments
17:35:46,337 INFO [Merger] Merging 9 intermediate segments out of a
total of 153
17:36:16,849 INFO [Merger] Merging 10 intermediate segments out of
a total of 145
17:36:47,615 INFO [Merger] Merging 10 intermediate segments out of
a total of 136
17:37:21,529 INFO [Merger] Merging 10 intermediate segments out of
a total of 127
17:37:59,883 INFO [Merger] Merging 10 intermediate segments out of
a total of 118
17:38:35,370 INFO [Merger] Merging 10 intermediate segments out of
a total of 109
17:39:14,795 INFO [Merger] Merging 10 intermediate segments out of
a total of 100
17:39:51,787 INFO [Merger] Merging 10 intermediate segments out of
a total of 91
17:40:28,721 INFO [Merger] Merging 10 intermediate segments out of
a total of 82
17:41:05,650 INFO [Merger] Merging 10 intermediate segments out of
a total of 73
17:41:43,285 INFO [Merger] Merging 10 intermediate segments out of
a total of 64
17:42:23,531 INFO [Merger] Merging 10 intermediate segments out of
a total of 55
17:43:01,709 INFO [Merger] Merging 10 intermediate segments out of
a total of 46
17:43:40,209 INFO [Merger] Merging 10 intermediate segments out of
a total of 37
17:44:20,707 INFO [Merger] Merging 10 intermediate segments out of
a total of 28
17:44:57,700 INFO [Merger] Merging 10 intermediate segments out of
a total of 19
