, March 24, 2011 5:56 PM
To: common-user@hadoop.apache.org
Subject: Re: Program freezes at Map 99% Reduce 33%
Hi Kevin,
thanks for the suggestion. I think I found the problem, because my code
is a chained map / reduce. In the previous iteration there is a
.lzo_deflate output which is 40 times larger
the tail of your input and
a couple of mappers and then compare that to your program run without
hadoop.
Kevin
-Original Message-
From: Shi Yu [mailto:sh...@uchicago.edu]
Sent: Thursday, March 24, 2011 3:57 PM
To: common-user@hadoop.apache.org
Subject: Re: Program freezes at Map 99% Reduce 33%
ommon-user@hadoop.apache.org
Subject: Re: Program freezes at Map 99% Reduce 33%
Hi Kevin,
thanks for reply. I could hardly imagine an example of incomplete
record. The mapper is very simple, just reading line by line as Strings,
splitting the line by tab, and outputting a Text Pair for sort and
secondary
nd?
Kevin
-Original Message-
From: Shi Yu [mailto:sh...@uchicago.edu]
Sent: Thursday, March 24, 2011 3:02 PM
To: hadoop user
Subject: Program freezes at Map 99% Reduce 33%
I am running a hadoop program processing Tera Byte size data. The code
was test successfully on a small sample (100G) and it worke
PM
To: hadoop user
Subject: Program freezes at Map 99% Reduce 33%
I am running a hadoop program processing Tera Byte size data. The code
was test successfully on a small sample (100G) and it worked. However,
when trying it on the full problem, the program freezes forever at Map
99% Reduce 33%. There
I am running a hadoop program processing Tera Byte size data. The code
was test successfully on a small sample (100G) and it worked. However,
when trying it on the full problem, the program freezes forever at Map
99% Reduce 33%. There is no error, and the size of userlog folder is
clean (<30M)
