On Fri, Aug 03, 2007 at 12:17:37PM -0700, Joydeep Sen Sarma wrote:
>I have a fairly simple job with a map, a local combiner and a reduce.
>The combiner and the reduce do the equivalent of a group_concat (mysql).
>
>
>I have horrible performance in the reduce stage:
>- the map jobs are done
>- all
ch in the first place).
-Original Message-
From: Stu Hood [mailto:[EMAIL PROTECTED]
Sent: Tuesday, August 07, 2007 2:58 PM
To: hadoop-user@lucene.apache.org
Subject: RE: extremely slow reduce jobs
Hi,
I have noticed the same thing. My cluster is only 2 machines, but I've
noticed
al Message-
From: Stu Hood [mailto:[EMAIL PROTECTED]
Sent: Tuesday, August 07, 2007 2:58 PM
To: hadoop-user@lucene.apache.org
Subject: RE: extremely slow reduce jobs
Hi,
I have noticed the same thing. My cluster is only 2 machines, but I've
noticed "Scheduled 0 of 1 known outputs (1 sl
17 pm
To: hadoop-user@lucene.apache.org
Subject: extremely slow reduce jobs
I have a fairly simple job with a map, a local combiner and a reduce.
The combiner and the reduce do the equivalent of a group_concat (mysql).
I have horrible performance in the reduce stage:
- the map jobs are done
- all the r
I met the save problem:
-but the copy rate is only (0.1MBps)
-10MBps bandwidth between using scp
May be the bandwidth affect the copy rate.
Can any one help us.
2007/8/4, Joydeep Sen Sarma <[EMAIL PROTECTED]>:
>
> I have a fairly simple job with a map, a local combiner and a reduce.
> The combi
I have a fairly simple job with a map, a local combiner and a reduce.
The combiner and the reduce do the equivalent of a group_concat (mysql).
I have horrible performance in the reduce stage:
- the map jobs are done
- all the reduce jobs claim they are copying data - but the copy rate is
abysma
