RE: Introducing Cloud MapReduce

2009-11-30 Thread huan.liu
From: Bruce Snyder [mailto:bruce.sny...@gmail.com] Sent: Friday, November 27, 2009 1:19 PM To: general@hadoop.apache.org Subject: Re: Introducing Cloud MapReduce On Fri, Nov 27, 2009 at 11:34 AM, Tim Robertson timrobertson...@gmail.com wrote: Hi Bruce, Interesting stuff. It looks

RE: what is the major difference between Hadoop and cloudMapReduce?

2009-11-30 Thread huan.liu
Todd, We do not keep all values for a key in memory. Instead, we only keep the partial reduce result in memory, but throw away the value as soon as it is used. The point you raised is still very valid if the reduce state maintained per key is large, which I hope is a very rare case. If you

Re: what is the major difference between Hadoop and cloudMapReduce?

2009-11-30 Thread Todd Lipcon
On Mon, Nov 30, 2009 at 1:48 AM, huan@accenture.com wrote: Todd, We do not keep all values for a key in memory. Instead, we only keep the partial reduce result in memory, but throw away the value as soon as it is used. The point you raised is still very valid if the reduce state