Bharath, This idea is kicking around in academia.. not made into apache yet.. https://issues.apache.org/jira/browse/MAPREDUCE-1211
You can get a working prototype from: http://code.google.com/p/hop/ Ashutosh On Thu, Mar 4, 2010 at 09:06, E. Sammer <e...@lifeless.net> wrote: > On 3/4/10 12:00 PM, bharath v wrote: >> >> Hi , >> >> Can we pipeline the map output directly into reduce phase without >> storing it in the local filesystem (avoiding disk IOs). >> If yes , how to do that ? > > Bharath: > > No, there's no way to avoid going to disk after the mappers. > > -- > Eric Sammer > e...@lifeless.net > http://esammer.blogspot.com >
