Re: Is continuous map reduce supported

Thanks Jeff. Those are all valuable links. It seems there are quite a few people out there working on incremental MapReduce. 2010/9/1 Jeff Hammerbacher > Hey Stephen, > > There have been several proposals for implementing such a feature. See > https://issues.apache.org/jira/browse/MAPREDUCE-121

Re: Is continuous map reduce supported

> That isn't true. We are actively adding new features. However, there > is certainly a focus on doing MapReduce well rather than trying to > implement all potential distributed computation paradigms. I suspect > that the right solution is doing two levels like Mesos: > > http://www.eecs.berkeley.e

Re: Is continuous map reduce supported

Try this: http://hadoop.apache.org/common/docs/r0.20.0/api/org/apache/hadoop/mapred/lib/ChainReducer.html On Wed, Sep 1, 2010 at 8:33 PM, Lance Norskog wrote: > Dead link: > > > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/ChainReducer.html > > On Wed, Sep 1, 201

Re: Is continuous map reduce supported

Dead link: http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/ChainReducer.html On Wed, Sep 1, 2010 at 9:56 AM, Owen O'Malley wrote: > On Wed, Sep 1, 2010 at 3:35 AM, Jeff Hammerbacher wrote: >> >> There have been several proposals for implementing such a feature. > >

Re: Is continuous map reduce supported

On Wed, Sep 1, 2010 at 3:35 AM, Jeff Hammerbacher wrote: > > There have been several proposals for implementing such a feature. Thanks for the great breakdown of the relevant work. > As for actually being implemented in Hadoop MapReduce: the Apache project > seems to have settled in to focus on

Re: Is continuous map reduce supported

Hey Stephen, There have been several proposals for implementing such a feature. See https://issues.apache.org/jira/browse/MAPREDUCE-1211 for an implementation from Berkeley, now maintained at http://code.google.com/p/hop. The paper at https://www.ideals.illinois.edu/handle/2142/14819 describes a s

Re: Is continuous map reduce supported

There's Chain-Mapping and Chain-Reducing available. With good docs: http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/ChainReducer.html However, something as simple as Twister (which has iterative mapreduces based on a while-like condition loop) isn't directly available

Is continuous map reduce supported

Hello, I have not used Hadoop but am researching it for an analytics project. I would like to know if Hadoop supports continuous or incremental map reduce functionality. If not, are there any plans to add it? Thanks, Stephen