+4.01. This is a terrific idea. On Fri, Apr 1, 2011 at 1:19 AM, Aaron T. Myers <a...@cloudera.com> wrote:
> Hello Hadoop Community, > > Given the tremendous positive feedback we've all had regarding the HDFS, > MapReduce, and Common project split, I'd like to propose we take the next > step and further separate the existing projects. > > I propose we begin by splitting the MapReduce project into separate "Map" > and "Reduce" sub-projects. This will provide us the opportunity to tease > out > the complex interdependencies between "map" and "reduce" that exist today, > to encourage us to write more modular and isolated code, which should speed > releases. This will also aid our users who exclusively run map-only or > reduce-only jobs. These are important use-cases, and so should be given > high > priority. > > Given that these two portions of the existing MapReduce project share a > great deal of code, we will likely need to release these two new projects > concurrently at first, but the eventual goal should certainly be to be able > to release "Map" and "Reduce" independently. This seems intuitive to me, > given the remarkable recent advancements in the academic community > regarding > "reduce," while the research coming out of the "map" academics has largely > stagnated of late. > > If this proposal is accepted, and it has the success I think it will, then > we should strongly consider splitting the other two projects as well. My > gut > instinct is that we should split "HDFS" into "HD" and "FS" sub-projects, > and > simply rename the "Common" project to "C'Mon." We can think about the > details of what exactly these project splits mean later. > > Please let me know what you think. > > Best, > Aaron > -- Todd Lipcon Software Engineer, Cloudera
