On Feb 14, 2011, at 1:34 PM, Arun C Murthy wrote:
As the final installment in this process, I've started a discussion on
us contributing a re-factor of Map-Reduce in
https://issues.apache.org/jira/browse/MAPREDUCE-279
.
Hi Folks,
We wanted to share our thoughts around the co-development of the
NextGen MapReduce branch (Jira MR-279), maintaining the branch-0.20-
security and merging the work on the security branch with trunk.
We've concluded that it does not make sense for us to port a very
small subset of the work from the branch-0.20-security to the Hadoop
mainline. The JIRAs we don't plan to port all effect areas of the
mainline that are going to be replaced by work in the NextGen
MapReduce branch (http://svn.apache.org/viewvc/hadoop/mapreduce/branches/MR-279/
).
We've been working on the NextGen MapReduce branch (MAPREDUCE-279)
within Apache for a while now and are excited about it's progress. We
think that this branch will be a huge improvement in scalability,
performance and functionality. We are now confident that we can get
it ready for release in in the next few months. We believe that the
next major release of Apache Hadoop we will test at Yahoo will include
the work in this branch and we are committed to merging the NextGen
branch into the mainline after the PMC approves the merge.
Meanwhile, we have continued to find and fix bugs on branch-0.20-
security and have been working to port that work into the Hadoop
mainline. Most of this work is done and we've also brought all the
patches in from our github branch into apache subversion, so that it
is easy for everyone to see the work remaining. What we've found is
that some of the work in branch-0.20-security is in code sections that
have been completely replaced / refactored in the NextGen MapReduce
branch. Since we are committed to the NextGen branch, we don't think
there is any upside in porting this code into portions of mainline we
expect to discard. All of these JIRAs will be fixed in the NextGen
MapReduce branch and through there ultimately in trunk (assuming the
PMC approves the merge).
So at this point it is our intent to not port the JIRAs listed above
to trunk, but to wait until we merge NextGen into trunk to resolve
these issues there. If you are interested in seeing these issues
ported to mainline, let us know. We are happy to help review your
patches and explain context to anyone who is interested in doing this
work.
Arun and Eric
