[ https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997599#comment-12997599 ]
Jeff Hammerbacher commented on MAPREDUCE-279: --------------------------------------------- Hey Arun, As long as that evolution is happening in a branch, that seems totally reasonable to me. When it comes time to migrate the code into trunk, I hope for the same end state as Matei: I think the resource management system should be a separate project from MapReduce so that each system can evolve and release separately. When we have more clients than just MapReduce for the resource manager, we'll want those new clients to evolve as separate projects rather than all living under the Apache Hadoop umbrella. Now seems like an excellent time to facilitate that end state. More specifically, in an ideal world, we'd have four separate projects here: Common (probably folded into Guava or Apache Commons), HDFS, Yahoo! Cluster Manager (Resource Manager + Node Manager), and MapReduce (the ApplicationMaster for MapReduce, I guess). Then, if someone wanted to write Pregel to run against the Cluster Manager, they could implement their own ApplicationMaster in a separate project. Similarly, if someone wanted to run MapReduce against a different cluster manager, that would be simple. More practically, we have the opportunity to get the Cluster Manager project started up as a separate ASF project once it has gestated in a branch here for a bit. Are there any technical barriers to making that happen? I'm a huge fan of this work, and having watched a number of ASF projects evolve over the past several years, I suspect that a small, focused project dedicated to cluster resource management will have the best chance of moving quickly. Thanks, Jeff > Map-Reduce 2.0 > -------------- > > Key: MAPREDUCE-279 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-279 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobtracker, tasktracker > Reporter: Arun C Murthy > Assignee: Arun C Murthy > Fix For: 0.23.0 > > > Re-factor MapReduce into a generic resource scheduler and a per-job, > user-defined component that manages the application execution. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira