[
https://issues.apache.org/jira/browse/MAPREDUCE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427432#comment-13427432
]
Chris A. Mattmann commented on MAPREDUCE-4495:
----------------------------------------------
Hi Josh:
bq. 1) Chris, it's the "Going from there" part of the incubator process that
involves the overhead.
I don't think I ever disputed that part. I simply stated that getting started
with an Incubator project was not that onerous. As for your metrics above, they
are your own experience. I've been involved in quite a few more Incubator
projects than you have, having contributed to the Foundation since 2005, so
simply throwing out a recent data point doesn't make it true generally.
bq. Setting up a project infrastructure takes time. You need to file a lot of
INFRA tickets, and those typically take about a week to get resolved. I'm not
hating on anybody here, the INFRA guys are doing yeoman's work and we cannot do
enough nice for things for them in return, but let's not pretend that there
isn't overhead involved in having someone setup a repo for you, create a JIRA
project, setup Jenkins, and set up CMS.
And in turn, let's not pretend that having an Incubator project *requires* all
of the things you mentioned above. Having an Incubator project at the ASF means
that you have a project that was VOTEd on and accepted by the Incubator Project
Management Committee (IPMC), nothing more nothing less. You then have N
months/years/whatever to sort out the rest of the stuff, which involves
creating infrastructure, setting up mailing lists, yadda yadda.
bq. The ASF has good reasons for not creating umbrella projects, as have been
outlined earlier in the thread. But let's not pretend that there isn't overhead
to creating a new project.
Again, creating a new project != having finished all of the items that you
outlined. *Graduating from the Incubator* == having finished all of the items
that you outlined.
> Workflow Application Master in YARN
> -----------------------------------
>
> Key: MAPREDUCE-4495
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4495
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.0.0-alpha
> Reporter: Bo Wang
> Assignee: Bo Wang
>
> It is useful to have a workflow application master, which will be capable of
> running a DAG of jobs. The workflow client submits a DAG request to the AM
> and then the AM will manage the life cycle of this application in terms of
> requesting the needed resources from the RM, and starting, monitoring and
> retrying the application's individual tasks.
> Compared to running Oozie with the current MapReduce Application Master,
> these are some of the advantages:
> - Less number of consumed resources, since only one application master will
> be spawned for the whole workflow.
> - Reuse of resources, since the same resources can be used by multiple
> consecutive jobs in the workflow (no need to request/wait for resources for
> every individual job from the central RM).
> - More optimization opportunities in terms of collective resource requests.
> - Optimization opportunities in terms of rewriting and composing jobs in the
> workflow (e.g. pushing down Mappers).
> - This Application Master can be reused/extended by higher systems like Pig
> and hive to provide an optimized way of running their workflows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
