[
https://issues.apache.org/jira/browse/HADOOP-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701557#action_12701557
]
Shevek commented on HADOOP-2491:
--------------------------------
I appreciate that I am a new contributor to Hadoop, but I am very interested in
running MR tasks on Hadoop which are not as strictly governed by the current MR
workflow as is the current implementation. I am currently reading the source
code to Hadoop, I'm slightly stuck in the gap between what appears to be a
version 2.0 implementation emerging from the works, but even so, I am very
interested in contributing towards the effort suggested by this ticket. I have
spent the last couple of years writing high-level compilers for MR-like
languages, and hope this expertise can be of use.
> generalize the TT / JT servers to handle more generic tasks
> -----------------------------------------------------------
>
> Key: HADOOP-2491
> URL: https://issues.apache.org/jira/browse/HADOOP-2491
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: eric baldeschwieler
>
> We've been discussing a proposal to generalize the TT / JT servers to handle
> more generic tasks and move job specific work out of the job tracker and into
> client code so the whole system is both much more general and has more
> coherent layering. The result would look more like condor/pbs like systems
> (or presumably borg) with map-reduce as a user job.
> Such a system would allow the current map-reduce code to coexist with other
> work-queuing libraries or maybe even persistent services on the same Hadoop
> cluster, although that would be a stretch goal. We'll kick off a thread with
> some documents soon.
> Our primary goal in going this way would be to get better utilization out of
> map-reduce clusters and support a richer scheduling model. The ability to
> support alternative job frameworks would just be gravy!
> ----
> Putting this in as a place holder. Hope to get folks talking about this to
> post some more detail.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
