On Fri, Oct 7, 2011 at 9:00 AM, Dmitriy Lyubimov <[email protected]> wrote:
> ... > AbstractJob as it currently exists is more a Tool than an individual step > in > a pipeline, perhaps historically driven by a fact that most Mahout > pipelines > are one step generic job agnstic of MR specific parameters passed to them, > so this needs some model design work before approach is truly applicable to > any given pipeline. AbstractJob is also very Hadoop centric. It doesn't apply at all to non-hadoop variants. > Pipeline execution plan also may be not so trivial which > tools such as oozie exist. Because doing it with utter flexibility is > expensive, and because individual steps are implementation detail not > exposed in API, I don't see big urgency in forcing any abstract > functionality in an internal pipeline execution for as long as they merge > managed parameters with unmanaged configuration passed in via Tool like > base. > Pipeline execution is something that can be handled outside of Mahout.
