On Feb 14, 2011, at 12:55 PM, arv...@cloudera.com wrote:
>
> - Support for different backends for Pig (MR, Piranha, Local, Oozie)
> - Execution engine that can generate plans specific to the underlying
> architecture and allow controlling routines to
> rewrite/re-optimize the plan
> mid-execution.
+1 for Oozie backend.
> - Allow execution to resume from where it left off after due to runtime
> failure.
> - May be done by allowing Oozie as a backend where the plan is
> converted into an Oozie workflow.
> - Alternatively Pig could delegate blocks of execution to Oozie.
See above.
This would be very beneficial for oozie (even more than pig), because no one
should be made to program in xml ! The current workflow specification is a
cruelty !
Pig has constructs to invoke hdfs commands, arbitrary jars, mapreduce codes
already. If it has a hive execution mode, e.g.
A = hive("select xyz from etc...");
all of these could be handed off to oozie (I believe alejandro has a hive
action for oozie already). Then there would be no "hive vs pig", instead pigs
will really eat anything.
- milind
---
Milind Bhandarkar
mbhandar...@linkedin.com
