[
https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767220#comment-13767220
]
Dmitriy V. Ryaboy commented on PIG-3419:
----------------------------------------
This is not just for Tez. The point is to enable POC work (in branches, forks,
etc) and not have each such attempt redo all the work in this ticket. It's the
same reason we provide things like pluggable LoadFuncs to let people work on
things they want to load we didn't think of loading.
We should certainly work to stabilize 0.12 and fix issues like PIG-3457
> Pluggable Execution Engine
> ---------------------------
>
> Key: PIG-3419
> URL: https://issues.apache.org/jira/browse/PIG-3419
> Project: Pig
> Issue Type: New Feature
> Affects Versions: 0.12
> Reporter: Achal Soni
> Assignee: Achal Soni
> Priority: Minor
> Fix For: 0.12
>
> Attachments: execengine.patch, mapreduce_execengine.patch,
> stats_scriptstate.patch, test_failures.txt, test_suite.patch,
> updated-8-22-2013-exec-engine.patch, updated-8-23-2013-exec-engine.patch,
> updated-8-27-2013-exec-engine.patch, updated-8-28-2013-exec-engine.patch,
> updated-8-29-2013-exec-engine.patch
>
>
> In an effort to adapt Pig to work using Apache Tez
> (https://issues.apache.org/jira/browse/TEZ), I made some changes to allow for
> a cleaner ExecutionEngine abstraction than existed before. The changes are
> not that major as Pig was already relatively abstracted out between the
> frontend and backend. The changes in the attached commit are essentially the
> barebones changes -- I tried to not change the structure of Pig's different
> components too much. I think it will be interesting to see in the future how
> we can refactor more areas of Pig to really honor this abstraction between
> the frontend and backend.
> Some of the changes was to reinstate an ExecutionEngine interface to tie
> together the front end and backend, and making the changes in Pig to delegate
> to the EE when necessary, and creating an MRExecutionEngine that implements
> this interface. Other work included changing ExecType to cycle through the
> ExecutionEngines on the classpath and select the appropriate one (this is
> done using Java ServiceLoader, exactly how MapReduce does for choosing the
> framework to use between local and distributed mode). Also I tried to make
> ScriptState, JobStats, and PigStats as abstract as possible in its current
> state. I think in the future some work will need to be done here to perhaps
> re-evaluate the usage of ScriptState and the responsibilities of the
> different statistics classes. I haven't touched the PPNL, but I think more
> abstraction is needed here, perhaps in a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
