[
https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752785#comment-13752785
]
Cheolsoo Park commented on PIG-3419:
------------------------------------
[~rohini],
{quote}
It makes lot of changes to the Exceptions thrown, removes public methods etc
and that might cause backward incompatibility during runtime with code compiled
with previous versions of pig.
{quote}
I am listing all the backward incompatible changes made to public API.
Hopefully, this helps us estimate the impact.
# PigServer constructor
{code}
- public PigServer(String execTypeString) throws ExecException, IOException {
- this(ExecType.fromString(execTypeString));
+ public PigServer(String execTypeString) throws PigException {
+ this(addExecTypeProperty(PropertiesUtil.loadDefaultProperties(),
execTypeString));
+ }
{code}
We can revert PigException back to EE and IOE for this constructor (and other
new constructors). Not hard to fix.
# PigServer.explain()
{code}
public void explain(String alias,
String format,
boolean verbose,
boolean markAsExecute,
PrintStream lps,
- PrintStream pps,
- PrintStream eps) throws IOException {
+ PrintStream eps,
+ File dir,
+ String suffix) throws IOException {
{code}
The method signature changed. Although it's possible that someone uses this
method in their applications, there is another method that wraps this one (i.e.
{{public void explain(String alias, PrintStream stream)}}), and that one is
more likely to be used.
# Name changes of several public classes:
** JobStats to MRJobStats
** PigStatsUtil to MRPigStatsUtil
** ScriptState to MRPScriptState
** HExecutionEngine to MRExecutionEngine
# PigContext.getExecutionEngine()
{code}
- public HExecutionEngine getExecutionEngine() {
+ public ExecutionEngine getExecutionEngine() {
{code}
Result of the class name change.
# SimplePigStats class is moved from {{org.apache.pig.tools.pigstats}} to
{{org.apache.pig.tools.pigstats.mapreduce}}.
# Launcher class is moved from
{{org.apache.pig.backend.hadoop.executionengine.mapReduceLayer}} to
{{org.apache.pig.backend.hadoop.executionengine}}.
# JobControlCompiler constructor
{code}
- public JobControlCompiler(PigContext pigContext, Configuration conf)
throws IOException {
+ public JobControlCompiler(PigContext pigContext, Configuration conf) {
{code}
The IOE was redundant in the first place. So we should remove it.
Here is my estimation:
# Major - But we can fix it.
# Minor - Unlikely used in user code.
# Minor - Unlikely used in user code.
# Minor - Unlikely used in user code.
# Minor - Unlikely used in user code.
# Minor - Unlikely used in user code.
# Minor - Unlikely used in user code.
As long as we fix #1, I think we can go ahead commit the patch to trunk. What
do you think?
> Pluggable Execution Engine
> ---------------------------
>
> Key: PIG-3419
> URL: https://issues.apache.org/jira/browse/PIG-3419
> Project: Pig
> Issue Type: New Feature
> Affects Versions: 0.12
> Reporter: Achal Soni
> Assignee: Achal Soni
> Priority: Minor
> Attachments: execengine.patch, mapreduce_execengine.patch,
> stats_scriptstate.patch, test_failures.txt, test_suite.patch,
> updated-8-22-2013-exec-engine.patch, updated-8-23-2013-exec-engine.patch,
> updated-8-27-2013-exec-engine.patch, updated-8-28-2013-exec-engine.patch
>
>
> In an effort to adapt Pig to work using Apache Tez
> (https://issues.apache.org/jira/browse/TEZ), I made some changes to allow for
> a cleaner ExecutionEngine abstraction than existed before. The changes are
> not that major as Pig was already relatively abstracted out between the
> frontend and backend. The changes in the attached commit are essentially the
> barebones changes -- I tried to not change the structure of Pig's different
> components too much. I think it will be interesting to see in the future how
> we can refactor more areas of Pig to really honor this abstraction between
> the frontend and backend.
> Some of the changes was to reinstate an ExecutionEngine interface to tie
> together the front end and backend, and making the changes in Pig to delegate
> to the EE when necessary, and creating an MRExecutionEngine that implements
> this interface. Other work included changing ExecType to cycle through the
> ExecutionEngines on the classpath and select the appropriate one (this is
> done using Java ServiceLoader, exactly how MapReduce does for choosing the
> framework to use between local and distributed mode). Also I tried to make
> ScriptState, JobStats, and PigStats as abstract as possible in its current
> state. I think in the future some work will need to be done here to perhaps
> re-evaluate the usage of ScriptState and the responsibilities of the
> different statistics classes. I haven't touched the PPNL, but I think more
> abstraction is needed here, perhaps in a separate patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
