-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14601/
-----------------------------------------------------------
Review request for pig, Daniel Dai, Mark Wagner, and Rohini Palaniswamy.
Bugs: PIG-3502
https://issues.apache.org/jira/browse/PIG-3502
Repository: pig-git
Description
-------
Initial implementation of TezLauncher. The patch includes the following changes:
- Refactors Launcher and MapReduceLauncher. Basically, I moved what's
MR-specific to MapReduceLauncher and kept what's common in Launcher.
- Removes kill() method from ExecutionEngine interface. It was used merely by
Launcher, and it didn't seem useful.
- Implements the DAG submission of TezLauncher. Basically, I overrode the
submit() method in TezJob (extension of ControlledJob) that calls
DAGClient.submit().
- Adds a skeleton TezStats class that extends PigStats. This class needs to be
implemented to monitor job status after submission.
- Adds a unit test called TestTezLauncher. For now, all it does is to compile
queries and submit a DAG. More meaningful unit tests will be added after
TezStats is implemented.
- Adds a TezLocalExecType. To start up a YARN mini cluster, I needed non-local
Tez mode. So I created "tez_local" and "tez" modes.
- Whitespace and indentation fixes.
Diffs
-----
build.xml 0439082
src/META-INF/services/org.apache.pig.ExecType e995cd5
src/org/apache/pig/backend/executionengine/ExecutionEngine.java 7d480d7
src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java
b886f9d
src/org/apache/pig/backend/hadoop/executionengine/Launcher.java 77543c2
src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
f3be738
src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecType.java
8315e76
src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java 04ffba0
src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControlCompiler.java
b7d4d98
src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java
ff9e950
src/org/apache/pig/backend/hadoop/executionengine/tez/TezLocalExecType.java
e69de29
src/org/apache/pig/backend/hadoop/executionengine/tez/TezStats.java e69de29
src/org/apache/pig/impl/PigContext.java 225526f
src/org/apache/pig/impl/util/Utils.java 3720c94
test/org/apache/pig/tez/TestTezCompiler.java b9e6b48
test/org/apache/pig/tez/TestTezJobControlCompiler.java f14d49d
test/org/apache/pig/tez/TestTezLauncher.java e69de29
Diff: https://reviews.apache.org/r/14601/diff/
Testing
-------
TestTezLauncher is added.
Thanks,
Cheolsoo Park
