-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20148/
-----------------------------------------------------------

(Updated April 10, 2014, 11:43 p.m.)


Review request for pig, Cheolsoo Park and Daniel Dai.


Changes
-------

Addressed review comments from Cheolsoo and Daniel. PigServer/GruntParser 
changes moved to PIG-3882.

Cheolsoo,
   I need to create a separate jira/ patch for getting the MR changes into 
trunk right?


Bugs: PIG-3842
    https://issues.apache.org/jira/browse/PIG-3842


Repository: pig


Description
-------

Issues fixed:

Multiquery issues:
NoOpFilter has to be removed
Wrong input output counters
Nested splits are not merged
multiquery off mode launches too many jobs as it is not processed in batch. 
Also was not able to do explain in multi-query off mode.

Other:
Pig on tez job hangs when AM has a failure
TezDAG is not required anymore as DAG now has getCredentials method.
Ensured that delegation token is fetched only once for a NN by having the same 
DAG Credentials object referenced everywhere.


Diffs (updated)
-----

  http://svn.apache.org/repos/asf/pig/branches/tez/ivy/libraries.properties 
1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/MultiQueryOptimizerTez.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POStoreTez.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDAG.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControl.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControlCompiler.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperPlan.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperator.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezSessionManager.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/optimizers/NoopFilterRemover.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/JobStats.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/PigStatsUtil.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/tez/TezStats.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/tez/TezTaskStats.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestSecondarySort.java
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC20.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC21.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC6.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC7.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC8.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC9.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-1-OPTOFF.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-1.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-2-OPTOFF.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-2.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-3-OPTOFF.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-3.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-4-OPTOFF.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-4.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-5-OPTOFF.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-5.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Rank-1.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Rank-2.gld
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Union-2.gld
 1585314 
  
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/tez/TestTezCompiler.java
 1585314 

Diff: https://reviews.apache.org/r/20148/diff/


Testing
-------

Multiquery e2e tests and Unit tests pass except 
TestCombiner.testMultiCombinerUse() which hangs. Issue 
(https://issues.apache.org/jira/browse/TEZ-1040) with tez-0.4.0-incubating. 


Thanks,

Rohini Palaniswamy

Reply via email to