> On April 10, 2014, 8:59 p.m., Daniel Dai wrote: > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/optimizers/NoopFilterRemover.java, > > line 53 > > <https://reviews.apache.org/r/20148/diff/2/?file=553898#file553898line53> > > > > Do we need to limit to splitter? Remove constant filter seems always > > applicable.
Don't think constant filter is added anywhere else. So the check will make it efficient. > On April 10, 2014, 8:59 p.m., Daniel Dai wrote: > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/PigServer.java, > > line 239 > > <https://reviews.apache.org/r/20148/diff/2/?file=553884#file553884line239> > > > > Let's take the PigServer/GruntParser change out to a separate ticket. > > This is trunk code and need to track separately. Will create a separate jira for that and remove it from here. - Rohini ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20148/#review40081 ----------------------------------------------------------- On April 10, 2014, 9:46 p.m., Rohini Palaniswamy wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20148/ > ----------------------------------------------------------- > > (Updated April 10, 2014, 9:46 p.m.) > > > Review request for pig, Cheolsoo Park and Daniel Dai. > > > Bugs: PIG-3842 > https://issues.apache.org/jira/browse/PIG-3842 > > > Repository: pig > > > Description > ------- > > Issues fixed: > > Multiquery issues: > NoOpFilter has to be removed > Wrong input output counters > Nested splits are not merged > multiquery off mode launches too many jobs as it is not processed in batch. > Also was not able to do explain in multi-query off mode. > > Other: > Pig on tez job hangs when AM has a failure > TezDAG is not required anymore as DAG now has getCredentials method. > Ensured that delegation token is fetched only once for a NN by having the > same DAG Credentials object referenced everywhere. > > > Diffs > ----- > > http://svn.apache.org/repos/asf/pig/branches/tez/ivy/libraries.properties > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/PigServer.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/MultiQueryOptimizerTez.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POStoreTez.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDAG.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControl.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControlCompiler.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperPlan.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperator.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezSessionManager.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/optimizers/NoopFilterRemover.java > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/grunt/GruntParser.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/mapreduce/MRJobStats.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/tez/TezStats.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/tools/pigstats/tez/TezTaskStats.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestSecondarySort.java > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC20.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC21.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC6.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC7.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC8.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC9.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-1-OPTOFF.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-1.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-2-OPTOFF.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-2.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-3-OPTOFF.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-3.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-4-OPTOFF.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-4.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-5-OPTOFF.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-MQ-5.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Rank-1.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Rank-2.gld > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Union-2.gld > 1585314 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/tez/TestTezCompiler.java > 1585314 > > Diff: https://reviews.apache.org/r/20148/diff/ > > > Testing > ------- > > Multiquery e2e tests and Unit tests pass except > TestCombiner.testMultiCombinerUse() which hangs. Issue > (https://issues.apache.org/jira/browse/TEZ-1040) with tez-0.4.0-incubating. > > > Thanks, > > Rohini Palaniswamy > >
