----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16928/#review32245 -----------------------------------------------------------
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java <https://reviews.apache.org/r/16928/#comment60975> Will remove this before commit. - Aniket Mokashi On Jan. 16, 2014, 10:04 p.m., Aniket Mokashi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/16928/ > ----------------------------------------------------------- > > (Updated Jan. 16, 2014, 10:04 p.m.) > > > Review request for pig, Cheolsoo Park, Daniel Dai, Dmitriy Ryaboy, and Julien > Le Dem. > > > Bugs: PIG-3463 > https://issues.apache.org/jira/browse/PIG-3463 > > > Repository: pig > > > Description > ------- > > If pig.auto.local.enabled is set, JCC will modify Configuration of all the > jobs with one reducer and input size less than pig.auto.local.input.maxbytes, > so that they are forced to run in local mode. Output of local run is also > written to hdfs. > > > Diffs > ----- > > trunk/src/org/apache/pig/ExecTypeProvider.java 1558572 > trunk/src/org/apache/pig/PigConfiguration.java 1558572 > trunk/src/org/apache/pig/backend/hadoop/datastorage/ConfigurationUtil.java > 1558572 > > trunk/src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java > 1558572 > > trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java > 1558572 > > trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java > 1558572 > > trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceOper.java > 1558572 > > trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigInputFormat.java > 1558572 > trunk/src/org/apache/pig/impl/PigImplConstants.java 1558572 > trunk/src/org/apache/pig/tools/pigstats/EmbeddedPigStats.java 1558572 > trunk/src/org/apache/pig/tools/pigstats/PigStats.java 1558572 > trunk/src/org/apache/pig/tools/pigstats/mapreduce/MRPigStatsUtil.java > 1558572 > trunk/src/org/apache/pig/tools/pigstats/mapreduce/SimplePigStats.java > 1558572 > trunk/test/org/apache/pig/test/TestAutoLocalMode.java PRE-CREATION > > Diff: https://reviews.apache.org/r/16928/diff/ > > > Testing > ------- > > Tried few scenarios with the patch- > Load small data, group all, count - works in local mode. > Load small data, another small data and replicated join - works in local mode. > Load small data and order by key - all 3 jobs work in local mode and . > Load small data and large data for replicated join - first job runs in local > mode, second runs in MR mode. > Load large data and order by key - works in first stages in local mode and > last stage in MR mode. > > > Thanks, > > Aniket Mokashi > >
