----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17681/#review33863 -----------------------------------------------------------
Ship it! Ship It! - Daniel Dai On Feb. 4, 2014, 10:08 p.m., Rohini Palaniswamy wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17681/ > ----------------------------------------------------------- > > (Updated Feb. 4, 2014, 10:08 p.m.) > > > Review request for pig, Cheolsoo Park and Daniel Dai. > > > Bugs: PIG-3742 > https://issues.apache.org/jira/browse/PIG-3742 > > > Repository: pig > > > Description > ------- > > Changes made: > 1) Converted the relevant MR settings to equivalent Tez settings and set them > on AM, Vertex and Edge. > 2) Moved the util and helper classes (SecurityHelper and TezCompilerUtil) to > a util package. Does not show up cleanly in review board. Will be doing a svn > mv while committing. > 3) Fixed a issue with 1-1 edge in orderby while running pigmix where > parallelism was not reflected in the second edge when the parallelism of > first vertex changed after input split calculation. Also made > POIdentityOutTez work with shuffle input as well when trying to test > performance with 1-1 ege or shuffle edge with round robin partitioner. > Shuffle edge with round robin partitioner or hash partitioner was very bad > compared to MR. Even with 1-1 edge, performance is bad for L10.pig which > orders by multiple columns. Still need to work on order by performance. > Hoping unsorted shuffle with TEZ-661 might make it better. > > > Diffs > ----- > > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POIdentityInOutTez.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/SecurityHelper.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompilerUtil.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperator.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezSessionManager.java > 1563492 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/util/MRToTezHelper.java > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/util/SecurityHelper.java > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezCompilerUtil.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/17681/diff/ > > > Testing > ------- > > Unit and tez.conf e2e tests pass. > > > Thanks, > > Rohini Palaniswamy > >
