[ https://issues.apache.org/jira/browse/PIG-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070800#comment-15070800 ]
Pallavi Rao commented on PIG-4766: ---------------------------------- With this patch, all tests in TestCombiner pass. Couple of call outs: 1. The patch modifies PORelationToExprProject (changes are in-line with what is there on trunk). 2. Makes SelfSpillBag$MemoryLimits serializable 3. Modifies TestCombiner as follows. Couldn't think of a better alternative as the spark optimizer does not generate a separate combine plan. {code} if (pigServer.getPigContext().getExecType().name().equalsIgnoreCase("spark")) { combinerFound = baos.toString().contains("Reduce By"); } else { combinerFound = baos.toString().matches("(?si).*combine plan.*"); } {code} > Ensure GroupBy is optimized for all algebraic Operations > -------------------------------------------------------- > > Key: PIG-4766 > URL: https://issues.apache.org/jira/browse/PIG-4766 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: Pallavi Rao > Assignee: Pallavi Rao > Labels: spork > Fix For: spark-branch > > Attachments: PIG-4766.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)