[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1321: Attachment: jira-1321-3.patch Repost the pre-condition: 1. two consecutive foreach statements. 2. the second foreach statement is a simple inner plan in which the ognly statement is a GENERATE statement. In other words, the second foreach statement must be something like "FOREACH A GENERATE " 3. The first foreach statement cannot contain flatten due to its complexity 4. No 1st foreach output is referred more than once in second foreach, eg: "B = foreach ; C = foreach B generate $0, $1, $0" will not be merged. The reason if we merge, $0 will be calculated twice, which defeat the benefit of merging. All tests pass. test-patch result: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: jira-1321-2.patch, jira-1321-3.patch, pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1321: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Patch committed. Thanks Xuefu! > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: jira-1321-2.patch, jira-1321-3.patch, pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated PIG-1321: - Status: Patch Available (was: Open) > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: jira-1321-2.patch, pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated PIG-1321: - Attachment: jira-1321-2.patch Regenerate the patch to fix some test failures as well as rebasing with trunk's latest code changes. > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: jira-1321-2.patch, pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated PIG-1321: - Status: Open (was: Patch Available) > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: jira-1321-2.patch, pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1321: Release Note: This rule allows to merge togther two foreach statements if the following preconditions are met: - foreach statements are consecutive - the second foreach is not nested - the first foreach statement does not contain flatten Example: (1) Original code: A = load 'file.txt' as (a, b, c); B = foreach A generate a+b as u, c-b as v; C = foreach B generate $0+5, v; . (2) Optimized code: A = load 'file.txt' as (a, b, c); C = foreach A generate a+b+5, c-b; .. > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated PIG-1321: - Attachment: pig-1321.patch > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated PIG-1321: - Status: Patch Available (was: Open) > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > Attachments: pig-1321.patch > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1321: Fix Version/s: 0.8.0 > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Xuefu Zhang > Fix For: 0.8.0 > > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1321: Issue Type: Sub-task (was: New Feature) Parent: PIG-1319 > Logical Optimizer: Merge cascading foreach > -- > > Key: PIG-1321 > URL: https://issues.apache.org/jira/browse/PIG-1321 > Project: Pig > Issue Type: Sub-task > Components: impl >Affects Versions: 0.7.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > > We can merge consecutive foreach statement. > Eg: > b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1; > c = foreach b generate b0#'kk1', b0#'kk2', b1, a1; > => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
