[
https://issues.apache.org/jira/browse/PIG-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052306#comment-13052306
]
Daniel Dai commented on PIG-1916:
---------------------------------
Here is my review comments:
1. Parser change is good. LOCross is correct in logical plan
2. Logical plan is still good after the optimizer
3. Physical plan is not good. we shall have POCross in physical plan. The
current physical plan shows "Global Rearrange" + "Local Rearrange" + "GFCross",
which is good for top level cross, but not nested cross
Here is suggested change:
1. Add a flag inner to LOCross
2. Change LogToPhyTranslationVisitor, if top level cross, keep the old logic;
if nested cross, convert LOCross to POCross
> Nested cross
> ------------
>
> Key: PIG-1916
> URL: https://issues.apache.org/jira/browse/PIG-1916
> Project: Pig
> Issue Type: New Feature
> Components: impl
> Reporter: Daniel Dai
> Labels: gsoc2011
> Fix For: 0.10
>
> Attachments: PIG-1916_1.patch
>
>
> It is useful to have cross inside foreach nested statement. One typical use
> case for nested foreach is after cogroup two relations, we want to flatten
> the records of the same key, and do some processing. This is naturally to be
> achieved by cross. Eg:
> {code}
> C = cogroup user by uid, session by uid;
> D = foreach C {
> crossed = cross user, session; -- To flatten two input bags
> filtered = filter crossed by user::region == session::region;
> result = foreach crossed generate processSession(user::age, user::gender,
> session::ip); --Nested foreach Jira: PIG-1631
> generate result;
> }
> {code}
> If we don't have cross, user have to write a UDF process the bag user,
> session. It is much harder than a UDF process flattened tuples. This is
> especially true when we have nested foreach statement(PIG-1631).
> This is a candidate project for Google summer of code 2011. More information
> about the program can be found at http://wiki.apache.org/pig/GSoc2011
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
