[
https://issues.apache.org/jira/browse/DATAFU-12?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196058#comment-16196058
]
Eyal Allweil commented on DATAFU-12:
------------------------------------
[~matterhayes], anyone, what do you think? I wouldn't "waste" our time on
something that can already be done in Pig via Hive, and I'd like to close
jira's that are no longer relevant.
> Implement Lead UDF based on version from SQL
> --------------------------------------------
>
> Key: DATAFU-12
> URL: https://issues.apache.org/jira/browse/DATAFU-12
> Project: DataFu
> Issue Type: New Feature
> Reporter: Matthew Hayes
>
> Min Zhou has provided this suggestion ([Issue #88 on
> GitHub|https://github.com/linkedin/datafu/pull/88]):
> Lead is an analytic function like Oracle's Lead function. It provides access
> to more than one tuple of a bag at the same time without a self join. Given a
> bag of tuple returned from a query, LEAD provides access to a tuple at a
> given physical offset beyond that position. Generates pairs of all items in a
> bag.
> If you do not specify offset, then its default is 1. Null is returned if the
> offset goes beyond the scope of the bag.
> Example 1:
> {noformat}
> register ba-pig-0.1.jar
> define Lead datafu.pig.bags.Lead('2');
> -- INPUT: ({(1),(2),(3),(4)})
> data = LOAD 'input' AS (data: bag {T: tuple(v:INT)});
> describe data;
> -- OUTPUT: ({((1),(2),(3)),((2),(3),(4)),((3),(4),),((4),,)})
> -- OUTPUT SCHEMA: data2: {lead_data: {(elem0: (v: int),elem1: (v:
> int),elem2: (v: int))}}
> data2 = FOREACH data GENERATE Lead(data);
> describe data2;
> DUMP data2;
> {noformat}
> Example 2
> {noformat}
> register ba-pig-0.1.jar
> define Lead datafu.pig.bags.Lead();
> -- INPUT:
> ({(10,{(1),(2),(3)}),(20,{(4),(5),(6)}),(30,{(7),(8)}),(40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})})
> data = LOAD 'input' AS (data: bag {T: tuple(v1:INT,B: bag{T:
> tuple(v2:INT)})});
> --describe data;
> -- OUPUT:
> ({((10,{(1),(2),(3)}),(20,{(4),(5),(6)})),((20,{(4),(5),(6)}),(30,{(7),(8)})),((30,{(7),(8)}),(40,{(9),(10),(11)})),((40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})),((50,{(12),(13),(14),(15)}),)})
> data2 = FOREACH data GENERATE Lead(data);
> --describe data2;
> DUMP data2;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
