[ https://issues.apache.org/jira/browse/DATAFU-12?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972397#comment-15972397 ]
Eyal Allweil commented on DATAFU-12: ------------------------------------ It looks like this functionality is implemented in HIve - see the following two links: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics#LanguageManualWindowingAndAnalytics-LEADusingdefault1rowleadandnotspecifyingdefaultvalue https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLead.java Since Pig now supports using Hive UDF's, I think this Jira can be closed. Alternately, if we want to provide a DataFu implementation, I'll copy the proposed patch and discussion from the Github issue mentioned in the description, so it's easier for a possible-implementer to continue where work stalled. > Implement Lead UDF based on version from SQL > -------------------------------------------- > > Key: DATAFU-12 > URL: https://issues.apache.org/jira/browse/DATAFU-12 > Project: DataFu > Issue Type: New Feature > Reporter: Matthew Hayes > > Min Zhou has provided this suggestion ([Issue #88 on > GitHub|https://github.com/linkedin/datafu/pull/88]): > Lead is an analytic function like Oracle's Lead function. It provides access > to more than one tuple of a bag at the same time without a self join. Given a > bag of tuple returned from a query, LEAD provides access to a tuple at a > given physical offset beyond that position. Generates pairs of all items in a > bag. > If you do not specify offset, then its default is 1. Null is returned if the > offset goes beyond the scope of the bag. > Example 1: > {noformat} > register ba-pig-0.1.jar > define Lead datafu.pig.bags.Lead('2'); > -- INPUT: ({(1),(2),(3),(4)}) > data = LOAD 'input' AS (data: bag {T: tuple(v:INT)}); > describe data; > -- OUTPUT: ({((1),(2),(3)),((2),(3),(4)),((3),(4),),((4),,)}) > -- OUTPUT SCHEMA: data2: {lead_data: {(elem0: (v: int),elem1: (v: > int),elem2: (v: int))}} > data2 = FOREACH data GENERATE Lead(data); > describe data2; > DUMP data2; > {noformat} > Example 2 > {noformat} > register ba-pig-0.1.jar > define Lead datafu.pig.bags.Lead(); > -- INPUT: > ({(10,{(1),(2),(3)}),(20,{(4),(5),(6)}),(30,{(7),(8)}),(40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})}) > data = LOAD 'input' AS (data: bag {T: tuple(v1:INT,B: bag{T: > tuple(v2:INT)})}); > --describe data; > -- OUPUT: > ({((10,{(1),(2),(3)}),(20,{(4),(5),(6)})),((20,{(4),(5),(6)}),(30,{(7),(8)})),((30,{(7),(8)}),(40,{(9),(10),(11)})),((40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})),((50,{(12),(13),(14),(15)}),)}) > data2 = FOREACH data GENERATE Lead(data); > --describe data2; > DUMP data2; > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)