[ https://issues.apache.org/jira/browse/PIG-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945728#comment-15945728 ]
Koji Noguchi commented on PIG-5201: ----------------------------------- {code:title=test.pig} A = load 'input.txt' as (a1:bag {(a1_1:chararray, a1_2:chararray)}, a2:bag{(a2_1:chararray, a2_2:chararray)}, a3:chararray); B = FOREACH A generate flatten(a1), flatten(a2), a3; DUMP B; {code} Input ||a1 || a2 || a3 || | {(1,2)} | | 1 | | {(2,3),(3,4)} | {(a,b),(c,d)} | 2 | | | | 3 | | | {(e,f)} | 4 | | {,(7,8)} | {(g,h),(i,j)} | 5 | Output {noformat} (1,2,,1) (2,3,a,b,2) (2,3,c,d,2) (3,4,a,b,2) (3,4,c,d,2) (,,3) (,e,f,4) (7,8,g,h,5) (7,8,i,j,5) {noformat} Formatting the output in a table ||a2_1|| a2_2 || a3_1 || a3_2 || a4 || | 1 | 2 | | 1 | | 2 | 3 | a | b | 2 | | 2 | 3 | c | d | 2 | | 3 | 4 | a | b | 2 | | 3 | 4 | c | d | 2 | | | | 3 | | | e | f | 4 | | 7 | 8 | g | h | 5 | | 7 | 8 | i | j | 5 | Using different loader({{mock.Storage}}), I can also see null and \{(g,h),(i,j)\} pair creating ||a2_1|| a2_2 || a3_1 || a3_2 || a4 || | | g | h | 5 | | | i | j | 5 | A couple of quetions. (1) What should be the behavior for FLATTEN(null)? For FLATTEN(empty_bag), we drop them. Shall we do the same? (Offline, Rohini said no.) (2) What should be the behavior for null record within the bag in FLATTEN(bag-with-null) ? and probably a separate jira, but (3) Why is PigStorage not showing the empty item for \{,(7,8)\} ? > Null handling on FLATTEN > ------------------------ > > Key: PIG-5201 > URL: https://issues.apache.org/jira/browse/PIG-5201 > Project: Pig > Issue Type: Bug > Reporter: Koji Noguchi > Assignee: Koji Noguchi > Priority: Minor > > Sometimes, FLATTEN(null) or FLATTEN(bag-with-null) seem to produce incorrect > results. > Test code/script to follow. -- This message was sent by Atlassian JIRA (v6.3.15#6346)