[
https://issues.apache.org/jira/browse/PIG-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Noguchi updated PIG-5201:
------------------------------
Attachment: pig-5201-v05.patch
Thanks for the review Rohini!
bq. 1) Can we rename flattenNumFieldsForNull to flattenNumFields?
Done.
bq. 2) POForeach clone should copy flattenNumFieldsForNull
Completely missed that. Done.
bq. 3) isToBeFlattenedArray[ i ] is checked in 3 if statements. Can we create a
outer block with that condition to simplify?
Took a look but I'm already afraid of introducing a regression with this
current change. Prefer not to touch other places if not necessary.
bq. 4) TestFlatten.java - license is modified. Unintended change?
Ouch. Fixed.
bq. System.err.println("KOJIKOJI");
Another ouch. Wiped.
{quote}
7) testFlattenOnNullBagWithColumnPrune is redundant. ColumPrune does not affect
Flatten. Currently we don’t column prune nested data structures - tuple, bag
(PIG-1324).
{quote}
Actually this was testing what would happen when a simple field is pruned. In
the test with schema {noformat}
(a0:int, bag1:bag {(a1_1:int, a1_2:chararray)}, bag2:bag{(a2_1:chararray,
a2_2:chararray)})
{noformat}
this test is pruning a0:int and checking if {{flattenNumFields}} needs to be
shifted.
Lastly, I added a document for this new behavior.
> Null handling on FLATTEN
> ------------------------
>
> Key: PIG-5201
> URL: https://issues.apache.org/jira/browse/PIG-5201
> Project: Pig
> Issue Type: Bug
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Fix For: 0.18.0
>
> Attachments: pig-5201-v00-testonly.patch, pig-5201-v01.patch,
> pig-5201-v02.patch, pig-5201-v03.patch, pig-5201-v04.patch, pig-5201-v05.patch
>
>
> Sometimes, FLATTEN(null) or FLATTEN(bag-with-null) seem to produce incorrect
> results.
> Test code/script to follow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
