[
https://issues.apache.org/jira/browse/HIVE-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216579#comment-14216579
]
Ryan Blue commented on HIVE-8359:
---------------------------------
I think with [~mickaellcr]'s addition, this is ready to go in.
Good catch in the SerDe code, I didn't realize that the nulls were stripped at
that point as well. I'm a little confused about why we're translating the
ArrayWritable again though: isn't this properly constructed by the Converter
code? Why can't we just pass the ArrayWritable that was created already? It
seems like we're doing a lot of unnecessary work here that we might be able to
remove (in future patches). Ideally, we would detect that the structure matches
what is expected by the following Hive code and pass it along.
> Map containing null values are not correctly written in Parquet files
> ---------------------------------------------------------------------
>
> Key: HIVE-8359
> URL: https://issues.apache.org/jira/browse/HIVE-8359
> Project: Hive
> Issue Type: Bug
> Components: File Formats
> Affects Versions: 0.13.1
> Reporter: Frédéric TERRAZZONI
> Assignee: Sergio Peña
> Attachments: HIVE-8359.1.patch, HIVE-8359.2.patch, HIVE-8359.4.patch,
> HIVE-8359.5.patch, map_null_val.avro
>
>
> Tried write a map<string,string> column in a Parquet file. The table should
> contain :
> {code}
> {"key3":"val3","key4":null}
> {"key3":"val3","key4":null}
> {"key1":null,"key2":"val2"}
> {"key3":"val3","key4":null}
> {"key3":"val3","key4":null}
> {code}
> ... and when you do a query like {code}SELECT * from mytable{code}
> We can see that the table is corrupted :
> {code}
> {"key3":"val3"}
> {"key4":"val3"}
> {"key3":"val2"}
> {"key4":"val3"}
> {"key1":"val3"}
> {code}
> I've not been able to read the Parquet file in our software afterwards, and
> consequently I suspect it to be corrupted.
> For those who are interested, I generated this Parquet table from an Avro
> file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
