[
https://issues.apache.org/jira/browse/BEAM-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743030#comment-16743030
]
Jozef Vilcek commented on BEAM-6394:
------------------------------------
I did manage to get prove of concept running. However, at the end I have
decided to move with avro instead of protobuf, although most of other data
structures in my pipeline are protobuf. Seems like proto writer produces a bit
different schema for equivalent data structure in avro. E.g. protobuf Map seems
not to be well supported which caused me some issue when mapping results to
Hive.
Feel free to keep this open if anyone else would need it, but right now, I will
not have need or spare time to draft a PR
> Support for writing protobuf data to parquet
> --------------------------------------------
>
> Key: BEAM-6394
> URL: https://issues.apache.org/jira/browse/BEAM-6394
> Project: Beam
> Issue Type: New Feature
> Components: io-java-parquet
> Reporter: Jozef Vilcek
> Assignee: Lukasz Gajowy
> Priority: Major
>
> Parquet infrastructure does support writing protobuf data to parquet. Beam's
> ParquetIO could give pipeline developers an option to write protobuf data
> instead of converting them to avro.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
