[
https://issues.apache.org/jira/browse/PARQUET-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120606#comment-14120606
]
Dmitriy V. Ryaboy commented on PARQUET-61:
------------------------------------------
merged https://github.com/apache/incubator-parquet-mr/pull/28
> Avoid fixing protocol events when there is not required field missing
> ---------------------------------------------------------------------
>
> Key: PARQUET-61
> URL: https://issues.apache.org/jira/browse/PARQUET-61
> Project: Parquet
> Issue Type: Improvement
> Reporter: tianshuo
> Assignee: tianshuo
> Fix For: parquet-mr_1.6.0
>
>
> Currently, due to the projection pushdown, in the ThriftRecordConverter, it
> will check if there is any protocol events missing for the required fields
> and fix them if there is. This happens for all the records.
> The fixing of missing protocol can be done in a more precise condition to
> improve performance:
> Only when the requested schema missing some required fields that are present
> in the full schema
> So even if there a projection, as long as the projection is not getting rid
> of the required field, the protocol events amender will not be called.
> https://github.com/apache/incubator-parquet-mr/pull/28
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
