[
https://issues.apache.org/jira/browse/ORC-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146030#comment-16146030
]
ASF GitHub Bot commented on ORC-162:
------------------------------------
Github user dain commented on the issue:
https://github.com/apache/orc/pull/163
We were considering doing this internally and then we ran into a production
bug where files got truncated to zero bytes. Since empty files are illegal we
could find all of the effected partitions easily, but without this you would be
stuck. A good work around would be to require empty files to end with ".empty"
but I'm not sure you can do that with M/R.
> Handle 0 byte files as empty ORC files
> --------------------------------------
>
> Key: ORC-162
> URL: https://issues.apache.org/jira/browse/ORC-162
> Project: ORC
> Issue Type: Bug
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
>
> Hive often creates empty files for empty buckets, which can introduce
> significant load on the HDFS cluster. Therefore, they made the Hive
> OrcOutputFormat and OrcInputFormat use 0 byte ORC files as a special case.
> We need to make the other readers treat them reasonably.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
