Cheolsoo Park created PIG-2909:
----------------------------------
Summary: Add a new option for ignoring corrupted files to
AvroStorage load func
Key: PIG-2909
URL: https://issues.apache.org/jira/browse/PIG-2909
Project: Pig
Issue Type: Bug
Components: piggybank
Affects Versions: 0.10.0
Reporter: Cheolsoo Park
Assignee: Cheolsoo Park
Currently, AvroStorage load fails with AvroRuntimeException when encountering
corrupted input files. For example,
{code}
ERROR 2997: Unable to recreate exception from backed error:
java.io.IOException: org.apache.avro.AvroRuntimeException: java.io.IOException:
Invalid sync!
at
org.apache.pig.piggybank.storage.avro.AvroStorage.getNext(AvroStorage.java:283)
{code}
But it is not always desirable to fail the Pig job for bad files. It is
sometimes more useful to skip them and continue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
