Navdeep Poonia created HIVE-16352:
-------------------------------------
Summary: Ability to skip or repair out of sync blocks with HIVE at
runtime
Key: HIVE-16352
URL: https://issues.apache.org/jira/browse/HIVE-16352
Project: Hive
Issue Type: New Feature
Reporter: Navdeep Poonia
When a file is corrupted it raises the error java.io.IOException: Invalid sync!
with hive.
Can we have some functionality to skip or repair such blocks at runtime to make
avro more error resilient in case of data corruption.
Error: java.io.IOException: java.io.IOException: java.io.IOException: While
processing file
s3n://dev.canopydata.com/navdeepp/warehouse/avro_test/354dc34474404f4bbc0d8013fc8e6e4b_000042.
java.io.IOException: Invalid sync!
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:334)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
