[ https://issues.apache.org/jira/browse/PIG-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279725#comment-13279725 ]
Mike Percy commented on PIG-2709: --------------------------------- Old stack trace: java.lang.RuntimeException: java.io.IOException: Not a data file. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:236) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:118) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: java.io.IOException: Not a data file. at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102) at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97) at org.apache.pig.piggybank.storage.avro.PigAvroRecordReader.<init>(PigAvroRecordReader.java:53) at org.apache.pig.piggybank.storage.avro.PigAvroInputFormat.createRecordReader(PigAvroInputFormat.java:66) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:227) ... 9 more New stack trace: java.lang.RuntimeException: java.io.IOException: Error initializing data file reader for file (hdfs://localhost/logs/data.avro) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:236) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>(PigRecordReader.java:109) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:118) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:614) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: java.io.IOException: Error initializing data file reader for file (hdfs://localhost/logs/data.avro) at org.apache.pig.piggybank.storage.avro.PigAvroRecordReader.<init>(PigAvroRecordReader.java:56) at org.apache.pig.piggybank.storage.avro.PigAvroInputFormat.createRecordReader(PigAvroInputFormat.java:66) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader(PigRecordReader.java:227) ... 9 more Caused by: java.io.IOException: Not a data file. at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102) at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97) at org.apache.pig.piggybank.storage.avro.PigAvroRecordReader.<init>(PigAvroRecordReader.java:54) ... 11 more > PigAvroRecordReader should specify which file has a problem when throwing > IOException > ------------------------------------------------------------------------------------- > > Key: PIG-2709 > URL: https://issues.apache.org/jira/browse/PIG-2709 > Project: Pig > Issue Type: Improvement > Reporter: Mike Percy > Attachments: PIG-2709-1.patch > > > Today, if AvroStorage opens a file that is not valid Avro, an exception is > thrown but it's unclear which file actually had the problem form looking at > the M/R logs. The PigAvroRecordReader should specify the file being opened > when an exception is thrown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira