HAR sequence files throw errors in MR jobs ------------------------------------------
Key: MAPREDUCE-1712 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1712 Project: Hadoop Map/Reduce Issue Type: Bug Components: harchive Affects Versions: 0.20.1 Reporter: Paul Yang When a HAR is specified as the input for a map reduce job and the file format is sequence file, an error similar to the following is thrown (this one is from Hive). {code} java.lang.IllegalArgumentException: Offset 0 is outside of file (0..-1) at org.apache.hadoop.mapred.FileInputFormat.getBlockIndex(FileInputFormat.java:299) at org.apache.hadoop.mapred.FileInputFormat.getSplitHosts(FileInputFormat.java:455) at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:260) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:261) at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:827) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:798) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:747) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:663) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) {code} This is caused by the dummy block location returned by HarFileSystem.getFileBlockLocations(). -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira