[
https://issues.apache.org/jira/browse/TEZ-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated TEZ-2348:
----------------------------------
Attachment: TEZ-2348.1.patch
[~jdere] - The exception from IFile is valid as higher level API is not
expected to call nextRawKey() when the end of file is reached. Can you please
check if your Hive patch is calling UnorderedKVReader.next() even after it
returns false?. In such situations, this error is possible.
For instance, you can check for "readers.UnorderedKVReader: Num Records read:"
in the task attempt log. This indicates that the UnoderedKVReader has finished
processing and no more data is available. However, if higher level APIs invoke
UnorderedKVReader.next() again, it would end up throwing EOF exception from
IFile.
Attaching the patch which handles this situation from tez side. [~sseth] - Can
you please review?
> EOF exception during UnorderedKVReader.next()
> ---------------------------------------------
>
> Key: TEZ-2348
> URL: https://issues.apache.org/jira/browse/TEZ-2348
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.2
> Reporter: Jason Dere
> Attachments: TEZ-2348.1.patch, _tez_session_dir.tgz
>
>
> {noformat}
> Caused by: java.lang.RuntimeException: java.io.IOException: Reached EOF.
> Completed reading 516605
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:278)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:184)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
> ... 13 more
> Caused by: java.io.IOException: Reached EOF. Completed reading 516605
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile.checkState(IFile.java:817)
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.positionToNextRecord(IFile.java:698)
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.readRawKey(IFile.java:731)
> at
> org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.nextRawKey(IFile.java:727)
> at
> org.apache.tez.runtime.library.common.readers.UnorderedKVReader.readNextFromCurrentReader(UnorderedKVReader.java:151)
> at
> org.apache.tez.runtime.library.common.readers.UnorderedKVReader.next(UnorderedKVReader.java:112)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$KeyValuesFromKeyValue.next(ReduceRecordSource.java:439)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:232)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
