cxzl25 opened a new pull request #979:
URL: https://github.com/apache/orc/pull/979
### What changes were proposed in this pull request?
Use buffer limit as `readSize` to avoid `IndexOutOfBoundsException`.
### Why are the changes needed?
ORC-251 remove `ReaderImpl.extractFileTail`
ORC-685 Add `ReaderImpl.extractFileTail` back
In ORC-685, file length is used as readsize, which causes that if the buffer
is read from the cache, the use of length is incorrect, resulting in
IndexOutOfBoundsException.
```
long readSize = fileLen != -1? fileLen: buffer.limit();
int psLen = buffer.get((int) (readSize-1)) & 0xff;
```
```
Caused by: java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:540)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:726)
at
org.apache.hadoop.hive.ql.io.orc.LocalCache.getAndValidate(LocalCache.java:103)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.getSplits(OrcInputFormat.java:798)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.runGetSplitsSync(OrcInputFormat.java:916)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$ETLSplitStrategy.generateSplitWork(OrcInputFormat.java:885)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.scheduleSplits(OrcInputFormat.java:1759)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1703)
```
### How was this patch tested?
local test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@orc.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
