[
https://issues.apache.org/jira/browse/HIVE-9979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363528#comment-14363528
]
Gopal V commented on HIVE-9979:
-------------------------------
Even the String dictionary readers are triggering exceptions
{code}
2015-03-16 10:20:51,439
[pool-2-thread-3(container_222212222_1141_01_000192_gopal_20150316102020_c8c92488-6a61-401e-8298-401dace286dc:1_Map
1_191_0)] INFO org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl: Getting
data for column 9 RG 112 stream DATA at 62278935, 1057137 index position 0:
compressed [62614934, 63139228)
2015-03-16 10:20:51,439
[pool-2-thread-6(container_222212222_1141_01_000211_gopal_20150316102020_c8c92488-6a61-401e-8298-401dace286dc:1_Map
1_210_0)] INFO org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl: Getting
stripe-level stream [LENGTH, kind: DICTIONARY_V2
dictionarySize: 3
] for column 9 RG 91 at 64139927, 5
...
Caused by: java.io.EOFException
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderUtils.readDirect(RecordReaderUtils.java:286)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderUtils.readDiskRanges(RecordReaderUtils.java:266)
at
org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:234)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:280)
at
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:44)
at
org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
... 4 more
{code}
> LLAP: LLAP Cached readers for StringDirectTreeReaders over-read data
> --------------------------------------------------------------------
>
> Key: HIVE-9979
> URL: https://issues.apache.org/jira/browse/HIVE-9979
> Project: Hive
> Issue Type: Sub-task
> Affects Versions: llap
> Reporter: Gopal V
> Assignee: Sergey Shelukhin
>
> When the cache is enabled, queries throws different over-read exceptions.
> Looks like the batchSize changes as you read data, the end of stripe
> batchSize is smaller than the default size (the super calls change it).
> {code}
> Caused by: java.io.EOFException: Can't finish byte read from uncompressed
> stream DATA position: 262144 length: 262144 range: 0 offset: 46399488 limit:
> 46399488
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$BytesColumnVectorUtil.commonReadByteArrays(RecordReaderImpl.java:1556)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$BytesColumnVectorUtil.readOrcByteArrays(RecordReaderImpl.java:1569)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringDirectTreeReader.nextVector(RecordReaderImpl.java:1691)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringTreeReader.nextVector(RecordReaderImpl.java:1517)
> at
> org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:115)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:108)
> at
> org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:35)
> at
> org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:314)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:280)
> at
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:44)
> at
> org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
