[ https://issues.apache.org/jira/browse/HIVE-17915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koifman reassigned HIVE-17915: ------------------------------------- Assignee: Teddy Choi > Enable VectorizedOrcAcidRowBatchReader to be used with LLAP IO elevator over > original acid files > ------------------------------------------------------------------------------------------------ > > Key: HIVE-17915 > URL: https://issues.apache.org/jira/browse/HIVE-17915 > Project: Hive > Issue Type: Sub-task > Components: Transactions > Affects Versions: 3.0.0 > Reporter: Eugene Koifman > Assignee: Teddy Choi > Priority: Critical > > Since HIVE-12631, LLAP IO can support Acid tables but when reading "original" > files. > HIVE-17458 enables VectorizedOrcAcidRowBatchReader to vectorize reads over > "original" files but not with LLAP IO. > Current implementation of _OrcSplit.canUseLlapIo()_ is the same as in > HIVE-12631. > This can/should be improved. There are 2 parts to this: > When a read of "original" file is performed such that data doesn't need to be > decorated with ROW__ID (see > __VectorizedOrcAcidRowBatchReader.canUseLlapForAcid()_) then > VectorizedOrcAcidRowBatchReader as of HIVE-17458 should be usable with LLAP > IO but when I tried it I got _ArrayIndexOutOfBoundsException_ in various > places of the stack. > This is the more important one. > The 2nd issue is that reading "original" acid files (when ROW__IDs are > needed) requires using > _org.apache.hadoop.hive.ql.io.orc.RecordReader.getRowNumber()_ in > __VectorizedOrcAcidRowBatchReader_ > This API is not available on the reader that _LlapRecordReader_ provides. > It would be better if getRowNumber() was available for performance as well as > simpler logic in the code. > cc [~sershe], [~teddy.choi] -- This message was sent by Atlassian JIRA (v6.4.14#64029)