[
https://issues.apache.org/jira/browse/IMPALA-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16901980#comment-16901980
]
Daniel Becker commented on IMPALA-8840:
---------------------------------------
I think the fix to IMPALA-8833 solves this, but not necessarily other uses of
RleBatchDecoder.
The problem is that RleBatchedReader<T>::Reset() doesn't check whether the bit
width is at most sizeof(T) * 8. It DCHECKs if it is more than
BatchedBitReader::MAX_BITWIDTH but that is not enough as later it unpacks
values to T, which DCHECKs if the bit widths is too high for T. Also, Reset()
returns nothing so we cannot signal an error.
As this interface is called by other Impala functions and not with
user-provided data, I'll not add a runtime check but a DCHECK and document that
the bit width should be at most sizeof(T) * 8.
> Check failed: num_bytes <= sizeof(T) (5 vs. 4)
> -----------------------------------------------
>
> Key: IMPALA-8840
> URL: https://issues.apache.org/jira/browse/IMPALA-8840
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.3.0
> Reporter: Xiaomeng Zhang
> Assignee: Daniel Becker
> Priority: Blocker
> Labels: broken-build, crash
>
> Not sure if this is due to same issue as
> https://issues.apache.org/jira/browse/IMPALA-8833#, the error message is a
> little different.
> {code:java}
> F0805 18:48:08.737411 5488 bit-stream-utils.inline.h:173]
> 284731e5d1aad693:05c8830200000001] Check failed: num_bytes <= sizeof(T) (8
> vs. 4)
> *** Check failure stack trace: ***
> @ 0x52fb9bc google::LogMessage::Fail()
> @ 0x52fd261 google::LogMessage::SendToLog()
> @ 0x52fb396 google::LogMessage::Flush()
> @ 0x52fe95d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2b2b867 impala::BatchedBitReader::GetBytes<>()
> @ 0x2aeda65 impala::RleBatchDecoder<>::NextCounts()
> @ 0x2a82896 impala::RleBatchDecoder<>::NextNumRepeats()
> @ 0x2b1927f impala::ScalarColumnReader<>::ReadSlotsNoConversion()
> @ 0x2ac7c2c impala::ScalarColumnReader<>::ReadSlots()
> @ 0x2a7b861
> impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
> @ 0x2a5b3b0 impala::ScalarColumnReader<>::ReadValueBatch<>()
> @ 0x2a256a4 impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
> @ 0x29b6eb6 impala::HdfsParquetScanner::AssembleRows()
> @ 0x29b1cf8 impala::HdfsParquetScanner::GetNextInternal()
> @ 0x29afc70 impala::HdfsParquetScanner::ProcessSplit()
> @ 0x2494bc3 impala::HdfsScanNode::ProcessSplit()
> @ 0x2493d98 impala::HdfsScanNode::ScannerThread()
> @ 0x2493121
> _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @ 0x24956e9
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x1ea0241 boost::function0<>::operator()()
> @ 0x23de77a impala::Thread::SuperviseThread()
> @ 0x23e6afe boost::_bi::list5<>::operator()<>()
> @ 0x23e6a22 boost::_bi::bind_t<>::operator()()
> @ 0x23e69e5 boost::detail::thread_data<>::run()
> @ 0x4224819 thread_proxy
> @ 0x7fc1818c5e24 start_thread
> @ 0x7fc17e01f34c __clone
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
