Jim Apple has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13436
Change subject: IMPALA-5031: NULL is undefined in memcpy and memcmp ...................................................................... IMPALA-5031: NULL is undefined in memcpy and memcmp This patch fixes UBSAN "null pointer passed as argument" errors in the end-to-end tests. These are undefined behavior according to "7.1.4 Use of library functions" in the C99 standard (which is included in C++14 in section [intro.refs]): If an argument to a function has an invalid value (such as a value outside the domain of the function, or a pointer outside the address space of the program, or a null pointer, or a pointer to non-modifiable storage when the corresponding parameter is not const-qualified) or a type (after promotion) not expected by a function with variable number of arguments, the behavior is undefined. The interesting parts of the backtraces are: runtime/sorter.cc:575:18: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/string.h:43:45: note: nonnull attribute specified here #0 Sorter::Run::CopyVarLenData(vector<StringValue*> const&, unsigned char*) runtime/sorter.cc:575:5 #1 Status Sorter::Run::AddBatchInternal<true, true>(RowBatch*, int, int*) runtime/sorter.cc:232:11 #2 Sorter::Run::AddInputBatch(RowBatch*, int, int*) runtime/sorter.cc:660:12 #3 Sorter::AddBatchNoSpill(RowBatch*, int, int*) runtime/sorter.cc:882:58 #4 Sorter::AddBatch(RowBatch*) runtime/sorter.cc:862:45 #5 SortNode::SortInput(RuntimeState*) exec/sort-node.cc:177:54 #6 SortNode::Open(RuntimeState*) exec/sort-node.cc:90:43 runtime/tuple.cc:105:25: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/string.h:43:45: note: nonnull attribute specified here #0 Tuple::DeepCopyVarlenData(TupleDescriptor const&, MemPool*) runtime/tuple.cc:105:5 #1 Tuple::DeepCopy(Tuple*, TupleDescriptor const&, MemPool*) runtime/tuple.cc:94:35 #2 Tuple::DeepCopy(TupleDescriptor const&, MemPool*) runtime/tuple.cc:85:3 #3 KrpcDataStreamSender::Channel::AddRow(TupleRow*) runtime/krpc-data-stream-sender.cc:509:43 #4 KrpcDataStreamSender::AddRowToChannel(int, TupleRow*) runtime/krpc-data-stream-sender.cc:846 #5 (<unknown module>) runtime/tuple.cc:146:19: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/string.h:43:45: note: nonnull attribute specified here #0 Tuple::DeepCopyVarlenData(TupleDescriptor const&, char**, int*, bool) runtime/tuple.cc:146:5 #1 Tuple::DeepCopy(TupleDescriptor const&, char**, int*, bool) runtime/tuple.cc:135:35 #2 RowBatch::SerializeInternal(long, FixedSizeHashTable<Tuple*, int>*, vector<int>*, string*) runtime/row-batch.cc:392:14 #3 RowBatch::Serialize(bool, vector<int>*, string*, long*, bool*) runtime/row-batch.cc:290:45 #4 RowBatch::Serialize(OutboundRowBatch*) runtime/row-batch.cc:259:43 #5 KrpcDataStreamSender::SerializeBatch(RowBatch*, OutboundRowBatch*, int) runtime/krpc-data-stream-sender.cc:955:50 #6 KrpcDataStreamSender::Send(RuntimeState*, RowBatch*) runtime/krpc-data-stream-sender.cc:870:45 runtime/tuple.h:106:12: runtime error: null pointer passed as argument 1, which is declared to never be null /usr/include/string.h:62:79: note: nonnull attribute specified here #0 Tuple::ClearNullBits(int, int) runtime/tuple.h:106:5 #1 HdfsScanner::InitTuple(TupleDescriptor const*, Tuple*, Tuple*) exec/hdfs-scanner.h:512:14 #2 HdfsOrcScanner::AssembleCollection(OrcComplexColumnReader const&, int, CollectionValueBuilder*) exec/hdfs-orc-scanner.cc:742:7 #3 OrcCollectionReader::ReadValue(int, Tuple*, MemPool*) exec/orc-column-readers.cc:375:20 #4 OrcStructReader::ReadValue(int, Tuple*, MemPool*) exec/orc-column-readers.cc:322:52 #5 OrcListReader::ReadChildrenValue(int, int, Tuple*, MemPool*) const exec/orc-column-readers.cc:473:52 #6 HdfsOrcScanner::AssembleCollection(OrcComplexColumnReader const&, int, CollectionValueBuilder*) exec/hdfs-orc-scanner.cc:743:60 #7 OrcCollectionReader::ReadValue(int, Tuple*, MemPool*) exec/orc-column-readers.cc:375:20 #8 OrcStructReader::TransferTuple(Tuple*, MemPool*) exec/orc-column-readers.cc:346:52 #9 HdfsOrcScanner::TransferTuples(OrcComplexColumnReader*, RowBatch*) exec/hdfs-orc-scanner.cc:669:58 #10 HdfsOrcScanner::AssembleRows(RowBatch*) exec/hdfs-orc-scanner.cc:629:45 #11 HdfsOrcScanner::GetNextInternal(RowBatch*) exec/hdfs-orc-scanner.cc:507:19 #12 HdfsOrcScanner::ProcessSplit() exec/hdfs-orc-scanner.cc:426:21 #13 HdfsScanNode::ProcessSplit(vector<FilterContext> const&, MemPool*, io::ScanRange*, long*) exec/hdfs-scan-node.cc:514:21 #14 HdfsScanNode::ScannerThread(bool, long) exec/hdfs-scan-node.cc:415:7 #15 HdfsScanNode::ThreadTokenAvailableCb(ThreadResourcePool*) ::$_0::operator()() const exec/hdfs-scan-node.cc:337:13 Change-Id: I9362ce6b9ba470ed90e5bd2dc313b66ebd8c6af5 --- M be/src/runtime/sorter.cc M be/src/runtime/tuple.cc M be/src/runtime/tuple.h 3 files changed, 7 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/13436/1 -- To view, visit http://gerrit.cloudera.org:8080/13436 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9362ce6b9ba470ed90e5bd2dc313b66ebd8c6af5 Gerrit-Change-Number: 13436 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple <jbap...@apache.org>