[ https://issues.apache.org/jira/browse/IMPALA-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839692#comment-17839692 ]
ASF subversion and git services commented on IMPALA-13004: ---------------------------------------------------------- Commit a4a755d173822d3e123a871ffad6203ca98ff9f5 in impala's branch refs/heads/branch-4.4.0 from Yida Wu [ https://gitbox.apache.org/repos/asf?p=impala.git;h=a4a755d17 ] IMPALA-13004: Fix heap-use-after-free error in ExprTest AiFunctionsTest The issue is that the code previously used a std::string_view to hold the data which is actually returned by rapidjson::Document. However, the rapidjson::Document object gets destroyed after creating the std::string_view. This meant the std::string_view referenced memory that was no longer valid, leading to a heap-use-after-free error. This patch fixes this issue by modifying the function to return a std::string instead of a std::string_view. When the function returns a string, it creates a copy of the data from rapidjson::Document. This ensures the returned string has its own memory allocation and doesn't rely on the destroyed rapidjson::Document. Tests: Reran the asan build and passed. Change-Id: I3bb9dcf9d72cce7ad37d5bc25821cf6ee55a8ab5 Reviewed-on: http://gerrit.cloudera.org:8080/21315 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> > heap-use-after-free error in ExprTest AiFunctionsTest > ----------------------------------------------------- > > Key: IMPALA-13004 > URL: https://issues.apache.org/jira/browse/IMPALA-13004 > Project: IMPALA > Issue Type: Bug > Components: be > Affects Versions: Impala 4.4.0 > Reporter: Andrew Sherman > Assignee: Yida Wu > Priority: Critical > Fix For: Impala 4.4.0 > > > In an ASAN test, expr-test fails: > {code} > ==1601==ERROR: AddressSanitizer: heap-use-after-free on address > 0x63100152c826 at pc 0x00000298f841 bp 0x7ffc91fff460 sp 0x7ffc91fff458 > READ of size 2 at 0x63100152c826 thread T0 > #0 0x298f840 in rapidjson::GenericValue<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::GetType() const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:936:62 > #1 0x298d852 in bool rapidjson::GenericValue<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, > rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator, 0u> > >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, > rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator, 0u>&) const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1769:16 > #2 0x298d8d0 in bool rapidjson::GenericValue<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, > rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator, 0u> > >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, > rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator, 0u>&) const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1790:21 > #3 0x298d9e8 in bool rapidjson::GenericValue<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, > rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator, 0u> > >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, > rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator, 0u>&) const > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1781:21 > #4 0x28a0707 in impala_udf::StringVal > impala::AiFunctions::AiGenerateTextInternal<false>(impala_udf::FunctionContext*, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, bool) > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:140:11 > #5 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody() > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12 > #6 0x8aeaa4c in void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c) > #7 0x8ae3ec4 in testing::Test::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae3ec4) > #8 0x8ae4007 in testing::TestInfo::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4007) > #9 0x8ae40e4 in testing::TestCase::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae40e4) > #10 0x8ae45db in testing::internal::UnitTestImpl::RunAllTests() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae45db) > #11 0x8ae4682 in testing::UnitTest::Run() > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4682) > #12 0x249ac19 in main > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10 > #13 0x7f4b0b911554 in __libc_start_main (/lib64/libc.so.6+0x22554) > #14 0x2396af6 in _start > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2396af6) > 0x63100152c826 is located 38 bytes inside of 65560-byte region > [0x63100152c800,0x63100153c818) > freed by thread T0 here: > #0 0x2466ea7 in __interceptor_free > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2466ea7) > #1 0x299656b in > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::Clear() > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:148:13 > #2 0x29964cd in > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::~MemoryPoolAllocator() > > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:140:9 > #3 0x2996499 in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>::Destroy() > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2391:9 > #4 0x298e47d in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>::~GenericDocument() > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2073:9 > #5 0x28a0682 in impala_udf::StringVal > impala::AiFunctions::AiGenerateTextInternal<false>(impala_udf::FunctionContext*, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, bool) > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:136:3 > #6 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody() > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12 > #7 0x8aeaa4c in void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c) > previously allocated by thread T0 here: > #0 0x246706f in __interceptor_malloc > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x246706f) > #1 0x298f3bf in > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::AddChunk(unsigned > long) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:240:81 > #2 0x298f2bc in > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>::Malloc(unsigned > long) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:182:18 > #3 0x299a81c in rapidjson::GenericValue<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > >::SetArrayRaw(rapidjson::GenericValue<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >*, unsigned int, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>&) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1937:68 > #4 0x299a7b6 in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>::EndArray(unsigned int) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2371:43 > #5 0x29989a0 in void rapidjson::GenericReader<rapidjson::UTF8<char>, > rapidjson::UTF8<char>, rapidjson::CrtAllocator>::ParseArray<0u, > rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>, rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator> > >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>&, rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>&) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/reader.h:686:21 > #6 0x299858e in void rapidjson::GenericReader<rapidjson::UTF8<char>, > rapidjson::UTF8<char>, rapidjson::CrtAllocator>::ParseObject<0u, > rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>, rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator> > >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>&, rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>&) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/reader.h:621:13 > #7 0x29976c8 in rapidjson::ParseResult > rapidjson::GenericReader<rapidjson::UTF8<char>, rapidjson::UTF8<char>, > rapidjson::CrtAllocator>::Parse<0u, > rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>, rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator> > >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>&, rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>&) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/reader.h:501:13 > #8 0x299726a in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>& rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>::ParseStream<0u, rapidjson::UTF8<char>, > rapidjson::EncodedInputStream<rapidjson::UTF8<char>, rapidjson::MemoryStream> > >(rapidjson::EncodedInputStream<rapidjson::UTF8<char>, > rapidjson::MemoryStream>&) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2159:40 > #9 0x2996f91 in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>& rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, > rapidjson::CrtAllocator>::Parse<0u, rapidjson::UTF8<char> > >(rapidjson::UTF8<char>::Ch const*, unsigned long) > /data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:2248:9 > #10 0x28a01ae in impala_udf::StringVal > impala::AiFunctions::AiGenerateTextInternal<false>(impala_udf::FunctionContext*, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, impala_udf::StringVal const&, > impala_udf::StringVal const&, bool) > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:109:15 > #11 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody() > /data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12 > #12 0x8aeaa4c in void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > (/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org