[ https://issues.apache.org/jira/browse/KUDU-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779855#comment-16779855 ]
Adar Dembo commented on KUDU-2716: ---------------------------------- This was fixed in an [upstream rapidjson commit|https://github.com/Tencent/rapidjson/commit/1a570c342dad0927503b35782263da006ef93351]. That commit is in the latest release (1.1.0); if we upgraded to it, we could probably fix this. Note that we're on 0.11 which is over six years old. > rapidjson mishandles kMinInt32 and kMinInt64 > -------------------------------------------- > > Key: KUDU-2716 > URL: https://issues.apache.org/jira/browse/KUDU-2716 > Project: Kudu > Issue Type: Bug > Components: util > Affects Versions: 1.9.0 > Reporter: Adar Dembo > Priority: Major > > The rapidjson code that parses numbers mishandles kMinInt32 and kMinInt64. > Moreover, it only appears to do so when compiled with -O3 (the default for > RELEASE builds) by certain compilers. I could not reproduce the mishandling > when compiling with no optimizations (DEBUG) or with -O1 (FASTDEBUG). > I can't speak to kMinInt64, but when parsing kMinInt32, rapidjson identifies > the number as a possible unsigned, unsigned long, signed, or signed long, > instead of just a possible signed or signed long. Currently, this manifests > as a SIGSEGV in jsonreader-test.cc: > {noformat} > *** Aborted at 1551300683 (unix time) try "date -d @1551300683" if you are > using GNU date *** > PC: @ 0x55d7ab64ae19 kudu::JsonReader::ExtractUint32() > *** SIGSEGV (@0x0) received by PID 10903 (TID 0x7f578ebdf840) from PID 0; > stack trace: *** > @ 0x7f578da84890 (unknown) > @ 0x55d7ab64ae19 kudu::JsonReader::ExtractUint32() > @ 0x55d7ab5f95cf kudu::JsonReaderTest_SignedAndUnsignedInts_Test::TestBody() > @ 0x55d7ab83d0cd testing::internal::HandleExceptionsInMethodIfSupported<>() > @ 0x55d7ab8331f2 testing::Test::Run() > @ 0x55d7ab8332d4 testing::TestInfo::Run() > @ 0x55d7ab833417 testing::TestCase::Run() > @ 0x55d7ab8338e8 testing::internal::UnitTestImpl::RunAllTests() > @ 0x55d7ab83d5ad testing::internal::HandleExceptionsInMethodIfSupported<>() > @ 0x55d7ab833a41 testing::UnitTest::Run() > @ 0x55d7ab5e5776 main > @ 0x7f578d6a2b97 __libc_start_main > @ 0x55d7ab5eba7a _start > {noformat} > The offending line is: > {noformat} > ASSERT_TRUE(r.ExtractUint32(r.root(), signed_small32, > nullptr).IsInvalidArgument()); > {noformat} > Internally, our JsonReader class successfully parses the number as an > unsigned, which then leads to a crash because we passed in 'nullptr' as the > result (because we expected an earlier parsing error). > We've reproduced this with the gcc found on Ubuntu 18, but not the one in > Ubuntu 16. It also repros if you use clang-6.0 from thirdparty with RELEASE > builds. > We already knew that this was an issue as early as commit 02e82ca14, which > disabled this test in UBSAN builds, because the mishandling triggers UBSAN: > {noformat} > thirdparty/installed/common/include/rapidjson/reader.h:644:18: runtime error: > negation of -2147483648 cannot be represented in type 'int'; cast to an > unsigned type to negate this value to itself > #0 0x7f75d4db9d58 in void rapidjson::GenericReader<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::ParseNumber<0u, > rapidjson::GenericStringStream<rapidjson::UTF8<char> >, > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > > >(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&, > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&) > ../thirdparty/installed/common/include/rapidjson/reader.h:644:18 > #1 0x7f75d4db6b84 in void rapidjson::GenericReader<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::ParseObject<0u, > rapidjson::GenericStringStream<rapidjson::UTF8<char> >, > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > > >(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&, > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&) > ../thirdparty/installed/common/include/rapidjson/reader.h:290:4 > #2 0x7f75d4db646f in bool rapidjson::GenericReader<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::Parse<0u, > rapidjson::GenericStringStream<rapidjson::UTF8<char> >, > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> > > >(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&, > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >&) > ../thirdparty/installed/common/include/rapidjson/reader.h:243:15 > #3 0x7f75d4db6105 in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >& > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::ParseStream<0u, > rapidjson::GenericStringStream<rapidjson::UTF8<char> > > >(rapidjson::GenericStringStream<rapidjson::UTF8<char> >&) > ../thirdparty/installed/common/include/rapidjson/document.h:712:23 > #4 0x7f75d4db543a in rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >& > rapidjson::GenericDocument<rapidjson::UTF8<char>, > rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::Parse<0u>(char > const*) ../thirdparty/installed/common/include/rapidjson/document.h:745:10 > #5 0x7f75d4db1db9 in kudu::JsonReader::Init() > ../src/kudu/util/jsonreader.cc:65:13 > #6 0x53aa5a in > kudu::JsonReaderTest_SignedAndUnsignedInts_Test::TestBody() > ../src/kudu/util/jsonreader-test.cc:159:3 > #7 0x7f75d2cc3f9c in void > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402 > #8 0x7f75d2cc3f9c in void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 > #9 0x7f75d2cb9621 in testing::Test::Run() > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2474 > #10 0x7f75d2cb9703 in testing::TestInfo::Run() > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2656 > #11 0x7f75d2cb9846 in testing::TestCase::Run() > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2774 > #12 0x7f75d2cb9d17 in testing::internal::UnitTestImpl::RunAllTests() > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4649 > #13 0x7f75d2cc447c in bool > testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2402 > #14 0x7f75d2cc447c in bool > testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, > bool>(testing::internal::UnitTestImpl*, bool > (testing::internal::UnitTestImpl::*)(), char const*) > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:2438 > #15 0x7f75d2cb9e70 in testing::UnitTest::Run() > /home/adar/Source/kudu/thirdparty/src/googletest-release-1.8.0/googletest/src/gtest.cc:4257 > #16 0x7f75d57ed2c6 in main ../src/kudu/util/test_main.cc:106:13 > #17 0x7f75d1943b96 in __libc_start_main csu/libc-start.c:310 > #18 0x4325e9 in _start > (/home/adar/Source/kudu/build/asan/bin/jsonreader-test+0x4325e9) > SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior > thirdparty/installed/common/include/rapidjson/reader.h:644:18 in > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)