I ended up just rebuilding arrow with debug symbols. It is indeed a duplicate of https://issues.apache.org/jira/browse/ARROW-9530
==13317== Memcheck, a memory error detector ==13317== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==13317== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==13317== Command: ./app/example/example ==13317== Hello World! ==13317== ==13317== HEAP SUMMARY: ==13317== in use at exit: 304 bytes in 1 blocks ==13317== total heap usage: 13 allocs, 12 frees, 75,181 bytes allocated ==13317== ==13317== 304 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==13317== at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==13317== by 0x40149CA: allocate_dtv (dl-tls.c:286) ==13317== by 0x40149CA: _dl_allocate_tls (dl-tls.c:532) ==13317== by 0x5A70322: allocate_stack (allocatestack.c:622) ==13317== by 0x5A70322: pthread_create@@GLIBC_2.2.5 (pthread_create.c:660) ==13317== by 0x554B443: je_arrow_private_je_pthread_create_wrapper (background_thread.c:48) ==13317== by 0x554B443: background_thread_create_signals_masked (background_thread.c:365) ==13317== by 0x554B443: background_thread_create_locked (background_thread.c:573) ==13317== by 0x554B63B: je_arrow_private_je_background_thread_create (background_thread.c:598) ==13317== by 0x4011B89: call_init.part.0 (dl-init.c:72) ==13317== by 0x4011C90: call_init (dl-init.c:30) ==13317== by 0x4011C90: _dl_init (dl-init.c:119) ==13317== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so) ==13317== ==13317== LEAK SUMMARY: ==13317== definitely lost: 0 bytes in 0 blocks ==13317== indirectly lost: 0 bytes in 0 blocks ==13317== possibly lost: 304 bytes in 1 blocks ==13317== still reachable: 0 bytes in 0 blocks ==13317== suppressed: 0 bytes in 0 blocks ==13317== ==13317== For lists of detected and suppressed errors, rerun with: -s ==13317== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) On Fri, Aug 13, 2021 at 6:32 PM James Van Alstine <[email protected]> wrote: > Is there a libarrow deb with debug symbols so I can resolve the (???)s > valgrind returns? > > On Fri, Aug 13, 2021 at 5:34 PM James Van Alstine <[email protected]> > wrote: > >> Yes, >> >> For you can repro: >> >> Program: >> ``` >> #include <iostream> >> >> int main() { >> std::cout << "Hello World!" << std::endl; >> >> return 0; >> } >> ``` >> Executable: >> ``` >> clang++-10 -g -Wall -O2 -l:libarrow.so main.cc -o hello >> ``` >> >> with --leak-check=full >> ``` >> ind --leak-check=full ./hello >> ==6456== Memcheck, a memory error detector >> ==6456== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. >> ==6456== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright >> info >> ==6456== Command: ./hello >> ==6456== >> Hello World! >> ==6456== >> ==6456== HEAP SUMMARY: >> ==6456== in use at exit: 7,094 bytes in 100 blocks >> ==6456== total heap usage: 3,550 allocs, 3,450 frees, 347,875 bytes >> allocated >> ==6456== >> ==6456== 368 bytes in 1 blocks are possibly lost in loss record 12 of 15 >> ==6456== at 0x483DD99: calloc (in >> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) >> ==6456== by 0x40149CA: allocate_dtv (dl-tls.c:286) >> ==6456== by 0x40149CA: _dl_allocate_tls (dl-tls.c:532) >> ==6456== by 0x6A1D322: allocate_stack (allocatestack.c:622) >> ==6456== by 0x6A1D322: pthread_create@@GLIBC_2.2.5 >> (pthread_create.c:660) >> ==6456== by 0x58E5AB5: ??? (in >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) >> ==6456== by 0x58E6B62: ??? (in >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) >> ==6456== by 0x58E6CEB: ??? (in >> /usr/lib/x86_64-linux-gnu/libarrow.so.500.0.0) >> ==6456== by 0x4011B89: call_init.part.0 (dl-init.c:72) >> ==6456== by 0x4011C90: call_init (dl-init.c:30) >> ==6456== by 0x4011C90: _dl_init (dl-init.c:119) >> ==6456== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so) >> ==6456== >> ==6456== LEAK SUMMARY: >> ==6456== definitely lost: 0 bytes in 0 blocks >> ==6456== indirectly lost: 0 bytes in 0 blocks >> ==6456== possibly lost: 368 bytes in 1 blocks >> ==6456== still reachable: 6,726 bytes in 99 blocks >> ==6456== suppressed: 0 bytes in 0 blocks >> ==6456== Reachable blocks (those to which a pointer was found) are not >> shown. >> ==6456== To see them, rerun with: --leak-check=full --show-leak-kinds=all >> ==6456== >> ==6456== For lists of detected and suppressed errors, rerun with: -s >> ==6456== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) >> ``` >> >> If by `ARROW_DEFAULT_MEMORY_POOL=system` you mean setting that as an env >> var and then running the same executable, then >> the valgrind output is the same. >> >> I think what is happening here is glibc is running _dl_init() when >> linking, which is kicking off the constructor of some global object in >> Arrow. >> >> >> On Fri, Aug 13, 2021 at 5:20 PM Weston Pace <[email protected]> >> wrote: >> >>> Can you try running your program with ARROW_DEFAULT_MEMORY_POOL=system >>> >>> I am wondering if you are encountering >>> https://issues.apache.org/jira/browse/ARROW-9530 >>> >>> Otherwise, can you run with `--leak-check=full` which should give us a >>> hint where the possibly lost memory is coming from? >>> >>> On Fri, Aug 13, 2021 at 11:52 AM James Van Alstine >>> <[email protected]> wrote: >>> > >>> > Greetings, >>> > >>> > When running a simple C++ hello world program, simply linking against >>> Arrow produces a valgrind report that looks like the one below. Obviously, >>> there's some constructors allocating memory before main(). Is this >>> intentional like the allocators in STL? If it is, is there some way to free >>> whatever is being allocated? If not, then how do people typically profile >>> their program's allocations when using Arrow? >>> > >>> > ==1768== Memcheck, a memory error detector >>> > ==1768== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et >>> al. >>> > ==1768== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright >>> info >>> > ==1768== Command: ./hello >>> > ==1768== >>> > Hello World! >>> > ==1768== >>> > ==1768== HEAP SUMMARY: >>> > ==1768== in use at exit: 7,094 bytes in 100 blocks >>> > ==1768== total heap usage: 3,550 allocs, 3,450 frees, 347,875 bytes >>> allocated >>> > ==1768== >>> > ==1768== LEAK SUMMARY: >>> > ==1768== definitely lost: 0 bytes in 0 blocks >>> > ==1768== indirectly lost: 0 bytes in 0 blocks >>> > ==1768== possibly lost: 368 bytes in 1 blocks >>> > ==1768== still reachable: 6,726 bytes in 99 blocks >>> > ==1768== suppressed: 0 bytes in 0 blocks >>> > ==1768== Rerun with --leak-check=full to see details of leaked memory >>> > ==1768== >>> > ==1768== For lists of detected and suppressed errors, rerun with: -s >>> > ==1768== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) >>> >>
