[
https://issues.apache.org/jira/browse/ARROW-17119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17568575#comment-17568575
]
Jin Chengcheng commented on ARROW-17119:
----------------------------------------
Yeah, I agree, this is a good fix as configuring the JNI library to link Arrow
dynamically.
Because I don't need the JNI library now, I will just remove this lib.
I will configure the JNI library to link Arrow dynamically if needed next time.
> [C++] Invalid free at shutdown in Asof node static data
> -------------------------------------------------------
>
> Key: ARROW-17119
> URL: https://issues.apache.org/jira/browse/ARROW-17119
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: ubuntu20
> Reporter: Jin Chengcheng
> Assignee: Jin Chengcheng
> Priority: Major
>
> When I run [gluten|[oap-project/gluten
> (github.com)|https://github.com/oap-project/gluten]] project google test, it
> will show a error message after all the simple tests passed.
> {code:java}
> gluten/cpp/build/src# ./exec_backend_test
> Running main() from
> /build/googletest-j5yxiC/googletest-1.10.0/googletest/src/gtest_main.cc
> [==========] Running 2 tests from 1 test suite.
> [----------] Global test environment set-up.
> [----------] 2 tests from TestExecBackend
> [ RUN ] TestExecBackend.CreateBackend
> Set backend factory.
> [ OK ] TestExecBackend.CreateBackend (0 ms)
> [ RUN ] TestExecBackend.GetResultIterator
> [ OK ] TestExecBackend.GetResultIterator (0 ms)
> [----------] 2 tests from TestExecBackend (0 ms total)[----------] Global
> test environment tear-down
> [==========] 2 tests from 1 test suite ran. (0 ms total)
> [ PASSED ] 2 tests.
> corrupted size vs. prev_size in fastbins
> Aborted (core dumped)
> {code}
> I use valgrind to detect, here is the details
> {code:java}
> // code placeholder
> ==32256== Invalid read of size 8
> ==32256== at 0x5E493B7: std::set<std::shared_ptr<arrow::DataType>,
> std::less<std::shared_ptr<arrow::DataType> >,
> std::allocator<std::shared_ptr<arrow::DataType> > >::~set() (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x77E0FDD: __cxa_finalize (cxa_finalize.c:83)
> ==32256== by 0x5955816: ??? (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x4011F6A: _dl_fini (dl-fini.c:138)
> ==32256== by 0x77E08A6: __run_exit_handlers (exit.c:108)
> ==32256== by 0x77E0A5F: exit (exit.c:139)
> ==32256== by 0x77BE089: (below main) (libc-start.c:342)
> ==32256== Address 0xd984680 is 16 bytes inside a block of size 48 free'd
> ==32256== at 0x483CFBF: operator delete(void*) (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==32256== by 0x5E493CD: std::set<std::shared_ptr<arrow::DataType>,
> std::less<std::shared_ptr<arrow::DataType> >,
> std::allocator<std::shared_ptr<arrow::DataType> > >::~set() (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x77E0FDD: __cxa_finalize (cxa_finalize.c:83)
> ==32256== by 0x7FF65B6: ??? (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow_dataset_jni.so.900.0.0)
> ==32256== by 0x4011F6A: _dl_fini (dl-fini.c:138)
> ==32256== by 0x77E08A6: __run_exit_handlers (exit.c:108)
> ==32256== by 0x77E0A5F: exit (exit.c:139)
> ==32256== by 0x77BE089: (below main) (libc-start.c:342)
> ==32256== Block was alloc'd at
> ==32256== at 0x483BE63: operator new(unsigned long) (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==32256== by 0x5E4E5E9: std::set<std::shared_ptr<arrow::DataType>,
> std::less<std::shared_ptr<arrow::DataType> >,
> std::allocator<std::shared_ptr<arrow::DataType> >
> >::set(std::initializer_list<std::shared_ptr<arrow::DataType> >,
> std::less<std::shared_ptr<arrow::DataType> > const&,
> std::allocator<std::shared_ptr<arrow::DataType> > const&) (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x7FF4CC4: _GLOBAL__sub_I_asof_join_node.cc (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow_dataset_jni.so.900.0.0)
> ==32256== by 0x4011B99: call_init.part.0 (dl-init.c:72)
> ==32256== by 0x4011CA0: call_init (dl-init.c:30)
> ==32256== by 0x4011CA0: _dl_init (dl-init.c:119)
> ==32256== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so)
> ==32256==
> ==32256== Invalid free() / delete / delete[] / realloc()
> ==32256== at 0x483CFBF: operator delete(void*) (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==32256== by 0x5E493CD: std::set<std::shared_ptr<arrow::DataType>,
> std::less<std::shared_ptr<arrow::DataType> >,
> std::allocator<std::shared_ptr<arrow::DataType> > >::~set() (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x77E0FDD: __cxa_finalize (cxa_finalize.c:83)
> ==32256== by 0x5955816: ??? (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x4011F6A: _dl_fini (dl-fini.c:138)
> ==32256== by 0x77E08A6: __run_exit_handlers (exit.c:108)
> ==32256== by 0x77E0A5F: exit (exit.c:139)
> ==32256== by 0x77BE089: (below main) (libc-start.c:342)
> ==32256== Address 0xd984670 is 0 bytes inside a block of size 48 free'd
> ==32256== at 0x483CFBF: operator delete(void*) (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==32256== by 0x5E493CD: std::set<std::shared_ptr<arrow::DataType>,
> std::less<std::shared_ptr<arrow::DataType> >,
> std::allocator<std::shared_ptr<arrow::DataType> > >::~set() (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x77E0FDD: __cxa_finalize (cxa_finalize.c:83)
> ==32256== by 0x7FF65B6: ??? (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow_dataset_jni.so.900.0.0)
> ==32256== by 0x4011F6A: _dl_fini (dl-fini.c:138)
> ==32256== by 0x77E08A6: __run_exit_handlers (exit.c:108)
> ==32256== by 0x77E0A5F: exit (exit.c:139)
> ==32256== by 0x77BE089: (below main) (libc-start.c:342)
> ==32256== Block was alloc'd at
> ==32256== at 0x483BE63: operator new(unsigned long) (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==32256== by 0x5E4E5E9: std::set<std::shared_ptr<arrow::DataType>,
> std::less<std::shared_ptr<arrow::DataType> >,
> std::allocator<std::shared_ptr<arrow::DataType> >
> >::set(std::initializer_list<std::shared_ptr<arrow::DataType> >,
> std::less<std::shared_ptr<arrow::DataType> > const&,
> std::allocator<std::shared_ptr<arrow::DataType> > const&) (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow.so.900.0.0)
> ==32256== by 0x7FF4CC4: _GLOBAL__sub_I_asof_join_node.cc (in
> /mnt/jcc/code/gluten/cpp/build/releases/libarrow_dataset_jni.so.900.0.0)
> ==32256== by 0x4011B99: call_init.part.0 (dl-init.c:72)
> ==32256== by 0x4011CA0: call_init (dl-init.c:30)
> ==32256== by 0x4011CA0: _dl_init (dl-init.c:119)
> ==32256== by 0x4001139: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.31.so)
> {code}
> I found it is static std::set in file {color:#FF0000}asof_join_node.cc{color}
> {color:#172b4d}Can you help me to check if it has invalid memory free in your
> convienent?{color}
> {color:#172b4d}Thank you very much!
> {color}
> {code:java}
> private:
> static const std::set<std::shared_ptr<DataType>> kSupportedOnTypes_;
> static const std::set<std::shared_ptr<DataType>> kSupportedByTypes_;
> static const std::set<std::shared_ptr<DataType>> kSupportedDataTypes_;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
