[ https://issues.apache.org/jira/browse/IMPALA-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030031#comment-17030031 ]
Sahil Takiar commented on IMPALA-9314: -------------------------------------- There is a thread leak as well: {code:java} 58/114 Test #58: statestore-test ..................***Failed 33.76 sec [==========] Running 3 tests from 2 test cases. [----------] Global test environment set-up. [----------] 1 test from StatestoreTest [ RUN ] StatestoreTest.SmokeTest [ OK ] StatestoreTest.SmokeTest (2076 ms) [----------] 1 test from StatestoreTest (2076 ms total)[----------] 2 tests from StatestoreSslTest [ RUN ] StatestoreSslTest.ValidCertSmokeTest [ OK ] StatestoreSslTest.ValidCertSmokeTest (2044 ms) [ RUN ] StatestoreSslTest.InvalidCertSmokeTest [ OK ] StatestoreSslTest.InvalidCertSmokeTest (27041 ms) [----------] 2 tests from StatestoreSslTest (29085 ms total)[----------] Global test environment tear-down [==========] 3 tests from 2 test cases ran. (31162 ms total) [ PASSED ] 3 tests. ================== WARNING: ThreadSanitizer: thread leak (pid=122454) Thread T39 (tid=122496, finished) created by main thread at: #0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (statestore-test+0x1899d9b) #1 boost::thread::start_thread_noexcept() <null> (statestore-test+0x2f3bf19) #2 boost::thread::thread<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala: :Promise<long, (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugIn fo*, impala::Promise<long, (impala::PromiseMode)0>*) /home/systest/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13 (statestore-test+0x1eceaeb) #3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /home/systest/Impala/be/src/util/thread.cc:317:13 (statestore-test+0x1ecbdd5) #4 impala::Status impala::Thread::Create<void (impala::ThriftServer::ThriftServerEventProcessor::*)(), impala::ThriftServer::ThriftServerEventProcessor*>(std::string const&, std::string const&, void (impala::ThriftServer::ThriftServerEventProcessor::* const&)(), imp ala::ThriftServer::ThriftServerEventProcessor* const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /home/systest/Impala/be/src/util/thread.h:81:12 (statestore-test+0x27158c7) #5 impala::ThriftServer::ThriftServerEventProcessor::StartAndWaitForServer() /home/systest/Impala/be/src/rpc/thrift-server.cc:116:43 (statestore-test+0x2712b22) #6 impala::ThriftServer::Start() /home/systest/Impala/be/src/rpc/thrift-server.cc:447:60 (statestore-test+0x27147fc) #7 impala::Statestore::Init(int) /home/systest/Impala/be/src/statestore/statestore.cc:478:59 (statestore-test+0x1d88771) #8 impala::StatestoreTest_SmokeTest_Test::TestBody() /home/systest/Impala/be/src/statestore/statestore-test.cc:55:132 (statestore-test+0x1904d61) #9 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (statestore-test+0x4122232) #10 __libc_start_main <null> (libc.so.6+0x2082f)SUMMARY: ThreadSanitizer: thread leak (/home/systest/Impala/be/build/debug/statestore/statestore-test+0x2f3bf19) in boost::thread::start_thread_noexcept() ================== {code} > Thrift server shutdown is racey > ------------------------------- > > Key: IMPALA-9314 > URL: https://issues.apache.org/jira/browse/IMPALA-9314 > Project: IMPALA > Issue Type: Sub-task > Components: Backend > Reporter: Sahil Takiar > Priority: Major > > There are multiple issues with shutting down the {{ThriftServer}}: > * The docs for {{ThriftServer}} say "TODO: shutdown is buggy (which only > harms tests)" > * TSAN reports a thread leak when using the {{ThriftServer}} in > {{statestore-test}} > ** Attempts to shutdown the {{ThriftServer}} in the {{Statestore}} don't > seem to work properly > * The {{TAcceptQueueServer}} actually uses a {{volatile}} {{boolean}} called > {{stop_}} to coordinate the shutdown (see {{TAcceptQueueServer::stop()}}) > which is not-thread safe (TSAN complains about this as well) > According to the docs this only affects test, specifically > {{statestore-test}}. We should consider whether it is worth fixing the > shutdown logic in the {{ThriftServer}}. > {code:java} > WARNING: ThreadSanitizer: thread leak (pid=67752) > Thread T39 (tid=67794, finished) created by main thread at: > #0 pthread_create > /mnt/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 > (statestore-test+0x18903bb) > #1 boost::thread::start_thread_noexcept() <null> > (statestore-test+0x2f30b59) > #2 boost::thread::thread<void (*)(std::string const&, std::string const&, > boost::function<void ()>, impala::ThreadDebugInfo const*, > impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, > boost::function<void ()>, impala::ThreadDebugInfo*, impala: > :Promise<long, (impala::PromiseMode)0>*>(void (*)(std::string const&, > std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, > impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, > boost::function<void ()>, impala::ThreadDebugIn > fo*, impala::Promise<long, (impala::PromiseMode)0>*) > /home/systest/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13 > (statestore-test+0x1ebf8eb) > #3 impala::Thread::StartThread(std::string const&, std::string const&, > boost::function<void ()> const&, std::unique_ptr<impala::Thread, > std::default_delete<impala::Thread> >*, bool) > /home/systest/Impala/be/src/util/thread.cc:317:13 (statestore-test+0x1ebcbd5) > #4 impala::Status impala::Thread::Create<void > (impala::ThriftServer::ThriftServerEventProcessor::*)(), > impala::ThriftServer::ThriftServerEventProcessor*>(std::string const&, > std::string const&, void (impala::ThriftServer::ThriftServerEventProcessor::* > const&)(), imp > ala::ThriftServer::ThriftServerEventProcessor* const&, > std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) > /home/systest/Impala/be/src/util/thread.h:81:12 (statestore-test+0x2707497) > #5 > impala::ThriftServer::ThriftServerEventProcessor::StartAndWaitForServer() > /home/systest/Impala/be/src/rpc/thrift-server.cc:116:43 > (statestore-test+0x27046f2) > #6 impala::ThriftServer::Start() > /home/systest/Impala/be/src/rpc/thrift-server.cc:447:60 > (statestore-test+0x27063cc) > #7 impala::Statestore::Init(int) > /home/systest/Impala/be/src/statestore/statestore.cc:478:59 > (statestore-test+0x1d7a431) > #8 impala::StatestoreTest_SmokeTest_Test::TestBody() > /home/systest/Impala/be/src/statestore/statestore-test.cc:55:132 > (statestore-test+0x18fb381) > #9 void > testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, > void>(testing::Test*, void (testing::Test::*)(), char const*) <null> > (statestore-test+0x41051e2) > #10 __libc_start_main > /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291 > (libc.so.6+0x2082f){code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org