[ https://issues.apache.org/jira/browse/IMPALA-8090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-8090. ----------------------------------- Resolution: Fixed Fix Version/s: Impala 3.2.0 > DiskIoMgrTest.SyncReadTest hits file_ != nullptr DCHECK in > LocalFileReader::ReadFromPos() > ----------------------------------------------------------------------------------------- > > Key: IMPALA-8090 > URL: https://issues.apache.org/jira/browse/IMPALA-8090 > Project: IMPALA > Issue Type: Bug > Components: Backend > Affects Versions: Impala 3.2.0 > Reporter: David Knupp > Assignee: Tim Armstrong > Priority: Critical > Fix For: Impala 3.2.0 > > > *Test output*: > {noformat} > 45/99 Test #45: disk-io-mgr-test .................***Exception: Other 43.29 > sec > Turning perftools heap leak checking off > [==========] Running 25 tests from 1 test case. > [----------] Global test environment set-up. > [----------] 25 tests from DiskIoMgrTest > [ RUN ] DiskIoMgrTest.SingleWriter > 19/01/16 15:57:09 INFO util.JvmPauseMonitor: Starting JVM pause monitor > [ OK ] DiskIoMgrTest.SingleWriter (3407 ms) > [ RUN ] DiskIoMgrTest.InvalidWrite > [ OK ] DiskIoMgrTest.InvalidWrite (281 ms) > [ RUN ] DiskIoMgrTest.WriteErrors > [ OK ] DiskIoMgrTest.WriteErrors (235 ms) > [ RUN ] DiskIoMgrTest.SingleWriterCancel > [ OK ] DiskIoMgrTest.SingleWriterCancel (1165 ms) > [ RUN ] DiskIoMgrTest.SingleReader > [ OK ] DiskIoMgrTest.SingleReader (5835 ms) > [ RUN ] DiskIoMgrTest.SingleReaderSubRanges > [ OK ] DiskIoMgrTest.SingleReaderSubRanges (16404 ms) > [ RUN ] DiskIoMgrTest.AddScanRangeTest > [ OK ] DiskIoMgrTest.AddScanRangeTest (1210 ms) > [ RUN ] DiskIoMgrTest.SyncReadTest > *** Check failure stack trace: *** > @ 0x4825dcc > @ 0x4827671 > @ 0x48257a6 > @ 0x4828d6d > @ 0x1af39ec > @ 0x1ae90a4 > @ 0x1ac30ea > @ 0x1accad3 > @ 0x1acc660 > @ 0x1acbf3e > @ 0x1acb62d > @ 0x1b03671 > @ 0x1f79988 > @ 0x1f82b60 > @ 0x1f82a84 > @ 0x1f82a47 > @ 0x3751579 > @ 0x3ea4807850 > @ 0x3ea44e894c > Wrote minidump to > /data/jenkins/workspace/<...>/repos/Impala/logs/be_tests/minidumps/disk-io-mgr-test/5bbf76f7-e5d6-4ac9-bdae9d9b-065c32ec.dmp > {noformat} > *Error*: > {noformat} > Operating system: Linux > 0.0.0 Linux 2.6.32-358.14.1.el6.centos.plus.x86_64 #1 SMP > Tue Jul 16 21:33:24 UTC 2013 x86_64 > CPU: amd64 > family 6 model 45 stepping 7 > 8 CPUs > GPU: UNKNOWN > Crash reason: SIGABRT > Crash address: 0x45200002fa1 > Process uptime: not available > Thread 205 (crashed) > 0 libc-2.12.so + 0x328e5 > rax = 0x0000000000000000 rdx = 0x0000000000000006 > rcx = 0xffffffffffffffff rbx = 0x0000000006adf9c0 > rsi = 0x0000000000000563 rdi = 0x0000000000002fa1 > rbp = 0x00007f8009b8ffe0 rsp = 0x00007f8009b8fc78 > r8 = 0x00007f8009b8fd00 r9 = 0x0000000000000563 > r10 = 0x0000000000000008 r11 = 0x0000000000000202 > r12 = 0x0000000006adfa40 r13 = 0x000000000000001f > r14 = 0x0000000006ae7384 r15 = 0x0000000006adf9c0 > rip = 0x0000003ea44328e5 > Found by: given as instruction pointer in context > 1 libc-2.12.so + 0x340c5 > rbp = 0x00007f8009b8ffe0 rsp = 0x00007f8009b8fc80 > rip = 0x0000003ea44340c5 > Found by: stack scanning > 2 disk-io-mgr-test!boost::_bi::bind_t<void, boost::_mfi::mf1<void, > impala::io::DiskQueue, impala::io::DiskIoMgr*>, > boost::_bi::list2<boost::_bi::value<impala::io::DiskQueue*>, > boost::_bi::value<impala::io::DiskIoMgr*> > >::operator()() > [bind_template.hpp : 20 + 0x21] > rbp = 0x00007f8009b8ffe0 rsp = 0x00007f8009b8fc88 > rip = 0x0000000001acbf3e > Found by: stack scanning > 3 disk-io-mgr-test!google::LogMessage::Flush() + 0x157 > rbx = 0x0000000000000007 rbp = 0x0000000006adf980 > rsp = 0x00007f8009b8fff0 rip = 0x00000000048257a7 > Found by: call frame info > 4 disk-io-mgr-test!google::LogMessageFatal::~LogMessageFatal() + 0xe > rbx = 0x00007f8009b90110 rbp = 0x00007f8009b903f0 > rsp = 0x00007f8009b90070 r12 = 0x0000000000000001 > r13 = 0x0000000006aee8b8 r14 = 0x000000000c213538 > r15 = 0x0000000000000007 rip = 0x0000000004828d6e > Found by: call frame info > 5 disk-io-mgr-test!impala::io::LocalFileReader::ReadFromPos(long, unsigned > char*, long, long*, bool*) [local-file-reader.cc : 67 + 0x10] > rbx = 0x0000000000000001 rbp = 0x00007f8009b903f0 > rsp = 0x00007f8009b90090 r12 = 0x0000000000000001 > r13 = 0x0000000006aee8b8 r14 = 0x000000000c213538 > r15 = 0x0000000000000007 rip = 0x0000000001af39ed > Found by: call frame info > 6 disk-io-mgr-test!impala::io::ScanRange::DoRead(int) [scan-range.cc : 219 > + 0x5b] > rbx = 0x000000000c4f71e0 rbp = 0x00007f8009b90620 > rsp = 0x00007f8009b90400 r12 = 0x0000000001af36e4 > r13 = 0x000000000000000d r14 = 0x000000000c213538 > r15 = 0x0000000000000007 rip = 0x0000000001ae90a5 > Found by: call frame info > 7 > disk-io-mgr-test!impala::io::DiskQueue::DiskThreadLoop(impala::io::DiskIoMgr*) > [disk-io-mgr.cc : 425 + 0x17] > rbx = 0x000000000c0e0f00 rbp = 0x00007f8009b906c0 > rsp = 0x00007f8009b90630 r12 = 0x00007fff99de21c0 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001ac30eb > Found by: call frame info > 8 disk-io-mgr-test!boost::_mfi::mf1<void, impala::io::DiskQueue, > impala::io::DiskIoMgr*>::operator()(impala::io::DiskQueue*, > impala::io::DiskIoMgr*) const [mem_fn_template.hpp : 165 + 0x1e] > rbx = 0x000000000c0e0f00 rbp = 0x00007f8009b906f0 > rsp = 0x00007f8009b906d0 r12 = 0x00007fff99de21c0 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001accad4 > Found by: call frame info > 9 disk-io-mgr-test!void > boost::_bi::list2<boost::_bi::value<impala::io::DiskQueue*>, > boost::_bi::value<impala::io::DiskIoMgr*> > >::operator()<boost::_mfi::mf1<void, impala::io::DiskQueue, > impala::io::DiskIoMgr*>, boost::_bi::list0>(boost::_bi::type<void>, > boost::_mfi::mf1<void, impala::io::DiskQueue, impala::io::DiskIoMgr*>&, > boost::_bi::list0&, int) [bind.hpp : 313 + 0x4f] > rbx = 0x000000000c0e0f00 rbp = 0x00007f8009b90730 > rsp = 0x00007f8009b90700 r12 = 0x00007fff99de21c0 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001acc661 > Found by: call frame info > 10 disk-io-mgr-test!boost::_bi::bind_t<void, boost::_mfi::mf1<void, > impala::io::DiskQueue, impala::io::DiskIoMgr*>, > boost::_bi::list2<boost::_bi::value<impala::io::DiskQueue*>, > boost::_bi::value<impala::io::DiskIoMgr*> > >::operator()() > [bind_template.hpp : 20 + 0x22] > rbx = 0x0000000000000563 rbp = 0x00007f8009b90780 > rsp = 0x00007f8009b90740 r12 = 0x0000000000000000 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001acbf3f > Found by: call frame info > 11 > disk-io-mgr-test!boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, > boost::_mfi::mf1<void, impala::io::DiskQueue, impala::io::DiskIoMgr*>, > boost::_bi::list2<boost::_bi::value<impala::io::DiskQueue*>, > boost::_bi::value<impala::io::DiskIoMgr*> > >, > void>::invoke(boost::detail::function::function_buffer&) > [function_template.hpp : 153 + 0xc] > rbx = 0x0000000000000563 rbp = 0x00007f8009b907b0 > rsp = 0x00007f8009b90790 r12 = 0x0000000000000000 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001acb62e > Found by: call frame info > 12 disk-io-mgr-test!boost::function0<void>::operator()() const > [function_template.hpp : 767 + 0x11] > rbx = 0x0000000000000563 rbp = 0x00007f8009b907f0 > rsp = 0x00007f8009b907c0 r12 = 0x0000000000000000 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001b03672 > Found by: call frame info > 13 disk-io-mgr-test!impala::Thread::SuperviseThread(std::string const&, > std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, > impala::Promise<long, (impala::PromiseMode)0>*) [thread.cc : 359 + 0xf] > rbx = 0x0000000000000563 rbp = 0x00007f8009b90b50 > rsp = 0x00007f8009b90800 r12 = 0x0000000000000000 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > r15 = 0x0000000000000007 rip = 0x0000000001f79989 > Found by: call frame info > 14 disk-io-mgr-test!void boost::_bi::list5<boost::_bi::value<std::string>, > boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, > boost::_bi::value<impala::ThreadDebugInfo*>, > boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()<void (*)(std::string const&, std::string const&, > boost::function<void ()>, impala::ThreadDebugInfo const*, > impala::Promise<long, (impala::PromiseMode)0>*), > boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, > std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, > impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) > [bind.hpp : 525 + 0x15] > rbx = 0x0000000001f79622 rbp = 0x00007f8009b90bc0 > rsp = 0x00007f8009b90b60 r12 = 0x0000000000000000 > r13 = 0x00007fff99de1a90 r14 = 0x000000000c4f5dc8 > {noformat} > It can be reproduced by looping the test for a few minutes, e.g. > {noformat} > $ ninja -j $IMPALA_BUILD_THREADS disk-io-mgr-test && while > ./be/build/latest/runtime/io/disk-io-mgr-test --gtest_filter=*SyncReadTest*; > do date; don > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org