[
https://issues.apache.org/jira/browse/IMPALA-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16476076#comment-16476076
]
Tim Armstrong commented on IMPALA-7030:
---------------------------------------
I also saw a crash in codegen'd code on the same job. I wasn't able to get a
backtrace but I reconfigured the job to collect the hs_err_pid* file in future.
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2183/
{noformat}
Crash reason: SIGSEGV
Crash address: 0x7f87bd39d000
Process uptime: not available
Thread 223 (crashed)
0 0x7f87bd39d000
rax = 0x00007f87bd39d000 rdx = 0x000000000ec12000
rcx = 0x000000000f547540 rbx = 0x0000000000000000
rsi = 0x000000000f547540 rdi = 0x000000000ec12000
rbp = 0x00007f8767575850 rsp = 0x00007f8767575658
r8 = 0x000000000f7255a0 r9 = 0x00007f87675751ef
r10 = 0x0000000017966000 r11 = 0x0000000000000020
r12 = 0x0000000000000400 r13 = 0x0000000000000011
r14 = 0x000000002d2a9000 r15 = 0x000000000f549210
rip = 0x00007f87bd39d000
Found by: given as instruction pointer in context
1 impalad + 0x1961f63
rbp = 0x00007f8767575b20 rsp = 0x00007f8767575860
rip = 0x0000000001d61f63
Found by: previous frame's frame pointer
2 impalad + 0x195e6cf
rbp = 0x00007f87675760e0 rsp = 0x00007f8767575b30
rip = 0x0000000001d5e6cf
Found by: previous frame's frame pointer
3 impalad + 0x195c89e
rbp = 0x00007f87675761a0 rsp = 0x00007f87675760f0
rip = 0x0000000001d5c89e
Found by: previous frame's frame pointer
4 impalad + 0x18e42c8
rbp = 0x00007f8767576510 rsp = 0x00007f87675761b0
rip = 0x0000000001ce42c8
Found by: previous frame's frame pointer
5 impalad + 0x18e3669
rbp = 0x00007f87675768c0 rsp = 0x00007f8767576520
rip = 0x0000000001ce3669
Found by: previous frame's frame pointer
6 impalad + 0x18e2adc
rbp = 0x00007f87675768e0 rsp = 0x00007f87675768d0
rip = 0x0000000001ce2adc
Found by: previous frame's frame pointer
Crash reason: SIGSEGV
Crash address: 0x7f87bd39d000
Process uptime: not available
Thread 223 (crashed)
0 0x7f87bd39d000
rax = 0x00007f87bd39d000 rdx = 0x000000000ec12000
rcx = 0x000000000f547540 rbx = 0x0000000000000000
rsi = 0x000000000f547540 rdi = 0x000000000ec12000
rbp = 0x00007f8767575850 rsp = 0x00007f8767575658
r8 = 0x000000000f7255a0 r9 = 0x00007f87675751ef
r10 = 0x0000000017966000 r11 = 0x0000000000000020
r12 = 0x0000000000000400 r13 = 0x0000000000000011
r14 = 0x000000002d2a9000 r15 = 0x000000000f549210
rip = 0x00007f87bd39d000
Found by: given as instruction pointer in context
1 impalad + 0x1961f63
rbp = 0x00007f8767575b20 rsp = 0x00007f8767575860
rip = 0x0000000001d61f63
Found by: previous frame's frame pointer
2 impalad + 0x195e6cf
rbp = 0x00007f87675760e0 rsp = 0x00007f8767575b30
rip = 0x0000000001d5e6cf
Found by: previous frame's frame pointer
3 impalad + 0x195c89e
rbp = 0x00007f87675761a0 rsp = 0x00007f87675760f0
rip = 0x0000000001d5c89e
Found by: previous frame's frame pointer
4 impalad + 0x18e42c8
rbp = 0x00007f8767576510 rsp = 0x00007f87675761b0
rip = 0x0000000001ce42c8
Found by: previous frame's frame pointer
5 impalad + 0x18e3669
rbp = 0x00007f87675768c0 rsp = 0x00007f8767576520
rip = 0x0000000001ce3669
Found by: previous frame's frame pointer
6 impalad + 0x18e2adc
rbp = 0x00007f87675768e0 rsp = 0x00007f87675768d0
rip = 0x0000000001ce2adc
Found by: previous frame's frame pointer
Crash reason: SIGSEGV
Crash address: 0x7f87bd39d000
Process uptime: not available
Thread 223 (crashed)
0 0x7f87bd39d000
rax = 0x00007f87bd39d000 rdx = 0x000000000ec12000
rcx = 0x000000000f547540 rbx = 0x0000000000000000
rsi = 0x000000000f547540 rdi = 0x000000000ec12000
rbp = 0x00007f8767575850 rsp = 0x00007f8767575658
r8 = 0x000000000f7255a0 r9 = 0x00007f87675751ef
r10 = 0x0000000017966000 r11 = 0x0000000000000020
r12 = 0x0000000000000400 r13 = 0x0000000000000011
r14 = 0x000000002d2a9000 r15 = 0x000000000f549210
rip = 0x00007f87bd39d000
Found by: given as instruction pointer in context
1 impalad + 0x1961f63
rbp = 0x00007f8767575b20 rsp = 0x00007f8767575860
rip = 0x0000000001d61f63
Found by: previous frame's frame pointer
2 impalad + 0x195e6cf
rbp = 0x00007f87675760e0 rsp = 0x00007f8767575b30
rip = 0x0000000001d5e6cf
Found by: previous frame's frame pointer
3 impalad + 0x195c89e
rbp = 0x00007f87675761a0 rsp = 0x00007f87675760f0
rip = 0x0000000001d5c89e
Found by: previous frame's frame pointer
4 impalad + 0x18e42c8
rbp = 0x00007f8767576510 rsp = 0x00007f87675761b0
rip = 0x0000000001ce42c8
Found by: previous frame's frame pointer
5 impalad + 0x18e3669
rbp = 0x00007f87675768c0 rsp = 0x00007f8767576520
rip = 0x0000000001ce3669
Found by: previous frame's frame pointer
6 impalad + 0x18e2adc
rbp = 0x00007f87675768e0 rsp = 0x00007f87675768d0
rip = 0x0000000001ce2adc
Found by: previous frame's frame pointer
7 impalad + 0x18e4b38
rbp = 0x00007f8767576910 rsp = 0x00007f87675768f0
rip = 0x0000000001ce4b38
Found by: previous frame's frame pointer
8 impalad + 0x1518fc0
rbp = 0x00007f8767576950 rsp = 0x00007f8767576920
rip = 0x0000000001918fc0
Found by: previous frame's frame pointer
9 impalad + 0x1836997
rbp = 0x00007f8767576cb0 rsp = 0x00007f8767576960
rip = 0x0000000001c36997
Found by: previous frame's frame pointer
10 impalad + 0x183ee6d
rbp = 0x00007f8767576d20 rsp = 0x00007f8767576cc0
rip = 0x0000000001c3ee6d
Found by: previous frame's frame pointer
11 impalad + 0x183ed91
rbp = 0x00007f8767576d70 rsp = 0x00007f8767576d30
rip = 0x0000000001c3ed91
Found by: previous frame's frame pointer
12 impalad + 0x183ed54
rbp = 0x00007f8767576d90 rsp = 0x00007f8767576d80
rip = 0x0000000001c3ed54
Found by: previous frame's frame pointer
13 impalad + 0x2b033ea
rsp = 0x00007f8767576da0 rip = 0x0000000002f033ea
Found by: stack scanning
14 libpthread-2.23.so + 0x76ba
rsp = 0x00007f8767576de0 rip = 0x00007f87fb0346ba
Found by: stack scanning
15 impalad + 0x400000
rsp = 0x00007f8767576e18 rip = 0x0000000000800000
Found by: stack scanning
16 libc-2.23.so + 0x10741d
rsp = 0x00007f8767576e80 rip = 0x00007f87fad6a41d
Found by: stack scanning
{noformat}
{noformat}
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f87bd39d000, pid=90251, tid=0x00007f8767577700
#
# JRE version: OpenJDK Runtime Environment (8.0_171-b11) (build
1.8.0_171-8u171-b11-0ubuntu0.16.04.1-b11)
# Java VM: OpenJDK 64-Bit Server VM (25.171-b11 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# C 0x00007f87bd39d000
#
# Core dump written. Default location: /home/ubuntu/Impala/core or core.90251
#
# An error report file with more information is saved as:
# /home/ubuntu/Impala/hs_err_pid90251.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
{noformat}
> crash in impala::PartitionedAggregationNode::ProcessBatchNoGrouping
> --------------------------------------------------------------------
>
> Key: IMPALA-7030
> URL: https://issues.apache.org/jira/browse/IMPALA-7030
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Michael Brown
> Priority: Blocker
> Attachments: crash.dump.gz, gdb.out.gz, hs_err_pid1621.log.gz
>
>
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2176/
> {noformat}
> #0 0x00007fc896430428 in __GI_raise (sig=sig@entry=6) at
> ../sysdeps/unix/sysv/linux/raise.c:54
> #1 0x00007fc89643202a in __GI_abort () at abort.c:89
> #2 0x00007fc899379c59 in os::abort(bool) (dump_core=<optimized out>) at
> /build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/os/linux/vm/os_linux.cpp:1509
> #3 0x00007fc89952f047 in VMError::report_and_die()
> (this=this@entry=0x7fc7e90287d0) at
> /build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/share/vm/utilities/vmError.cpp:1060
> #4 0x00007fc8993836ef in JVM_handle_linux_signal(int, siginfo_t*, void*,
> int) (sig=sig@entry=11, info=info@entry=0x7fc7e9028a70,
> ucVoid=ucVoid@entry=0x7fc7e9028940,
> abort_if_unrecognized=abort_if_unrecognized@entry=1)
> at
> /build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/os_cpu/linux_x86/vm/os_linux_x86.cpp:541
> #5 0x00007fc899376d88 in signalHandler(int, siginfo_t*, void*) (sig=11,
> info=0x7fc7e9028a70, uc=0x7fc7e9028940) at
> /build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/os/linux/vm/os_linux.cpp:4432
> #6 0x00007fc8967d6390 in <signal handler called> () at
> /lib/x86_64-linux-gnu/libpthread.so.0
> #7 0x00007fc8584ca000 in
> impala::PartitionedAggregationNode::ProcessBatchNoGrouping(impala::RowBatch*)
> [clone .1] ()
> #8 0x0000000002cd5bcf in
> impala::PartitionedAggregationNode::Open(impala::RuntimeState*)
> (this=0x15795200, state=0x14e95d40) at
> /home/ubuntu/Impala/be/src/exec/partitioned-aggregation-node.cc:314
> #9 0x0000000001c94775 in impala::FragmentInstanceState::Open()
> (this=0x1cc19e00) at
> /home/ubuntu/Impala/be/src/runtime/fragment-instance-state.cc:268
> #10 0x0000000001c91faf in impala::FragmentInstanceState::Exec()
> (this=0x1cc19e00) at
> /home/ubuntu/Impala/be/src/runtime/fragment-instance-state.cc:81
> #11 0x0000000001ca175b in
> impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> (this=0x3a1f6000, fis=0x1cc19e00) at
> /home/ubuntu/Impala/be/src/runtime/query-state.cc:401
> #12 0x0000000001c9ffce in impala::QueryState::<lambda()>::operator()(void)
> const (__closure=0x7fc7e9029ce8) at
> /home/ubuntu/Impala/be/src/runtime/query-state.cc:341
> #13 0x0000000001ca2479 in
> boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::<lambda()>,
> void>::invoke(boost::detail::function::function_buffer &)
> (function_obj_ptr=...)
> at
> /home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #14 0x0000000001bd9e58 in boost::function0<void>::operator()() const
> (this=0x7fc7e9029ce0) at
> /home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #15 0x0000000001ec50a9 in impala::Thread::SuperviseThread(std::string const&,
> std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long>*) (name="exec-finstance
> (finst:844ac2a49a9a6486:396846db00000005)", category="fragment-execution",
> functor=..., parent_thread_info=0x7fc7ea82c990,
> thread_started=0x7fc7ea82b940) at
> /home/ubuntu/Impala/be/src/util/thread.cc:356
> #16 0x0000000001ecd245 in boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string
> const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo
> const*, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>,
> void (*&)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long>*), boost::_bi::list0&,
> int) (this=0xb84afc0, f=@0xb84afb8: 0x1ec4d42
> <impala::Thread::SuperviseThread(std::string const&, std::string const&,
> boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long>*)>, a=...)
> at
> /home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #17 0x0000000001ecd169 in boost::_bi::bind_t<void, void (*)(std::string
> const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo
> const*, impala::Promise<long>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0xb84afb8)
> at
> /home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
> #18 0x0000000001ecd12c in boost::detail::thread_data<boost::_bi::bind_t<void,
> void (*)(std::string const&, std::string const&, boost::function<void ()>,
> impala::ThreadDebugInfo const*, impala::Promise<long>*),
> boost::_bi::list5<boost::_bi::value<std::string>,
> boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >,
> boost::_bi::value<impala::ThreadDebugInfo*>,
> boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0xb84ae00) at
> /home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
> #19 0x00000000031bc0ea in thread_proxy ()
> #20 0x00007fc8967cc6ba in start_thread (arg=0x7fc7e902a700) at
> pthread_create.c:333
> #21 0x00007fc89650241d in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
