[
https://issues.apache.org/jira/browse/IMPALA-14876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18071259#comment-18071259
]
Quanlong Huang commented on IMPALA-14876:
-----------------------------------------
Here are some example backtraces.
A scanner thread with name "scanner-thread
(finst:594ed4f65014cbd3:4bae840b00000001, plan-node-id:0, thread-idx:7)":
{noformat}
Thread 254 (Thread 0x7f83a0627640 (LWP 807998) "impalad"):
...
#8 0x000000000239f19f in impala::HdfsTextScanner::ProcessRange
(this=this@entry=0xac3e2c0, row_batch=row_batch@entry=0xcdf1e00,
num_tuples=num_tuples@entry=0x7f83a0625c20) at
/root/impala/be/src/exec/text/hdfs-text-scanner.cc:423
#9 0x00000000023a169b in impala::HdfsTextScanner::GetNextInternal
(this=0xac3e2c0, row_batch=0xcdf1e00) at
/root/impala/be/src/exec/text/hdfs-text-scanner.cc:490
#10 0x0000000001e5bc37 in impala::HdfsScanner::ProcessSplit (this=0xac3e2c0) at
/root/impala/toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/include/c++/10.4.0/bits/unique_ptr.h:173
#11 0x0000000001f62ca4 in impala::HdfsScanNode::ProcessSplit
(this=this@entry=0xf6a8800, filter_ctxs=...,
expr_results_pool=expr_results_pool@entry=0x7f83a06261c0,
scan_range=scan_range@entry=0xf4b9080,
scanner_thread_reservation=scanner_thread_reservation@entry=0x7f83a0626108) at
/root/impala/be/src/exec/hdfs-scan-node.cc:504
#12 0x0000000001f63a7b in impala::HdfsScanNode::ScannerThread (this=0xf6a8800,
first_thread=false, scanner_thread_reservation=<optimized out>) at
/root/impala/be/src/exec/hdfs-scan-node.cc:422
...
#16 0x0000000001d7dd62 in impala::Thread::SuperviseThread(char const*,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
impala::Promise<long, (impala::PromiseMode)0>*) (name=0xd2248a0 "scanner-thread
(finst:594ed4f65014cbd3:4bae840b00000001, plan-node-id:0, thread-idx:7)",
category=..., functor=..., parent_thread_info=0x7f83a5630640,
thread_started=0x7f83a562f5d0) at /root/impala/be/src/util/thread.cc:390
...
#21 0x00007f847dc8b2ea in start_thread () from /lib64/libc.so.6
#22 0x00007f847dd103d0 in clone3 () from /lib64/libc.so.6{noformat}
A fragment instance thread with name "exec-finstance
(finst:594ed4f65014cbd3:4bae840b00000000)":
{noformat}
Thread 246 (Thread 0x7f83a4e30640 (LWP 807973) "impalad"):
...
#5 0x000000000170af11 in impala::KrpcDataStreamRecvr::SenderQueue::GetBatch
(this=0xef023c0, next_batch=next_batch@entry=0xea7fde0) at
/root/impala/be/src/runtime/krpc-data-stream-recvr.cc:251
#6 0x000000000170b868 in impala::KrpcDataStreamRecvr::GetBatch
(this=0xb1dcc60, next_batch=next_batch@entry=0xea7fde0) at
/root/impala/be/src/runtime/krpc-data-stream-recvr.cc:818
#7 0x0000000001e38339 in impala::ExchangeNode::FillInputRowBatch
(this=this@entry=0xea7fb80, state=state@entry=0xdc95d40) at
/root/impala/be/src/exec/exchange-node.cc:172
#8 0x0000000001e3a626 in impala::ExchangeNode::Open (this=0xea7fb80,
state=0xdc95d40) at /root/impala/be/src/exec/exchange-node.cc:149
#9 0x0000000001f75b80 in impala::AggregationNode::Open (this=0xdc95b00,
state=0xdc95d40) at /root/impala/be/src/exec/aggregation-node.cc:48
#10 0x00000000017b4da0 in impala::FragmentInstanceState::Open
(this=this@entry=0xe908340) at
/root/impala/be/src/runtime/fragment-instance-state.cc:437
#11 0x00000000017b960f in impala::FragmentInstanceState::Exec
(this=this@entry=0xe908340) at
/root/impala/be/src/runtime/fragment-instance-state.cc:106
#12 0x000000000172d305 in impala::QueryState::ExecFInstance (this=0xd480400,
fis=0xe908340) at /root/impala/be/src/runtime/query-state.cc:1013
...
#16 0x0000000001d7dd62 in impala::Thread::SuperviseThread(char const*,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
impala::Promise<long, (impala::PromiseMode)0>*) (name=0xd8a7440 "exec-finstance
(finst:594ed4f65014cbd3:4bae840b00000000)", category=..., functor=...,
parent_thread_info=0x7f83a5e31640, thread_started=0x7f83a5e2fc30) at
/root/impala/be/src/util/thread.cc:390{noformat}
More examples are uploaded in [^new-pstack.txt]
Uploaded the patch for review: https://gerrit.cloudera.org/c/24171/
> Use const char* for thread name in impala::Thread::SuperviseThread
> ------------------------------------------------------------------
>
> Key: IMPALA-14876
> URL: https://issues.apache.org/jira/browse/IMPALA-14876
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Attachments: new-pstack.txt
>
>
> impala::Thread::SuperviseThread() runs most of the backend threads. Its first
> parameter is the thread name:
> {code:cpp}
> static void SuperviseThread(const std::string& name, const std::string&
> category,
> const ThreadFunctor& functor, const ThreadDebugInfo* parent_thread_info,
> Promise<int64_t>* thread_started);{code}
> In backtraces printed in pstack/gdb, the thread name is usually omitted as
> "name=...", e.g.
> {noformat}
> #39 impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*) (name=..., category=...,
> functor=..., parent_thread_info=0x7f6bffa027f0,
> thread_started=0x7f6bffa01ac0) at thread.cc:360{noformat}
> It'd be helpful to print the thread name so we can directly find the thread
> of a given fragment instance. The first option is making the thread name be
> shown correctly:
> {noformat}
> Thread 1045 (Thread 0x7ff184860640 (LWP 1395025) "reacquire threa"):{noformat}
> However, pstack typically pulls the thread name from
> /proc/<pid>/task/<tid>/comm. Since the kernel only stores 15 characters
> there, pstack can only show 15 characters. That's not enough for fragment
> instance ids.
> The second option is changing the 'name' parameter type to const char* and
> make sure it's not optimized out. Then pstack will print its value in the
> backtrace.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
