acelyc111 opened a new issue #5308:
URL: https://github.com/apache/incubator-doris/issues/5308
**Describe the bug**
I found some BE's compactiom score metric not update anymore, I doubt the
compaction not works, and use `pstack` to show all threads, there is no
"compaction" words except the compaction producer thread.
And then use `gcore` to dump the process, the producer's stack:
```
(gdb) t 260
[Switching to thread 260 (Thread 0x7f499f6fd700 (LWP 5470))]
#0 0x00007f4a21bd36d5 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
(gdb) bt
#0 0x00007f4a21bd36d5 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x000000000263a23c in __gthread_cond_wait (__mutex=<optimized out>,
__cond=__cond@entry=0x4cf3e88) at
/var/local/gcc/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:864
#2 std::condition_variable::wait (this=this@entry=0x4cf3e88, __lock=...) at
../../../.././libstdc++-v3/src/c++11/condition_variable.cc:53
#3 0x0000000000e61bf3 in
wait<doris::CompactionPermitLimiter::request(int64_t)::<lambda()> > (__p=...,
__lock=..., this=0x4cf3e88) at /usr/include/c++/7.3.0/condition_variable:99
#4 doris::CompactionPermitLimiter::request (this=this@entry=0x4cf3e50,
permits=permits@entry=2792) at
/root/doris-env/doris/be/src/olap/compaction_permit_limiter.cpp:30
#5 0x0000000000dd28be in
doris::StorageEngine::_compaction_tasks_producer_callback (this=0x4cf3c00) at
/root/doris-env/doris/be/src/olap/olap_server.cpp:354
#6 0x0000000000dd31ac in operator() (__closure=<optimized out>) at
/root/doris-env/doris/be/src/olap/olap_server.cpp:93
#7 std::_Function_handler<void(),
doris::StorageEngine::start_bg_threads()::<lambda()> >::_M_invoke(const
std::_Any_data &) (__functor=...) at
/usr/include/c++/7.3.0/bits/std_function.h:316
#8 0x0000000001117718 in operator() (this=0x6ce84a68) at
/usr/include/c++/7.3.0/bits/std_function.h:706
#9 doris::Thread::supervise_thread (arg=0x6ce84a50) at
/root/doris-env/doris/be/src/util/thread.cpp:304
#10 0x00007f4a21bcfdc5 in start_thread () from /lib64/libpthread.so.0
#11 0x00007f4a21edb73d in clone () from /lib64/libc.so.6
(gdb) f 4
#4 doris::CompactionPermitLimiter::request (this=this@entry=0x4cf3e50,
permits=permits@entry=2792) at
/root/doris-env/doris/be/src/olap/compaction_permit_limiter.cpp:30
30 /root/doris-env/doris/be/src/olap/compaction_permit_limiter.cpp:
没有那个文件或目录.
(gdb) p _used_permits
$1 = {_value = 0}
(gdb) p permits
$2 = 2792
(gdb) p config::total_permits_for_compaction_score
$3 = 2000
(gdb)
```
That means the wait condition is satisfied but the cv is still wait, so
there must be some multi-threads sync problems.
**To Reproduce**
Steps to reproduce the behavior:
Very occasional to happen.
**Expected behavior**
All threads works well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
