Today was another attempt to upgrade to version 13.9 (git).
1. The result was https://issues.asterisk.org/jira/browse/ASTERISK-25970
Had to temporarily block this contact and look forward to advice of how
to fix it.
2. Also, an unpleasant surprise was the increase in CPU usage from
10-50% to 200-400% (4 cores). Stable CPU overusage with the same build
options (DONT_OPTIMIZE, DEBUG_THREADS, BETTER_BACKTRACES, BUILD_NATIVE)
3. After 20 minutes of this work the server has ceased to respond
promptly to registrations. "core show locks" is attached. Suggestions
about solving the problem are very welcome.
Dmitriy.
Setting max files open to 300000
=======================================================================
=== 13.9.0-rc1
=== Currently Held Locks
=======================================================================
===
=== <pending> <lock#> (<file>): <lock type> <line num> <function> <lock name>
<lock addr> (times locked)
===
=== Thread ID: 0x7fbaedb2c700 LWP:25598 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res_pjsip/pjsip_options.c:342 qualify_contact()
res_pjsip/pjsip_options.c:443 qualify_contact_task()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbaedab0700 LWP:25596 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res_pjsip/pjsip_options.c:342 qualify_contact()
res_pjsip/pjsip_options.c:443 qualify_contact_task()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbb25795700 LWP:25597 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res_pjsip/pjsip_options.c:342 qualify_contact()
res_pjsip/pjsip_options.c:443 qualify_contact_task()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+8CB8)
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbb254ad700 LWP:25895 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res_pjsip/pjsip_options.c:342 qualify_contact()
res_pjsip/pjsip_options.c:443 qualify_contact_task()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbb8abf4700 LWP:26536 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields()
(0x5bf860+FFFFFFFFFFFFFE63)
res/res_pjsip_endpoint_identifier_user.c:101 username_identify()
res/res_pjsip.c:2247 ast_sip_identify_endpoint() (0x7fbb2634d440+52)
res_pjsip/pjsip_distributor.c:384 endpoint_lookup()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbaec3ab700 LWP:26537 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res/res_pjsip.c:2247 ast_sip_identify_endpoint() (0x7fbb2634d440+52)
res_pjsip/pjsip_distributor.c:384 endpoint_lookup()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbaec693700 LWP:26535 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res_pjsip/pjsip_options.c:342 qualify_contact()
res_pjsip/pjsip_options.c:443 qualify_contact_task()
main/taskprocessor.c:852 ast_taskprocessor_execute()
(0x5e02d3+7FBB8E58394A)
res_pjsip/pjsip_distributor.c:461 distribute()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=== Thread ID: 0x7fbaeca73700 LWP:26534 (worker_start started at [
1077] threadpool.c worker_thread_start())
=== ---> Lock #0 (sorcery.c): RDLOCK 1883 ast_sorcery_retrieve_by_fields
&(&object_type->wizards)->lock 0x2243420 (1)
main/backtrace.c:59 __ast_bt_get_addresses() (0x466440+1D)
main/lock.c:866 __ast_rwlock_rdlock() (0x53652a+BA)
main/sorcery.c:1884 ast_sorcery_retrieve_by_fields() (0x5bf860+C2)
res_pjsip/pjsip_options.c:290 find_an_endpoint()
res_pjsip/pjsip_options.c:342 qualify_contact()
res_pjsip/pjsip_options.c:443 qualify_contact_task()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:1320 execute_tasks()
main/taskprocessor.c:852 ast_taskprocessor_execute() (0x5e02d3+10D)
main/threadpool.c:351 threadpool_execute()
main/threadpool.c:1103 worker_active()
main/threadpool.c:1024 worker_start()
main/utils.c:1235 dummy_start()
:0 start_thread()
:0 __clone() (0x7fbb8bb68860+6D)
=== -------------------------------------------------------------------
===
=======================================================================
--
_____________________________________________________________________
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
