Ok, i think i know why the numbers look strange: The output was at the shutdown, but a few seconds before i send the SIGUSR1...
this is the correct output: Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: Memory status (shm): Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: qm_status (0x7ff2182ea000): Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: heap size= 4294967296 Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: used= 88432160, used+overhead=95192920, free=4199774376 Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: max used (+overhead)= 119231640 Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: dumping summary of all alloc'ed. fragments: Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: ---------------------------------------------------- Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: total_bytes | num_allocations x [file: func, line] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: ---------------------------------------------------- Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 256 : 32 x [net/net_tcp.c: tcp_init, line 1728] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 32 : 1 x [dlist.c: new_dlist, line 901] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 14464 : 1 x [timer.c: tm_init_timers, line 533] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 16 : 1 x [dlist.c: new_dlist, line 909] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 38528 : 826 x [../../ut.h: shm_nt_str_dup, line 716] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 536 : 52 x [statistics.c: register_stat2, line 399] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 19840 : 1107 x [statistics.c: build_stat_name, line 122] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 32 : 1 x [dlg_timer.c: init_dlg_reinvite_ping_timer, line 185] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 256 : 1 x [lock.c: lock_initialize, line 88] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 131104 : 1 x [dlg_hash.c: init_dlg_table, line 136] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 3645840 : 31527 x [usr_avp.c: new_avp, line 117] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 17392 : 826 x [urecord.c: new_urecord, line 85] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 480 : 1 x [statistics.c: __add_stat_module, line 166] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 408 : 48 x [mi/mi.c: register_mi_cmd, line 174] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [timer.c: init_timer, line 81] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 5170344 : 3893 x [sip_msg.c: update_cloned_msg_from_msg, line 1190] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 32 : 1 x [dlg_timer.c: init_dlg_timer, line 55] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 98696 : 3893 x [t_fwd.c: add_phony_uac, line 507] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 16 : 1 x [t_hooks.c: init_tmcb_lists, line 64] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 1048576 : 1 x [hash.c: lcache_htable_init, line 50] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 208 : 12 x [statistics.c: __add_stat_module, line 182] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [dlg_timer.c: init_dlg_timer, line 64] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 128 : 2 x [ebr_data.c: add_ebr_event, line 79] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 168 : 14 x [map.c: map_get, line 150] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 85552 : 2165 x [map.c: map_create, line 79] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 5904 : 1 x [core_stats.c: init_pkg_stats, line 173] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 128 : 16 x [../../rw_locking.h: lock_init_rw, line 45] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 271632 : 826 x [ucontact.c: new_ucontact, line 105] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [timer.c: init_timer, line 82] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [usr_avp.c: init_extra_avps, line 83] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 480 : 6 x [statistics.c: register_stat2, line 385] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [net/net_tcp.c: tcp_init, line 1718] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 955024 : 3489 x [hash.c: lcache_htable_insert, line 126] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 24 : 1 x [rw_locking.h: lock_init_rw, line 40] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 32 : 1 x [dlg_timer.c: init_dlg_ping_timer, line 155] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [mem/shm_mem.c: shm_mem_init_mallocs, line 390] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 16384 : 1 x [udomain.c: new_udomain, line 88] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 80 : 10 x [evi/event_interface.c: evi_publish_event, line 75] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 3368 : 38 x [timer.c: new_os_timer, line 146] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 40 : 1 x [cachedb_local.c: parse_collections, line 608] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 136 : 1 x [event_route.c: fixup_scriptroute_fetch, line 564] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [usr_avp.c: init_extra_avps, line 74] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 984 : 1 x [core_stats.c: init_pkg_stats, line 174] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 4120 : 1 x [statistics.c: init_stats_collector, line 223] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 696 : 19 x [ucontact.c: mem_update_ucontact, line 250] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 565800 : 1 x [pt.c: init_multi_proc_support, line 70] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [timer.c: init_timer, line 83] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 400 : 1 x [evi/event_interface.c: evi_publish_event, line 61] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8208 : 1 x [../../evi/../lock_alloc.h: lock_set_alloc, line 66] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 79464 : 826 x [urecord.c: new_urecord, line 70] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 464 : 2 x [event_routing.c: ebr_parse, line 380] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 384 : 16 x [../../rw_locking.h: lock_init_rw, line 40] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 32 : 2 x [evi/evi_transport.c: register_event_mod, line 84] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [dlg_timer.c: init_dlg_ping_timer, line 162] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 262144 : 32 x [net/net_tcp.c: tcp_init, line 1741] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 24 : 1 x [ul_callback.c: init_ulcb_list, line 44] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 56 : 1 x [udomain.c: new_udomain, line 81] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 81792 : 1167 x [statistics.c: register_stat2, line 388] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 2253944 : 3893 x [mem/shm_mem.c: _shm_resize, line 226] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [rw_locking.h: lock_init_rw, line 45] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 16 : 1 x [daemonize.c: set_osips_state, line 576] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 40 : 1 x [t_hooks.c: insert_tmcb, line 92] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 2621448 : 1 x [h_table.c: init_hash_table, line 372] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 262144 : 32 x [net/net_tcp.c: tcp_init, line 1747] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 251864 : 3287 x [../../ut.h: shm_str_dup, line 692] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 33428816 : 3893 x [h_table.c: build_cell, line 244] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 72 : 3 x [evi/event_interface.c: evi_event_subscribe, line 334] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 176 : 1 x [event_route.c: scriptroute_parse, line 306] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 616 : 12 x [ucontact.c: mem_update_ucontact, line 255] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [dlg_timer.c: init_dlg_reinvite_ping_timer, line 192] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [sl_funcs.c: sl_startup, line 80] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 37022512 : 3893 x [sip_msg.c: sip_msg_cloner, line 534] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 1824 : 12 x [ucontact.c: mem_update_ucontact, line 271] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 55816 : 840 x [map.c: map_get, line 139] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 8 : 1 x [daemonize.c: create_status_pipe, line 92] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: 2064 : 1 x [../../lock_alloc.h: lock_set_alloc, line 66] Mar 10 23:00:03 ireg02 /usr/sbin/opensips[42351]: ---------------------------------------------------- ----- Ursprüngliche Mail ----- Von: "Liviu Chircu" <li...@opensips.org> An: "OpenSIPS users mailling list" <users@lists.opensips.org> Gesendet: Mittwoch, 11. März 2020 11:03:16 Betreff: Re: [OpenSIPS-Users] Debugging memory leaks On 11.03.2020 11:06, Fabian Gast wrote: > How can we continue from the memory status on hunting down the problems? Is > there any advice on this? Hey Fabian, When you ran the "shm_mem_dump" which produced the pasted output, what values did the "shmem:" statistics group hold? Based on the output, you were barely using 1 MB of shared memory, which is a bit strange. The table head tells exactly what the numbers represent: total bytes, number of allocations and the file/func/line which allocated them. Regards, -- Liviu Chircu www.twitter.com/liviuchircu | www.opensips-solutions.com OpenSIPS Summit, Amsterdam, May 2020 www.opensips.org/events _______________________________________________ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users _______________________________________________ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users