Hi Tsvetan, i've just pushed a safety check to master. Can you check, if this avoids the crash? If yes, can you share the output of the log? If I am correct, it's due to a miscalculation of available apps for the peer.
Thanks, Carsten 2018-01-16 9:49 GMT+01:00 Tsvetan Filev <tsvetan.fi...@inno-networks.com>: > Hi. > > I get a coredump which is caused by bad memory handling during the diameter > capability exchange process. > Here is part of the log file: > =================================== > 2018-01-16T09:16:39.890992+02:00 linux-o12d kamailio[13298]: 87(13388) INFO: > cdp [worker.c:332]: worker_process(): [0] Worker process started... > 2018-01-16T09:16:40.296393+02:00 linux-o12d systemd-coredump[13196]: Process > 13193 (kamailio) of user 479 dumped core. > 2018-01-16T09:16:40.871483+02:00 linux-o12d kamailio[13298]: 94(13395) > DEBUG: cdp [peermanager.c:263]: peer_timer(): peer_timer(): taking care of > peers... > 2018-01-16T09:16:40.871545+02:00 linux-o12d kamailio[13298]: 94(13395) > DEBUG: cdp [peermanager.c:280]: peer_timer(): peer_timer(): Peer > hss.epc.mnc019.mcc425.3gppnetwork.org State 0 > 2018-01-16T09:16:40.871596+02:00 linux-o12d kamailio[13298]: 94(13395) > DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer > hss.epc.mnc019.mcc425.3gppnetwork.org State Closed Event Start > 2018-01-16T09:16:40.871612+02:00 linux-o12d kamailio[13298]: 94(13395) INFO: > cdp [peerstatemachine.c:525]: I_Snd_Conn_Req(): I_Snd_Conn_Req(): Peer > hss.epc.mnc019.mcc425.3gppnetwork.org > 2018-01-16T09:16:40.871636+02:00 linux-o12d kamailio[13298]: 94(13395) INFO: > cdp [receiver.c:869]: peer_connect(): peer_connect(): Trying to connect to > 10.82.10.85 port 3868 > 2018-01-16T09:16:40.871782+02:00 linux-o12d kamailio[13298]: 94(13395) INFO: > cdp [receiver.c:937]: peer_connect(): peer_connect(): Peer > hss.epc.mnc019.mcc425.3gppnetwork.org:3868 connected > 2018-01-16T09:16:40.871813+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:697]: receive_loop(): select_recv(): There is > something on the fd exchange pipe > 2018-01-16T09:16:40.871828+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:706]: receive_loop(): select_recv(): fd exchange pipe > says fd [22] for peer 0x7f0626b91c98:[hss.epc.mnc019.mcc425.3gppnetwork.org] > 2018-01-16T09:16:40.871910+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer > hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_Conn_Ack Event > I_Rcv_Conn_Ack > 2018-01-16T09:16:40.871933+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [diameter_msg.c:184]: AAANewMessage(): AAANewMessage: param > session received null and it's a request!! > 2018-01-16T09:16:40.872011+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [diameter_msg.c:81]: AAABuildMsgBuffer(): AAABuildMsgBuffer(): > len=204 > 2018-01-16T09:16:40.872041+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:994]: peer_send_msg(): peer_send_msg(): Pipe push > [0x7f0626c02548] > 2018-01-16T09:16:40.872064+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:751]: receive_loop(): select_recv(): There is > something on the send pipe > 2018-01-16T09:16:40.872084+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:764]: receive_loop(): select_recv(): Send pipe says > [0x7f0626c02548] 8 > 2018-01-16T09:16:40.872104+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [diameter_msg.c:410]: AAAFreeMessage(): AAAFreeMessage: Freeing > message (0x7f0626c02548) 257 > 2018-01-16T09:16:40.872277+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:574]: do_receive(): receive_loop(): > [hss.epc.mnc019.mcc425.3gppnetwork.org] Recv Version 1 Length 360 > 2018-01-16T09:16:40.872322+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [receiver.c:1088]: receive_message(): receive_message(): > [hss.epc.mnc019.mcc425.3gppnetwork.org] Recv msg 257 > 2018-01-16T09:16:40.872345+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:90]: sm_process(): sm_process(): Peer > hss.epc.mnc019.mcc425.3gppnetwork.org State Wait_I_CEA Event I_Rcv_CEA > 2018-01-16T09:16:40.872372+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:692]: count_Supported_Vendor_Id_AVPS(): Found > 4 Supported_Vendor AVPS92(13393) DEBUG: cdp [peerstatemachine.c:743]: > save_peer_applications(): Found Supported Vendor for Application 0: 5535 > 2018-01-16T09:16:40.872389+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 10415 > 2018-01-16T09:16:40.872405+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 13019 > 2018-01-16T09:16:40.872420+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 58637 > 2018-01-16T09:16:40.872438+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 5535 > 2018-01-16T09:16:40.872453+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 10415 > 2018-01-16T09:16:40.872468+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 13019 > 2018-01-16T09:16:40.872486+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 58637 > 2018-01-16T09:16:40.872504+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 5535 > 2018-01-16T09:16:40.872523+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 10415 > 2018-01-16T09:16:40.872539+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 13019 > 2018-01-16T09:16:40.872554+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:743]: save_peer_applications(): Found > Supported Vendor for Application 0: 58637 > 2018-01-16T09:16:40.872570+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found > Supported Vendor for Application 1: 5535 > 2018-01-16T09:16:40.872586+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found > Supported Vendor for Application 1: 10415 > 2018-01-16T09:16:40.872601+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found > Supported Vendor for Application 1: 13019 > 2018-01-16T09:16:40.872616+02:00 linux-o12d kamailio[13298]: 92(13393) > DEBUG: cdp [peerstatemachine.c:756]: save_peer_applications(): Found > Supported Vendor for Application 1: 58637 > 2018-01-16T09:16:40.872634+02:00 linux-o12d kamailio[13298]: 92(13393) > CRITICAL: <core> [core/mem/q_malloc.c:145]: qm_debug_frag(): BUG: qm: prev. > fragm. tail overwritten(28af01000000, 0)[0x7f0626c038d0:0x7f0626c03908]! > Memory allocator was called from cdp: diameter_avp.c:365. Fragment marked by > cdp: diameter_avp.c:142. > 2018-01-16T09:16:41.054292+02:00 linux-o12d kamailio[13298]: 0(13298) ALERT: > <core> [main.c:746]: handle_sigs(): child process 13393 exited by a signal 6 > 2018-01-16T09:16:41.054403+02:00 linux-o12d kamailio[13298]: 0(13298) ALERT: > <core> [main.c:749]: handle_sigs(): core was generated > 2018-01-16T09:16:41.054422+02:00 linux-o12d kamailio[13298]: 0(13298) INFO: > <core> [main.c:771]: handle_sigs(): terminating due to SIGCHLD > 2018-01-16T09:16:41.054438+02:00 linux-o12d kamailio[13298]: 0(13298) DEBUG: > <core> [main.c:773]: handle_sigs(): terminating due to SIGCHLD > 2018-01-16T09:16:41.054464+02:00 linux-o12d kamailio[13298]: 2(13303) INFO: > <core> [main.c:826]: sig_usr(): signal 15 received > ... > =================================== > > Attached is wireshark trace. > Here is my DiameterPeer.xml: > > =================================== > <?xml version="1.0" encoding="UTF-8"?> > <DiameterPeer > FQDN="ims110-scscf.epc.mnc019.mcc425.3gppnetwork.org" > Realm="epc.mnc019.mcc425.3gppnetwork.org" > Vendor_Id="10415" > Product_Name="CDiameterPeer" > AcceptUnknownPeers="1" > DropUnknownOnDisconnect="1" > Tc="30" > Workers="4" > QueueLength="8" > TransactionTimeout="5" > SessionsHashSize="128" > DefaultAuthSessionTimeout="3600" > MaxAuthSessionTimeout="3600"> > > <Peer FQDN="hss.epc.mnc019.mcc425.3gppnetwork.org" > Realm="epc.mnc019.mcc425.3gppnetwork.org" port="3868" /> > <Acceptor port="3869" bind="10.82.10.56" /> > <Auth id="16777216" vendor="10415" /> <!--3GPP CxDX --> > <DefaultRoute FQDN="ims110-scscf.epc.mnc019.mcc425.3gppnetwork.org" > metric="10" /> > </DiameterPeer> > =================================== > > It looks like buffer overflow to me but I'm not sure. > > kamailio version is: > =================================== > kamailio -v > version: kamailio 5.1.0 (x86_64/linux) > flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, > USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, > TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, > USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, > HAVE_RESOLV_RES > ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, > MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB > poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. > id: unknown > compiled on 05:30:36 Jan 15 2018 with gcc 4.8.5 > =================================== > > OS is openSUSE Leap 42.3. > Kernel: Linux linux-o12d 4.4.104-39-default #1 SMP Thu Jan 4 08:11:03 UTC > 2018 (7db1912) x86_64 x86_64 x86_64 GNU/Linux > > Do you have any idea what might be wrong ? > It could be bad config but still it should say something in the log without > a crash. > > Regards. > > > _______________________________________________ > Kamailio (SER) - Users Mailing List > sr-users@lists.kamailio.org > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users > -- Carsten Bock CEO (Geschäftsführer) Meet us at MWC 2018 in Barcelona and see IMS and VoLTE in action! (February 26th - March 1st, Hall 7 - Booth 7L80) ng-voice GmbH Millerntorplatz 1 20359 Hamburg / Germany http://www.ng-voice.com mailto:cars...@ng-voice.com Office +49 40 5247593-40 Fax +49 40 5247593-99 Sitz der Gesellschaft: Hamburg Registergericht: Amtsgericht Hamburg, HRB 120189 Geschäftsführer: Carsten Bock Ust-ID: DE279344284 Hier finden Sie unsere handelsrechtlichen Pflichtangaben: http://www.ng-voice.com/imprint/ _______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
