Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-15 Thread Daniel-Constantin Mierla
I see that PTHREAD_MUTEX are set as default instead of POSIX, if fast locks are not used. Maybe someone itried to see when they work very long time ago and eventually it was left like this (I tried to track the change quickly, but couldn't find it easy, too many changes in the file). Pthread mut

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-15 Thread Daniel-Constantin Mierla
Closed #1021. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#event-1000680948___ sr-dev mailing list sr-dev@lists.sip-router.org http://lists.sip-route

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-15 Thread SubnetsRU
I can confirm that kamailio works perfectly without PTHREAD_MUTEX: ``` Version: kamailio 4.4.5 (x86_64/freebsd) f98162 Default config: /usr/local/etc/kamailio/kamailio.cfg Default paths to modules: /usr/local/lib/kamailio/modules Compile flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HO

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-15 Thread Daniel-Constantin Mierla
I cloned the git repository inside `/usr/local/src`, then compiled it with `gmake`. You can run `kamailio -I` to see some of the compile time flags. It is a difference between the one printed by `/usr/local/sbin/kamailio` and the one cloned by me in `/usr/local/src`. -- You are receiving this

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-15 Thread SubnetsRU
I've compiled kamailio using gmake, without any special options. I'll try to detect where was included using of PTHREAD_MUTEX -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-28666759

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-14 Thread Daniel-Constantin Mierla
How did you compile kamailio? It uses pthread mutexes, which should not happen, there is no proper synchronization done there, so crashing can happen at any time. I cloned kamailio and compiled it as usual and it uses the default FAST_LOCK for sync, not PTHREAD_MUTEX. -- You are receiving thi

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-14 Thread SubnetsRU
I've got core dump: it locates at /mnt/hdd3/coredumps/kamailio.20041.core -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-286429354___ sr-d

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-14 Thread SubnetsRU
No, but sometime it is hard to reproduce core dump... Yes, I can. I'll pass calls now. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-286426640___

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-14 Thread Daniel-Constantin Mierla
I connected to the box and started kamailio with `-x qm` -- it works, you said that it crashes immediately: ``` [VM-TEST@sip7:/home/daniel] ps auxw | grep kamai root 19951 0.0 1.8 222420 37196 - S 4:24PM0:00.05 /usr/local/sbin/kamailio -x qm -a no -E -e -ddd root 19952 0.

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-13 Thread SubnetsRU
It's ready for you. Send me your public key to po...@subnets.ru, please. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-286169780___ sr-de

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-13 Thread SubnetsRU
Ok, I'll write when it will be ready. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-286130837___ sr-dev mailing list sr-dev@lists.sip-ro

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-13 Thread Daniel-Constantin Mierla
Probably having access to a VM where I can reporduce myself is the best for analyzing. Let me know when it is available, I can provide a ssh key for getting access on it (I will also need sudo privileges). As I am not that familiar with freebsd, be sure that vim and gdb are installed. -- You a

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread SubnetsRU
Sorry, I've updated my previous post. I can create virtualbox image for you, Daniel, and provide ssh access to it, if it can helps you. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomme

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread Daniel-Constantin Mierla
... and the backtraces? -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-285699983___ sr-dev mailing list sr-dev@lists.sip-router.org http:/

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread SubnetsRU
Done: ``` ps ax | grep kama 27526 - S 0:00,05 /usr/local/sbin/kamailio -x qm 27527 - S 0:00,19 /usr/local/sbin/kamailio -x qm ... ``` And now I've got five core files without load... -- You are receiving this because you commented. Reply to this email directly or view it on Gi

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread Daniel-Constantin Mierla
Can you try running kamailio with `-x qm` command parameters to see if it is specific to fm memory manager? -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-285693019_

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread SubnetsRU
Yes, it is. But now it seems that they crash at the same point in code. Here is bt (only 4 core, not 5): http://tmp.lehis.ru/tmp/kam_bt_27333.txt http://tmp.lehis.ru/tmp/kam_bt_27334.txt http://tmp.lehis.ru/tmp/kam_bt_27336.txt http://tmp.lehis.ru/tmp/kam_bt_27337.txt -- You are receiving this b

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread Daniel-Constantin Mierla
If you compile with clang directly in the source tree, without using the ports, is it still crashing? Looking at the backtraces, the SIGBUS can happen only if there is a strict alignment requirement of 8 bytes, which is probably not ensured for all members of structures. There is typically an a

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-10 Thread Phil Lavin
Behalf Of Daniel-Constantin Mierla Sent: 09 March 2017 12:02 To: Kamailio Devel List Cc: Subscribed Subject: Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021) I don't know about *BSD, but in Linux there is an option to enable corefiles per process (per pid). It is something

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Daniel-Constantin Mierla
In the bt file, is `list` output taken for the same corefile in the frame 0? So 10 is `SIGBUS`, which typically appears due to unaligned memory access, but can be other reasons: * https://en.wikipedia.org/wiki/Bus_error Can you check on all core files that you got after enabling it per pid to

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread pi
FreeBSD 11.0-RELEASE-p2 FreeBSD 11.0-RELEASE-p2 #0: Mon Oct 24 06:55:27 UTC 2016 $ kill -l 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGEMT 8) SIGFPE 9) SIGKILL 10) SIGBUS 11) SIGSEGV 12) SIGSYS 13) SIGPIPE 14) SIGAL

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Daniel-Constantin Mierla
The crash happens due to `signlal 10` -- can you get the output of `kill -l` to see what is that on your system? It might be specific to the OS, like SIGUSR1 or SIGBUS... -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamai

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread SubnetsRU
Daniel, I can't reproduce core dump with gcc - it works perfectly. I've turned on debug symbols with clang: after last core dump I've five core files (for all childs?). Here is last logs: http://tmp.lehis.ru/tmp/kam_log_2.txt Here is bt of PID 82699: http://tmp.lehis.ru/tmp/kam_bt_2.txt -- You

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread SubnetsRU
I've built kamailio with gcc: > version: kamailio 4.4.5 (x86_64/freebsd) f98162 > flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, > USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, > TLSF_MALLOC, DBG_SR_MEMORY, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_C

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread SubnetsRU
I've compiled kamailio via ports. Well, I'll try to build it directly from sources with gcc. I'll be back :) -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-285359479_

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Daniel-Constantin Mierla
Proper gdb backtrace with debugging symbols is needed here. Do you compile from sources directly, or do you do it via ports/something else? Can you try with gcc instead if clang? Debugging symbols should be compiled by default, at least with gcc, as I haven't contributed the clang part, I can't

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread SubnetsRU
It's here: http://tmp.lehis.ru/tmp/kam_bt.txt -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-285343372___ sr-dev mailing list sr-dev@lists

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Daniel-Constantin Mierla
What is the gdb backtrace for the new core? -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1021#issuecomment-285342788___ sr-dev mailing list sr-dev@lists.s

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread SubnetsRU
Yes, I've got it. In FreeBSD corefiles per PID can sets via sysctl: sysctl kern.corefile=/mnt/coredumps/%N.%P.core, where are %N - name of process, %P - number of pid I've run kamailio again and got only one core, named as: ``` # ls -1 /mnt/coredumps/ kamailio.84671.core # ``` And logs says that

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Phil Lavin
I also know little of BSD, but this is what was required on Linux to get a dump into /tmp: ```phil@ua-proxy-01:/etc/sysctl.d$ cat 61-core-pattern.conf fs.suid_dumpable=2 kernel.core_uses_pid=1 kernel.core_pattern=/tmp/core.%e.%p.%h.%t``` -- You are receiving this because you commented. Reply to

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Daniel-Constantin Mierla
I don't know about *BSD, but in Linux there is an option to enable corefiles per process (per pid). It is something like: ``` echo "1" > /proc/sys/kernel/core_uses_pid ``` Can you see if there is something similar for *BSD, enable it and test again? -- You are receiving this because you are su

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread SubnetsRU
I've not use any shared memory parameters during kamailio starts. kamcmd shows: ``` kamcmd> core.shmmem { total: 67108864 free: 61856936 used: 5164808 real_used: 5251928 max_used: 5258528 fragments: 10 } ``` Yes, kamailio stops by itself. I'm sure t

Re: [sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-09 Thread Daniel-Constantin Mierla
The core is generated by the shutdown procedure, so not a runtime event, as side effect of running out of memory when building the transaction -- maybe some of the fields might be inconsistent. It needs to be fixed, but it is a bit hard to track it without proper core file and debugging symbols.

[sr-dev] [kamailio/kamailio] Kamailio core dumped (#1021)

2017-03-07 Thread SubnetsRU
### Description I've kamailio 4.2.3 (i386/freebsd) which works perfectly with 200-300 cps for years. Now I need migrate to freebsd amd64. I've installed kamailio 4.4.5 (amd64/freebsd), copied kamailio.cfg, made a test call - it was all right. But when I start work load (200-300 cps), kamailio cor