Hello, are you using pike.top rpc command? I noticed in the code that it uses system malloc, but I haven't investigated further yet, first to see if this would be a possibility ...
Cheers, Daniel On 06/10/16 16:33, Jurijs Ivolga wrote: > Hi Daniel, > > We do not do any external operations. > > We are using janson 2.7 everywhere. I will try to update to latest > janson version tomorrow. > All json operation is pretty much same, we are using only jansson_get. > > In attachment you can see memory consumption. On the right 2 servers > which are faced internet on the left which don't face internet. As you > can see memory consumption is pretty dramatic. > > Thank you for your help! > > With kind regards, > > Jurijs > > On Thu, Oct 6, 2016 at 5:17 PM, Daniel-Constantin Mierla > <mico...@gmail.com <mailto:mico...@gmail.com>> wrote: > > Hello, > > are you doing different external operations than on the other > instances, like mi/rpc commands. > > From the list of the modules you exposed, I think jansson has the > higher probability to work with system memory. Are you doing > different json operations in config that in the other instances of > kamailio? Are you using same version of libjansson everywhere? > > Cheers, > Daniel > > On 06/10/16 13:46, Jurijs Ivolga wrote: >> Hi Daniel, >> >> This modules what we are using: >> >> loadmodule "mi_fifo.so" >> loadmodule "kex.so" >> loadmodule "corex.so" >> loadmodule "tm.so" >> loadmodule "tmx.so" >> loadmodule "sl.so" >> loadmodule "rr.so" >> loadmodule "pv.so" >> loadmodule "maxfwd.so" >> loadmodule "textops.so" >> loadmodule "siputils.so" >> loadmodule "xlog.so" >> loadmodule "sanity.so" >> loadmodule "ctl.so" >> loadmodule "cfg_rpc.so" >> loadmodule "mi_rpc.so" >> loadmodule "dispatcher.so" >> loadmodule "utils.so" >> loadmodule "path.so" >> loadmodule "ipops.so" >> loadmodule "jansson.so" >> loadmodule "auth.so" >> loadmodule "nathelper.so" >> loadmodule "tls.so" >> loadmodule "htable.so" >> loadmodule "pike.so" >> >> We have several other Kamailio instances but they are not faced >> to internet and they do not have such memory issue. That other >> Kamailio instances have same modules, except modules listed >> below. So if you think that issue is inside external library, >> probably we need to check first modules from list below. >> >> loadmodule "ipops.so" >> loadmodule "auth.so" >> loadmodule "nathelper.so" >> loadmodule "pike.so" >> >> But maybe this other Kamailio instances do not have this memory >> issue, just because they did not face to internet and did not >> have same load as instances with memory issue. >> >> kamailio -v >> version: kamailio 4.4.3 (x86_64/linux) 5a2195 >> flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, >> USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, >> SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, >> DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, >> USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES >> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN >> 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB >> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. >> id: 5a2195 >> compiled on 08:30:51 Sep 15 2016 with gcc 4.4.7 >> >> >> With kind regards, >> >> >> Jurijs >> >> On Thu, Oct 6, 2016 at 12:52 PM, Daniel-Constantin Mierla >> <mico...@gmail.com <mailto:mico...@gmail.com>> wrote: >> >> Hello, >> >> it looks like a leak from the system memory, not from >> kamailio's pkg or shm memory. This can be due to an improper >> use of an external library (e.g., libxml2) by a kamailio >> module or because of a problem in the library. >> >> Can you list the modules used in your config (the loadmodule >> lines)? I will try to guess from the list which one relying >> on external libs with higher risk of leak issues. >> >> Also, provide the version of kamailio you are using (kamailio >> -v). >> >> Cheers, >> Daniel >> >> >> On 04/10/16 15:42, Jurijs Ivolga wrote: >>> Hi, >>> >>> Our Kamailio server is crashing once per week, with >>> following error: >>> >>> Oct 1 06:25:06 kamailio kernel: [26982632.803789] Out of >>> memory in UB 210: OOM killed process 12261 (kamailio) score >>> 0 vm:1614768kB, rss:280200kB, swap:131408kB >>> >>> Core dump was never created, probably it is because of my >>> environment, but I will try to get it. >>> >>> Server constantly eats memory, maybe some kind of memory leak? >>> >>> Any help is highly appreciated! >>> >>> Jurijs >>> >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing >>> list >>> sr-users@lists.sip-router.org >>> <mailto:sr-users@lists.sip-router.org> >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> <http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users> >> >> -- >> Daniel-Constantin Mierla >> http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - >> http://www.linkedin.com/in/miconda >> <http://www.linkedin.com/in/miconda> >> Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - >> http://www.asipto.com >> >> _______________________________________________ SIP Express >> Router (SER) and Kamailio (OpenSER) - sr-users mailing list >> sr-users@lists.sip-router.org >> <mailto:sr-users@lists.sip-router.org> >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >> <http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users> >> > -- > Daniel-Constantin Mierla > http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - > http://www.linkedin.com/in/miconda > <http://www.linkedin.com/in/miconda> > Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - > http://www.asipto.com > -- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Oct 24-26, 2016 - http://www.asipto.com
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users